-- Logs begin at Sun 2022-08-28 00:10:21 UTC, end at Sun 2022-08-28 01:52:08 UTC. -- Aug 28 00:46:21.504999 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Aug 28 00:46:21.514346 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Aug 28 00:46:21.516355 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Aug 28 00:46:21 2022] *** Aug 28 00:46:21.516355 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 28 00:46:21.516355 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 28 00:46:21.516355 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: nodename: dv-cinder-master-3-iscsi Aug 28 00:46:21.516355 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: machine: x86_64 Aug 28 00:46:21.516355 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: clock source: unix Aug 28 00:46:21.516355 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: pcre jit disabled Aug 28 00:46:21.516355 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: detected number of CPU cores: 4 Aug 28 00:46:21.516355 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: current working directory: / Aug 28 00:46:21.516355 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: detected binary path: /usr/bin/uwsgi-core Aug 28 00:46:21.516355 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: your processes number limit is 47680 Aug 28 00:46:21.516355 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: your memory page size is 4096 bytes Aug 28 00:46:21.516355 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: detected max file descriptor number: 2048 Aug 28 00:46:21.516355 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: lock engine: pthread robust mutexes Aug 28 00:46:21.517542 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: thunder lock: enabled Aug 28 00:46:21.517542 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Aug 28 00:46:21.517542 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 28 00:46:21.556757 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: Python main interpreter initialized at 0x560cdc3f99c0 Aug 28 00:46:21.556757 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: python threads support enabled Aug 28 00:46:21.556757 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: your server socket listen backlog is limited to 100 connections Aug 28 00:46:21.556757 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: your mercy for graceful operations on workers is 90 seconds Aug 28 00:46:21.557246 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: mapped 403077 bytes (393 KB) for 2 cores Aug 28 00:46:21.557246 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: *** Operational MODE: preforking *** Aug 28 00:46:21.557548 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: *** uWSGI is running in multiple interpreter mode *** Aug 28 00:46:21.557548 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: spawned uWSGI master process (pid: 105522) Aug 28 00:46:21.557781 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Aug 28 00:46:21.558594 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: spawned uWSGI worker 1 (pid: 105533, cores: 1) Aug 28 00:46:21.558719 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: spawned uWSGI worker 2 (pid: 105534, cores: 1) Aug 28 00:46:21.559033 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 28 00:46:22.227307 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 28 00:46:22.269372 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 28 00:46:23.321632 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 28 00:46:23.321632 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: from cryptography.utils import int_from_bytes Aug 28 00:46:23.322378 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 28 00:46:23.322378 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: from cryptography.utils import int_from_bytes Aug 28 00:46:23.323925 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 28 00:46:23.323925 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: from cryptography.utils import int_from_bytes Aug 28 00:46:23.324470 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 28 00:46:23.324470 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: from cryptography.utils import int_from_bytes Aug 28 00:46:25.846012 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 28 00:46:25.846692 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 28 00:46:25.846799 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 28 00:46:25.847158 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 28 00:46:25.847424 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 28 00:46:25.847679 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 28 00:46:25.848180 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.848414 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 28 00:46:25.848507 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.848587 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 28 00:46:25.848667 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.848932 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 28 00:46:25.849138 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.849301 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 28 00:46:25.849443 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.849574 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.849742 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.849885 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.850031 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.850185 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.850350 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.850471 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.850587 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.850722 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.850845 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.851033 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.851145 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.851357 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.851473 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.851679 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.851785 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.851982 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.852072 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.852254 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.852365 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.852565 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.852666 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.852883 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.853013 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.853227 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.853331 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.853532 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.853629 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.853835 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.854028 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.854146 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.854445 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.854615 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.854744 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.854871 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.855134 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.855400 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.855400 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.855620 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.855710 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.855852 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.855972 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.856130 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.856295 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.856417 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.856612 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.856754 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.856929 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.857093 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.857281 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.857391 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.857677 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.857791 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.857987 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.858095 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.858296 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.858451 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.858604 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.858702 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.859019 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.859170 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.859313 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.859401 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.859621 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.859734 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.859926 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.860061 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.860263 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.860360 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.860577 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.860704 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.860886 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.861075 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.861238 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.861419 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.861608 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.861735 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.861854 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.861941 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.862145 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.862249 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.862455 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.862600 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.862750 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.862895 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.863065 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.863191 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.863364 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.863514 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.863672 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.863809 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.863995 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.864192 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.864342 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.864501 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.864640 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.864787 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.864925 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.865106 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.865257 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.865383 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.865594 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.865734 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.865734 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.865977 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.866128 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.866273 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.866413 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.866566 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.866713 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.866879 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.866879 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.867169 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.867313 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.867450 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.867609 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.867749 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.867908 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.868074 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.868219 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.868341 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.868341 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.868548 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.868690 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.868830 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.869007 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.869165 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.869314 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.869468 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.869622 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.869763 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.869912 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.870250 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.870420 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.870571 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.870571 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.870828 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.42 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.870828 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.42 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.871147 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.871305 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.871462 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.871662 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.871819 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.871971 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.872131 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.42 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.872281 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.872424 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.42 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.872563 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.872702 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.872842 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.873052 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.873211 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.873350 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.873499 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.873642 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.873781 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.873900 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.874011 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.874092 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.874178 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.874317 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.874425 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.874579 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.874720 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.874872 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.875015 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.875156 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.875310 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.875452 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.875672 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.875820 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.875957 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.876109 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.876248 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.876392 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.876533 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.876675 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.876813 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.876953 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.877151 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.877291 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.877434 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.877576 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.877717 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.877858 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.878017 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.878161 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.878300 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.878400 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.878532 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.878678 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.878838 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.878981 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.879126 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.879265 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.879607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.879768 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.879768 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.879977 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.880121 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.880288 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.880433 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.880704 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.880863 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.881236 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.881389 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.881520 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.881685 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.881847 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.881991 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.882141 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.882284 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.882428 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.882586 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.882743 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.882900 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.883056 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.883193 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.883329 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.883472 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.883638 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.883784 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.883927 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.884075 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.884217 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.884363 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.884507 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.884594 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.884726 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.884865 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.885031 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.885202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.885370 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.885515 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.885689 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.885828 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.885969 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.886118 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.886252 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.886409 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.886569 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.886725 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.886868 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.887026 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.887187 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.887331 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.887491 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.887637 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.887792 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.887938 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.888070 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.888221 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.888381 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.888526 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.888669 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.888823 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.889045 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.889169 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.889272 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.889362 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.889459 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.889669 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.889825 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.889969 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.890124 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.890267 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.890414 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.890569 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.890712 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.890858 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.890996 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.891160 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:46:25.891389 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.891566 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.891727 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.891922 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.892057 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.892195 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.892355 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.892510 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.892656 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.892799 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.892947 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.893210 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.893372 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.893531 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.893663 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.893860 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.894020 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.894181 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.894319 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.894464 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.894626 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.894785 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.894938 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.895090 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.895299 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.895454 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.895637 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.895796 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.895944 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.896098 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.896244 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.896382 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.896521 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.896657 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.896795 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.896948 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.897140 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.897278 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.897418 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.897557 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.897698 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.897860 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.898011 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.898158 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.898313 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.898457 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.898616 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.898774 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.898918 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.899076 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.899220 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.899362 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.899526 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.899667 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.899885 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.900035 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.900187 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.900337 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.900496 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.900639 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.900796 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.900941 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.901162 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.901329 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.901598 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.901726 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.901850 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.901987 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.902108 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.902266 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.902411 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.902575 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.902720 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.902870 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.903011 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.903169 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.903312 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.903457 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.903604 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.903743 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.903743 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.904029 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.904167 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.904306 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.904445 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.904599 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.904762 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.904922 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.905147 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.905287 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.905424 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.905564 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.905700 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.905857 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.905999 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.906149 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.906269 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.906403 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.906569 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.906710 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.906853 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.906995 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.907147 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.907298 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.907450 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.907602 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.907746 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.907890 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.907985 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.908054 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.908141 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.908210 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.908283 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.908422 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.908575 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.908717 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.908872 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.909039 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.909182 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.909321 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.909458 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.909616 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.909758 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.909911 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.910110 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.910281 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.910424 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.910579 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.910721 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.910893 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.911034 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.911176 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.911318 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.911492 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.911649 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.911797 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.911937 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.912065 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.912205 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.912417 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.912578 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.912735 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.912887 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.913134 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.913302 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.913445 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.913582 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.913724 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.913857 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.913998 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.914146 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.914292 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.914433 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.914584 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.914725 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.914873 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.915204 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.915352 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.915498 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.915642 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.915796 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.915943 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.916103 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.916245 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.916385 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.916481 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.916570 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.916660 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.916753 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.916905 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.917073 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.917232 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.917373 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.917528 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.917681 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.917836 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.917986 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.918126 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.918268 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.918420 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.918576 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.918731 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.918868 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.919011 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.919159 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.919328 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.919482 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.919604 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.919716 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.919801 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.919885 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.920008 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.920166 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.920387 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.920544 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.920672 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.920828 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.921015 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.921161 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.921317 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.921477 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.921627 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.921776 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.921975 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.922176 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.922318 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.922517 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.922654 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.922807 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.922958 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.923104 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.923104 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.923371 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.923530 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.923670 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.923670 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.923928 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.924068 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.924206 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.924367 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.924513 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.924667 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.924809 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.924954 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.925122 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.925277 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.925421 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.925545 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.925642 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.925642 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.925816 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.925928 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.926020 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.926097 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.926164 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.926229 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.926316 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.926512 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.926604 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.926826 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.926929 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.927145 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.927241 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.927451 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.927575 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.927768 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.927909 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.928048 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.928287 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.928379 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.928448 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.928692 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.928781 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.929054 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.929165 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.929365 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.929453 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.929680 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.929783 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.929989 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.930086 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.930296 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.930409 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.930603 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.930693 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.930910 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.930997 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.931212 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.931307 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.931519 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.931607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.931833 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.931920 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.932156 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.932255 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.932444 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.932569 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.932749 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.932840 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.933093 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.933181 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.933399 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.933485 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.933714 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.933808 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.934016 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.934103 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.934327 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.934414 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.934655 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.934744 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.934963 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.935056 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.935279 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.935387 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.935637 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.935734 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.935884 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.935971 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.936187 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.936287 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.936492 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.936585 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.936796 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.936883 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.937125 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.937230 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.937431 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.937518 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.937740 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.937826 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.938050 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.938137 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.938352 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.938437 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.938656 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.938743 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.938958 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.939046 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.939259 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.939359 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.939562 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.939655 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.939862 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.939950 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.940163 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.940249 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.940470 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.940595 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.940790 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.940879 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.941116 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.941206 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.941431 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.941518 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.941738 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.941826 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.942066 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.942216 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.942381 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.942582 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.942753 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.942905 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.943045 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.943184 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.943330 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.943479 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.943606 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.943733 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.943854 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.943989 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.944178 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.944337 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.944487 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.944587 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.944789 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.944889 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.945138 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.945320 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.945473 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.945624 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.945765 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.945901 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.946052 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.946191 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.946358 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.946508 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.946643 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.946778 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.946932 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.947064 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.947295 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.947476 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.947614 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.947755 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.947906 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.948088 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.948283 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.948526 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.948704 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.948847 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.949079 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.949169 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.949238 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.949304 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.949368 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.949447 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.949649 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.949772 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.949963 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.950110 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.950257 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.950415 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.950545 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.951158 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.951261 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.951501 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.951733 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.951892 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.952058 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.952186 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.952370 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.952513 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.952677 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.952862 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.953071 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.953231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.953387 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.953651 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.953816 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.953958 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.954098 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.954245 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.954387 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.954527 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.954677 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.954816 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.954966 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.955116 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.955253 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.955389 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.955525 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.955659 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.955793 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.955945 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.956084 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.956235 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.956375 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.956525 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.956703 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.956855 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.957029 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.957161 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.957287 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.957405 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.957551 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.957678 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.957853 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.957978 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.958210 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.958458 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.958609 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.958773 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.958919 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.959052 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.959183 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.959315 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.959458 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.959588 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.959732 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.959863 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.959993 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.960138 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.960275 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.960414 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.960553 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.960688 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.960827 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.961020 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.961267 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.961380 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.961466 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.961582 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.961779 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.961926 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.962106 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.962233 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.962455 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.962628 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.962806 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.962970 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.963116 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.963261 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.963394 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.963525 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.963666 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.963847 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.963989 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.964135 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.964280 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.964411 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.964560 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.964710 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.964849 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.965057 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.965202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.965354 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.965513 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.965657 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.965809 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.965939 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.966080 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.966204 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.966345 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.966472 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.966596 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.966719 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.966840 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.966986 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.967112 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.967274 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.967404 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.967578 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.967712 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.967914 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.968042 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.968171 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.968297 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.968473 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.968607 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.968782 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.968925 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.969119 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.969256 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.969423 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.969553 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.969740 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.969875 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.970060 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.970204 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.970332 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.970494 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.970630 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.970748 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.971048 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.971189 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.971189 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.971425 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.971554 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.971675 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.971812 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.971935 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.972101 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.972248 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.972417 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.972552 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.972717 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.972850 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.973032 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.973203 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.973375 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.973501 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.974135 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.974290 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.974290 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.974569 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.974714 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.974858 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.974999 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.975147 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.975312 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.975449 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.975577 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.975700 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.975824 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.975975 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.976131 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.976272 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.976413 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.976551 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.976694 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.976838 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.977071 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.977209 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.977353 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.977494 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.977649 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.977797 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.977920 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.978058 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.978211 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.978353 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.978493 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.978629 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.978754 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.978887 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.979021 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.979154 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.979295 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.979450 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.979589 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.979715 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.979851 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.980005 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.980155 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.980424 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.980622 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.980784 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.980939 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.981148 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.981318 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.981461 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.981612 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.981767 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.981921 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.982063 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.982217 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.982358 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.982508 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.982650 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.982791 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.982932 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.983079 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.983222 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.983364 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.983562 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.983742 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.983897 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.984042 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.984172 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.984297 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.984427 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.984567 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.984715 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.984856 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.985078 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.985209 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.985357 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.985514 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.985670 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.985807 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.985955 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.986099 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.986242 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.986373 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.986514 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.986658 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.986797 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.986929 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.987058 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.987200 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.987354 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.987490 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.987618 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.987772 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.987913 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.988051 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.988213 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.988356 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.988500 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.988635 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.988770 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.988916 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.989111 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.989248 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.989401 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.989542 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.989542 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.989833 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.989978 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.990118 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.990260 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.990401 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.990536 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.990689 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.990828 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.990970 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.991111 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.991250 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.991405 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.991405 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.991666 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.991795 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.991920 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.992090 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.992332 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.992474 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.992609 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.992752 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.992892 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.993149 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.993329 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.993466 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.993656 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.993832 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.994270 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.994404 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.994563 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.994711 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.994913 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.995051 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.995188 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.995327 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.995482 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.995482 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.995743 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.995743 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.995999 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.996142 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.996295 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.996425 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.996566 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.996722 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.996848 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.997050 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.997189 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.997311 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.997422 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.997528 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.997642 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.997857 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.998007 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.998142 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.998272 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.998453 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.998611 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.998766 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.998908 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.999048 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.999197 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.999340 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.999493 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.999638 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:25.999785 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.000062 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.000223 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.000364 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.000523 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.000668 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.000803 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.001090 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.001239 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.001404 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.001562 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.001696 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.001847 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.002023 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.002165 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.002344 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.002492 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.002638 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.002793 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.002924 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.003053 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.003243 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.003392 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.003530 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.003681 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.003871 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.004056 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.004237 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.004379 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.004567 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.004709 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.004859 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.005019 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.005159 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.005295 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.005448 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.005599 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.005736 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.005869 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.006007 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.006142 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.006292 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.006427 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.006711 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.006867 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.007033 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.007184 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.007319 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.007453 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.007647 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.007790 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.007969 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.008132 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.008280 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.008415 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.008560 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.008693 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.008864 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.009197 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.009336 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.009470 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.009605 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.009745 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.009878 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.010027 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.010164 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.010308 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.010450 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.010598 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.010757 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.010897 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.011035 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.011170 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.011307 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.011444 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.011602 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.011743 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.011881 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.012019 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.012157 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.012308 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.012458 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.012596 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.012727 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.012865 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.013069 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.013218 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.013378 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.013515 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.013680 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.013816 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.014023 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.014179 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.014322 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.014463 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.014601 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.014737 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.014880 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.015028 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.015169 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.015298 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.015491 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.015637 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.015775 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.015926 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.016101 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.016252 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.016432 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.016580 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.016763 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.016914 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.017132 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.017266 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.017400 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.017533 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.017736 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.017875 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.018053 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.018191 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.018377 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.018514 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.018726 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.018867 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.019005 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.019150 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.019287 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.019425 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.019581 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.019713 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.019841 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.019976 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.020165 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.020334 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.020517 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.020664 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.020819 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.021013 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.021161 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.021319 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.021463 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.021617 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.021762 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.021897 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.022037 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.022200 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.022351 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.022488 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.022640 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.022783 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.022924 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.023068 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.023201 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.023344 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.023487 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.023630 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.023764 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.023907 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.024099 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.024300 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.024432 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.024623 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.024783 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.024921 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.025078 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.025215 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.025347 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.025483 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.025755 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.025893 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.026013 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.026145 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.026278 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.026407 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.026557 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.026696 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.026840 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.026995 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.027136 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.027273 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.027424 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.027568 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.027710 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.027859 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.027996 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.028151 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.028291 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.028430 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.028568 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.028716 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.028954 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.029154 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.029484 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.029632 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.029803 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.029942 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.030146 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.030343 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.030473 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.030597 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.030720 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.030856 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.031030 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.031188 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.031323 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.031454 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.031636 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.031781 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.031923 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.032056 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.032238 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.032386 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.032529 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.032663 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.032916 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.033107 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.033251 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.033392 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.033533 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.033675 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.033833 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.033977 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.034118 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.034235 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.034425 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.034639 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.034787 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.034929 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.035069 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.035223 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.035368 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.035522 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.035678 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.035847 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.035993 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.036146 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.036299 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.036427 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.036552 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.036690 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.036884 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.037049 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.037232 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.037389 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.037533 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.037677 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.037821 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.037988 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.038126 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.038275 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.038418 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.038568 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.038701 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.038836 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.038968 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.039120 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.039262 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.039403 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.039578 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.039745 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.039891 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.040027 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.040193 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.040320 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.040504 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.040635 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.040820 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.041045 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.041238 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.041381 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.041679 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.041824 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.042010 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.042162 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.042332 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.042471 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.042652 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.042793 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.042970 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.043121 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.043263 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.043406 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.043594 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.043744 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.043896 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.044036 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.044178 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.044400 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.044546 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.044738 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.044940 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.045120 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.045263 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.045434 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.045577 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.045717 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.045868 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.046006 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.046157 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.046317 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.046453 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.046594 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.046733 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.046875 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.046875 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.047257 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.047483 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.047631 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.047768 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.047923 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.048080 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.048223 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.048358 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.048497 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.048646 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.048789 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.048945 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.049210 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.049355 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.049498 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.049635 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.049768 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.049931 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.050075 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.050226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.050372 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.050515 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.050668 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.050826 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.050964 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.051101 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.051244 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.051418 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.051607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.051752 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.051906 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.052041 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.052178 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.052319 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.052473 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.052611 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.052768 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.052909 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.053071 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.053205 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.053363 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.053501 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.053630 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.053765 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.053918 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.054071 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.054228 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.054372 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.054497 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.054694 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.054902 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.055052 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.055190 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.055326 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.055471 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.055606 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.055744 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.055904 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.056044 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.056181 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.056323 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.056465 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.056602 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.056759 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.056898 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.057075 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.057213 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.057352 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.057512 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.057667 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.057816 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.057961 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.058112 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.058252 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.058399 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.058534 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.058668 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.058801 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.058957 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.059099 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.059268 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.059403 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.059546 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.059686 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.059829 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.059968 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.060124 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.060272 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.060417 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.060551 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.060687 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.060895 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.061194 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.061336 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.061470 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.062193 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.062346 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.062497 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.062634 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.062772 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.062907 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.063043 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.063229 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.063384 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.063528 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.063669 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.063811 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.063951 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.064091 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.064240 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.064378 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.064567 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.064758 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.065017 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.065227 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.065381 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.065500 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.065620 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.065738 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.065854 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.065982 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.066098 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.066223 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.066356 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.066496 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.066624 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.066767 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.066910 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.067030 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.067140 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.067254 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.067338 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.067430 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.067533 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.067613 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.067691 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.067767 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.067925 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.068069 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.068275 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.068419 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.068593 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.068738 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.068889 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.069078 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.069231 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.069369 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.069521 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.069655 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.069825 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.069974 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.070144 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.070290 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.070428 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.070586 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.070740 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.070878 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.071022 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.071164 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.071311 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.071468 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.071618 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.071760 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.071902 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.072043 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.072292 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.072448 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.072586 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.072726 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.072896 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.073142 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.073313 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.073436 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.073609 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.073761 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.073920 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.074070 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.074225 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.074370 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.074507 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.074726 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.074886 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.075076 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.075281 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.075425 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.075561 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.076000 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.076157 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.076312 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.076467 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.076609 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.076746 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.076876 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.077026 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.077173 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.077306 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.077430 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.077571 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.077698 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.077879 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.078058 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.078202 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.078342 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.078478 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.078607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.078782 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.078940 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.079081 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.079216 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.079358 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.079500 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.079641 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.079830 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.079971 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.080096 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.080227 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.080367 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.080513 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.080653 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.080790 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.080929 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.081142 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.081288 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.081447 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.081649 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.081793 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.081947 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.082097 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.082258 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.082400 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.082540 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.082739 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.082894 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.083039 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.083202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.083202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.083526 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.083666 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.083807 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.083965 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.084121 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.084273 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.084413 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.084558 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.084712 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.084712 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.085021 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.085219 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.085410 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.085577 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.085722 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.085886 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.086039 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.086175 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.086310 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.086503 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.086650 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.086650 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.086920 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.086920 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.087177 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.087312 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.087449 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.087659 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.087802 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.087802 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.088082 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.088082 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.088358 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.088510 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.088646 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.088782 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.088928 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.088928 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.089341 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.089341 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.089602 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.089602 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.089866 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.090005 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.090155 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.090155 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.090408 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.090408 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.090664 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.090803 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.090955 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.090955 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.091218 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.091356 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.091498 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.091651 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.091788 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.091934 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.092084 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.092220 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.092349 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.092496 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.092632 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.092764 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.092902 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.093090 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.093229 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.093384 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.093520 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.093691 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.093842 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.093985 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.094220 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.094359 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.094499 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.094635 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.094771 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.094944 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.095098 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.095235 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.095356 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.095547 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.095688 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.095824 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.095975 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.096176 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.096323 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.096459 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.096593 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.096713 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.096863 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.097090 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.097235 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.097373 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.097508 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.097655 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.097791 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.097940 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.098078 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.098335 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.098482 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.098663 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.098818 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.098955 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.099166 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.099310 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.099473 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.099627 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:46:26.099761 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105534) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 28 00:46:26.099908 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105533) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 28 00:46:26.256284 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO dbcounter [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] Registered counter for database nova_api Aug 28 00:46:26.256679 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO dbcounter [None req-2603b5c2-33db-4946-867e-ae50bb870ac9 None None] Registered counter for database nova_api Aug 28 00:46:26.279707 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_db.sqlalchemy.engines [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105533) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 28 00:46:26.284173 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG oslo_db.sqlalchemy.engines [None req-2603b5c2-33db-4946-867e-ae50bb870ac9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105534) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 28 00:46:26.286938 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG dbcounter [-] [105533] Writer thread running {{(pid=105533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 28 00:46:26.291952 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG dbcounter [-] [105534] Writer thread running {{(pid=105534) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 28 00:46:26.343290 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.context [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),0cad982d-83ea-40cf-82f0-5c4394803725(cell1) {{(pid=105533) load_cells /opt/stack/nova/nova/context.py:464}} Aug 28 00:46:26.347265 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:46:26.347670 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:46:26.349573 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:46:26.350346 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:46:26.350781 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:46:26.352793 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:46:26.354119 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.context [None req-2603b5c2-33db-4946-867e-ae50bb870ac9 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),0cad982d-83ea-40cf-82f0-5c4394803725(cell1) {{(pid=105534) load_cells /opt/stack/nova/nova/context.py:464}} Aug 28 00:46:26.354860 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO dbcounter [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] Registered counter for database nova_cell0 Aug 28 00:46:26.358102 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG oslo_concurrency.lockutils [None req-2603b5c2-33db-4946-867e-ae50bb870ac9 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105534) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:46:26.358586 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG oslo_concurrency.lockutils [None req-2603b5c2-33db-4946-867e-ae50bb870ac9 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105534) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:46:26.360168 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG oslo_concurrency.lockutils [None req-2603b5c2-33db-4946-867e-ae50bb870ac9 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105534) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:46:26.360556 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO dbcounter [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] Registered counter for database nova_cell1 Aug 28 00:46:26.360917 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG oslo_concurrency.lockutils [None req-2603b5c2-33db-4946-867e-ae50bb870ac9 None None] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105534) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:46:26.361395 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG oslo_concurrency.lockutils [None req-2603b5c2-33db-4946-867e-ae50bb870ac9 None None] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105534) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:46:26.363419 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG oslo_concurrency.lockutils [None req-2603b5c2-33db-4946-867e-ae50bb870ac9 None None] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105534) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:46:26.365565 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO dbcounter [None req-2603b5c2-33db-4946-867e-ae50bb870ac9 None None] Registered counter for database nova_cell0 Aug 28 00:46:26.368402 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_db.sqlalchemy.engines [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105533) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 28 00:46:26.370038 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO dbcounter [None req-2603b5c2-33db-4946-867e-ae50bb870ac9 None None] Registered counter for database nova_cell1 Aug 28 00:46:26.370149 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_db.sqlalchemy.engines [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105533) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 28 00:46:26.375875 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG dbcounter [-] [105533] Writer thread running {{(pid=105533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 28 00:46:26.377351 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG oslo_db.sqlalchemy.engines [None req-2603b5c2-33db-4946-867e-ae50bb870ac9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105534) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 28 00:46:26.378733 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG dbcounter [-] [105533] Writer thread running {{(pid=105533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 28 00:46:26.382825 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG dbcounter [-] [105534] Writer thread running {{(pid=105534) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 28 00:46:26.383471 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG oslo_db.sqlalchemy.engines [None req-2603b5c2-33db-4946-867e-ae50bb870ac9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105534) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 28 00:46:26.521033 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO dbcounter [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] Registered counter for database nova_cell0 Aug 28 00:46:26.523689 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG dbcounter [-] [105534] Writer thread running {{(pid=105534) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 28 00:46:26.530602 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_db.sqlalchemy.engines [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105533) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 28 00:46:26.533288 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO dbcounter [None req-2603b5c2-33db-4946-867e-ae50bb870ac9 None None] Registered counter for database nova_cell0 Aug 28 00:46:26.535187 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG dbcounter [-] [105533] Writer thread running {{(pid=105533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 28 00:46:26.542302 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG oslo_db.sqlalchemy.engines [None req-2603b5c2-33db-4946-867e-ae50bb870ac9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105534) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 28 00:46:26.547588 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG dbcounter [-] [105534] Writer thread running {{(pid=105534) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 28 00:46:26.869210 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: WARNING keystonemiddleware.auth_token [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 28 00:46:26.870139 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: WARNING keystonemiddleware.auth_token [None req-2603b5c2-33db-4946-867e-ae50bb870ac9 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 28 00:46:26.876472 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: WARNING keystonemiddleware.auth_token [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 28 00:46:26.877332 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: WARNING keystonemiddleware.auth_token [None req-2603b5c2-33db-4946-867e-ae50bb870ac9 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 28 00:46:27.215105 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: WARNING keystonemiddleware.auth_token [None req-2603b5c2-33db-4946-867e-ae50bb870ac9 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 28 00:46:27.215442 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: WARNING keystonemiddleware.auth_token [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 28 00:46:27.221101 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: WARNING keystonemiddleware.auth_token [None req-2603b5c2-33db-4946-867e-ae50bb870ac9 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 28 00:46:27.221432 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: WARNING keystonemiddleware.auth_token [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 28 00:46:27.226439 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560cdc3f99c0 pid: 105534 (default app) Aug 28 00:46:27.226802 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560cdc3f99c0 pid: 105533 (default app) Aug 28 00:46:27.228519 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-2603b5c2-33db-4946-867e-ae50bb870ac9 None None] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:46:27.229749 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-2603b5c2-33db-4946-867e-ae50bb870ac9 None None] 10.222.0.42 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001795 Aug 28 00:46:27.230169 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 788 bytes} [Sun Aug 28 00:46:27 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:47:46.331477 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:47:46.332647 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002036 Aug 28 00:47:46.333306 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 931 bytes} [Sun Aug 28 00:47:46 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:47:46.345179 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG dbcounter [-] [105534] Writing DB stats nova_api:SELECT=3 {{(pid=105534) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:47:46.346330 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG dbcounter [-] [105534] Writing DB stats nova_cell0:SELECT=3 {{(pid=105534) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:47:46.347606 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG dbcounter [-] [105534] Writing DB stats nova_cell1:SELECT=3 {{(pid=105534) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:47:46.348366 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG dbcounter [-] [105534] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105534) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:47:47.247436 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-c87526ab-c84e-4b4f-a1bc-378102c716c6 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:47:47.281170 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-c87526ab-c84e-4b4f-a1bc-378102c716c6 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.941200 Aug 28 00:47:47.282154 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 2/3] 10.222.0.42 () {60 vars in 1292 bytes} [Sun Aug 28 00:47:46 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 943 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:47:49.762217 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:47:49.763225 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-4994f927-36a5-4ea5-a5cf-f635cfd290c4 None None] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001451 Aug 28 00:47:49.765279 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 2/4] 10.222.0.42 () {54 vars in 931 bytes} [Sun Aug 28 00:47:49 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:47:50.251985 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-91e12236-826c-43aa-8e18-2d89fcd269a5 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "id": "c1", "name": "cirros256", "disk": 1, "ram": 256, "rxtx_factor": 1.0, "vcpus": 1, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:47:50.322738 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-91e12236-826c-43aa-8e18-2d89fcd269a5 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.552255 Aug 28 00:47:50.324084 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 3/5] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Aug 28 00:47:49 2022] POST /compute/v2.1/flavors => generated 407 bytes in 555 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 00:47:50.542997 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-a709a574-d0e4-4038-8dd6-8da0a68ce1cb admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:47:50.549646 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG dbcounter [-] [105533] Writing DB stats nova_api:SELECT=3 {{(pid=105533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:47:50.550498 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG dbcounter [-] [105533] Writing DB stats nova_cell0:SELECT=3 {{(pid=105533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:47:50.551399 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG dbcounter [-] [105533] Writing DB stats nova_cell1:SELECT=3 {{(pid=105533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:47:50.552103 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG dbcounter [-] [105533] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:47:50.611657 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-a709a574-d0e4-4038-8dd6-8da0a68ce1cb admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.278230 Aug 28 00:47:50.612256 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 3/6] 10.222.0.42 () {64 vars in 1336 bytes} [Sun Aug 28 00:47:50 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:47:52.979031 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-91e12236-826c-43aa-8e18-2d89fcd269a5 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:47:52.980245 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-91e12236-826c-43aa-8e18-2d89fcd269a5 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001673 Aug 28 00:47:52.980932 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 4/7] 10.222.0.42 () {54 vars in 931 bytes} [Sun Aug 28 00:47:52 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:47:53.828124 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-4df8288c-57cd-4662-90c5-a3750153baef admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d1", "swap": 0, "vcpus": 1, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 5, "name": "ds512M", "ram": 512}} {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:47:53.872484 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-4df8288c-57cd-4662-90c5-a3750153baef admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.884094 Aug 28 00:47:53.880008 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 4/8] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Aug 28 00:47:52 2022] POST /compute/v2.1/flavors => generated 404 bytes in 893 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 00:47:53.885629 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-51c27897-abee-4c6e-96ad-5e91a60e95ca admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:47:53.924487 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-51c27897-abee-4c6e-96ad-5e91a60e95ca admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.043029 Aug 28 00:47:53.925349 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 5/9] 10.222.0.42 () {64 vars in 1336 bytes} [Sun Aug 28 00:47:53 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:47:56.332400 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-4df8288c-57cd-4662-90c5-a3750153baef admin admin] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:47:56.333438 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-4df8288c-57cd-4662-90c5-a3750153baef admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001457 Aug 28 00:47:56.333797 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 931 bytes} [Sun Aug 28 00:47:56 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:47:56.735067 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-c67d11fd-708d-4809-bdf3-779e0a299a59 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "os-flavor-access:is_public": true, "swap": 0, "ram": 1024, "id": "d2", "OS-FLV-EXT-DATA:ephemeral": 0, "name": "ds1G", "disk": 10, "vcpus": 1}} {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:47:56.754996 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-c67d11fd-708d-4809-bdf3-779e0a299a59 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.414911 Aug 28 00:47:56.755643 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 6/11] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Aug 28 00:47:56 2022] POST /compute/v2.1/flavors => generated 404 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 00:47:56.767892 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-0724dd11-231e-4cc6-8e68-d46277c5d55d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:47:56.792243 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-0724dd11-231e-4cc6-8e68-d46277c5d55d admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.028388 Aug 28 00:47:56.793019 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 6/12] 10.222.0.42 () {64 vars in 1336 bytes} [Sun Aug 28 00:47:56 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:47:59.138129 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-c67d11fd-708d-4809-bdf3-779e0a299a59 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:47:59.139887 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-c67d11fd-708d-4809-bdf3-779e0a299a59 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001438 Aug 28 00:47:59.139887 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 7/13] 10.222.0.42 () {54 vars in 931 bytes} [Sun Aug 28 00:47:59 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:47:59.582490 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-1ff21cd9-1a8e-4fba-828d-69b967e80f63 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 2, "os-flavor-access:is_public": true, "name": "ds2G", "OS-FLV-EXT-DATA:ephemeral": 0, "id": "d3", "disk": 10, "ram": 2048, "rxtx_factor": 1.0, "swap": 0}} {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:47:59.605017 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-1ff21cd9-1a8e-4fba-828d-69b967e80f63 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.458588 Aug 28 00:47:59.605807 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 7/14] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Aug 28 00:47:59 2022] POST /compute/v2.1/flavors => generated 404 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 28 00:47:59.618451 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-3787b599-0e68-47d0-891e-524497faea14 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:47:59.643033 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-3787b599-0e68-47d0-891e-524497faea14 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.028285 Aug 28 00:47:59.643718 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 8/15] 10.222.0.42 () {64 vars in 1336 bytes} [Sun Aug 28 00:47:59 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:48:02.025193 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-1ff21cd9-1a8e-4fba-828d-69b967e80f63 admin admin] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:02.026261 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-1ff21cd9-1a8e-4fba-828d-69b967e80f63 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001548 Aug 28 00:48:02.026819 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 8/16] 10.222.0.42 () {54 vars in 931 bytes} [Sun Aug 28 00:48:02 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:48:02.473161 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-d8c8457a-c3d8-4227-b7f0-f517a121cf85 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d4", "vcpus": 4, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "ds4G", "swap": 0, "disk": 20, "ram": 4096, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:48:02.495939 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-d8c8457a-c3d8-4227-b7f0-f517a121cf85 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.461758 Aug 28 00:48:02.497250 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 9/17] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Aug 28 00:48:02 2022] POST /compute/v2.1/flavors => generated 404 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 00:48:02.509289 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-2d9fd48a-7cc1-4e58-9de0-36711099134a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:48:02.539666 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-2d9fd48a-7cc1-4e58-9de0-36711099134a admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.033951 Aug 28 00:48:02.540717 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 9/18] 10.222.0.42 () {64 vars in 1336 bytes} [Sun Aug 28 00:48:02 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:48:04.929663 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-d8c8457a-c3d8-4227-b7f0-f517a121cf85 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:04.931106 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-d8c8457a-c3d8-4227-b7f0-f517a121cf85 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002055 Aug 28 00:48:04.931597 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 10/19] 10.222.0.42 () {54 vars in 931 bytes} [Sun Aug 28 00:48:04 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:48:05.445564 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-4d7b6930-195c-4277-b0a4-5b4ca4327b2e admin admin] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:05.472701 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-4d7b6930-195c-4277-b0a4-5b4ca4327b2e admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.533987 Aug 28 00:48:05.473154 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 10/20] 10.222.0.42 () {60 vars in 1292 bytes} [Sun Aug 28 00:48:04 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:48:08.119434 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-d8c8457a-c3d8-4227-b7f0-f517a121cf85 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:08.121025 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-d8c8457a-c3d8-4227-b7f0-f517a121cf85 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001461 Aug 28 00:48:08.121025 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 11/21] 10.222.0.42 () {54 vars in 931 bytes} [Sun Aug 28 00:48:08 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:48:08.656722 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-3dd249f2-ba20-4115-aaa7-4e2bdfc3e828 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.tiny", "swap": 0, "id": "1", "disk": 1, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 512}} {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:48:08.690491 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-3dd249f2-ba20-4115-aaa7-4e2bdfc3e828 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.561532 Aug 28 00:48:08.691825 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 11/22] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Aug 28 00:48:08 2022] POST /compute/v2.1/flavors => generated 402 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 00:48:08.713620 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-36d1dec4-cf88-4755-abcc-3fe8f0bcdb6d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:48:08.744347 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-36d1dec4-cf88-4755-abcc-3fe8f0bcdb6d admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.037868 Aug 28 00:48:08.745208 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 12/23] 10.222.0.42 () {64 vars in 1333 bytes} [Sun Aug 28 00:48:08 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:48:11.267456 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-3dd249f2-ba20-4115-aaa7-4e2bdfc3e828 admin admin] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:11.268539 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-3dd249f2-ba20-4115-aaa7-4e2bdfc3e828 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001455 Aug 28 00:48:11.268937 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 12/24] 10.222.0.42 () {54 vars in 931 bytes} [Sun Aug 28 00:48:11 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:48:11.822495 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-c294d72b-99ea-4c01-b9d7-477746cc1d5d admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 2048, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "2", "swap": 0, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "vcpus": 1, "name": "m1.small", "disk": 20}} {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:48:11.848790 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-c294d72b-99ea-4c01-b9d7-477746cc1d5d admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.571959 Aug 28 00:48:11.849756 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 13/25] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Aug 28 00:48:11 2022] POST /compute/v2.1/flavors => generated 405 bytes in 574 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 28 00:48:11.863297 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-1c3c0eb4-a633-45ae-98bf-318a6641b695 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:48:11.891091 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-1c3c0eb4-a633-45ae-98bf-318a6641b695 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031711 Aug 28 00:48:11.891644 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 13/26] 10.222.0.42 () {64 vars in 1333 bytes} [Sun Aug 28 00:48:11 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:48:14.717677 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-c294d72b-99ea-4c01-b9d7-477746cc1d5d admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:14.719633 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-c294d72b-99ea-4c01-b9d7-477746cc1d5d admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002355 Aug 28 00:48:14.721148 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 14/27] 10.222.0.42 () {54 vars in 931 bytes} [Sun Aug 28 00:48:14 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:48:15.251404 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-1b6f3f21-02a8-439d-b111-75a5fb3f3864 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "id": "3", "os-flavor-access:is_public": true, "name": "m1.medium", "ram": 4096, "disk": 40, "vcpus": 2, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0}} {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:48:15.277255 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-1b6f3f21-02a8-439d-b111-75a5fb3f3864 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.547259 Aug 28 00:48:15.277800 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 14/28] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Aug 28 00:48:14 2022] POST /compute/v2.1/flavors => generated 406 bytes in 551 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 28 00:48:15.299454 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-26d48a84-7128-4ee2-9d42-dcde7b9a6c55 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:48:15.328154 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-26d48a84-7128-4ee2-9d42-dcde7b9a6c55 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.034117 Aug 28 00:48:15.353005 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 15/29] 10.222.0.42 () {64 vars in 1333 bytes} [Sun Aug 28 00:48:15 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:48:18.576441 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-1b6f3f21-02a8-439d-b111-75a5fb3f3864 admin admin] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:18.578186 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-1b6f3f21-02a8-439d-b111-75a5fb3f3864 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002346 Aug 28 00:48:18.578866 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 15/30] 10.222.0.42 () {54 vars in 931 bytes} [Sun Aug 28 00:48:18 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:48:19.344349 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-7628562c-1a8c-4bed-86c7-6f27857a2543 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "swap": 0, "disk": 80, "vcpus": 4, "id": "4", "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "name": "m1.large", "ram": 8192}} {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:48:19.358428 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG dbcounter [-] [105534] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=8,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=105534) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:48:19.387062 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-7628562c-1a8c-4bed-86c7-6f27857a2543 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.796106 Aug 28 00:48:19.388106 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 16/31] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Aug 28 00:48:18 2022] POST /compute/v2.1/flavors => generated 405 bytes in 798 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 00:48:19.404109 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-0efb90cc-47eb-4031-bde8-4eb76d442c72 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:48:19.453810 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-0efb90cc-47eb-4031-bde8-4eb76d442c72 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.055925 Aug 28 00:48:19.455322 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 16/32] 10.222.0.42 () {64 vars in 1333 bytes} [Sun Aug 28 00:48:19 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:48:22.100777 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-7628562c-1a8c-4bed-86c7-6f27857a2543 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:22.102438 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-7628562c-1a8c-4bed-86c7-6f27857a2543 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002091 Aug 28 00:48:22.114312 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 17/33] 10.222.0.42 () {54 vars in 931 bytes} [Sun Aug 28 00:48:22 2022] GET /compute/v2.1 => generated 388 bytes in 9 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:48:22.669548 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-85f19adb-a101-4e06-a08e-8dffaf57509c admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 160, "vcpus": 8, "name": "m1.xlarge", "ram": 16384, "rxtx_factor": 1.0, "id": "5", "os-flavor-access:is_public": true}} {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:48:22.676257 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG dbcounter [-] [105533] Writing DB stats nova_api:SELECT=45,nova_api:SAVEPOINT=5,nova_api:INSERT=9,nova_api:RELEASE=5 {{(pid=105533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:48:22.699493 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-85f19adb-a101-4e06-a08e-8dffaf57509c admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.588824 Aug 28 00:48:22.700057 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 17/34] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Aug 28 00:48:22 2022] POST /compute/v2.1/flavors => generated 408 bytes in 590 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 00:48:22.712664 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-ca4988ad-e41d-408d-a011-ce0f8a22655b admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:48:22.753100 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-ca4988ad-e41d-408d-a011-ce0f8a22655b admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.043915 Aug 28 00:48:22.753755 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 18/35] 10.222.0.42 () {64 vars in 1333 bytes} [Sun Aug 28 00:48:22 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:48:40.274517 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-85f19adb-a101-4e06-a08e-8dffaf57509c admin admin] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:40.275512 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-85f19adb-a101-4e06-a08e-8dffaf57509c admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001454 Aug 28 00:48:40.275908 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 18/36] 10.222.0.42 () {54 vars in 931 bytes} [Sun Aug 28 00:48:40 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:48:40.283049 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG dbcounter [-] [105534] Writing DB stats nova_api:INSERT=2,nova_api:SELECT=8,nova_api:SAVEPOINT=1,nova_api:RELEASE=1 {{(pid=105534) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:48:40.704142 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-b0b5c11f-685b-4b7e-9553-c9b96a0fe1e4 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:40.724819 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-b0b5c11f-685b-4b7e-9553-c9b96a0fe1e4 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.442486 Aug 28 00:48:40.725427 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 19/37] 10.222.0.42 () {60 vars in 1292 bytes} [Sun Aug 28 00:48:40 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:48:41.795410 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-85f19adb-a101-4e06-a08e-8dffaf57509c admin admin] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:41.796410 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-85f19adb-a101-4e06-a08e-8dffaf57509c admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001427 Aug 28 00:48:41.796794 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 19/38] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Aug 28 00:48:41 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:48:42.347176 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-f986a2a6-ed52-4e64-a519-f232ff4f930e admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:42.352949 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.wsgi [None req-f986a2a6-ed52-4e64-a519-f232ff4f930e admin admin] HTTP exception thrown: Flavor 42 could not be found. Aug 28 00:48:42.356226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-f986a2a6-ed52-4e64-a519-f232ff4f930e admin admin] Request method failure captured: Aug 28 00:48:42.356226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Aug 28 00:48:42.356226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Accept: application/json Aug 28 00:48:42.356226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Accept-Encoding: gzip, deflate Aug 28 00:48:42.356226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Connection: keep-alive Aug 28 00:48:42.356226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Content-Length: 0 Aug 28 00:48:42.356226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Host: 10.222.0.42 Aug 28 00:48:42.356226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Openstack-Api-Version: compute 2.90 Aug 28 00:48:42.356226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Openstack-System-Scope: None Aug 28 00:48:42.356226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: User-Agent: python-novaclient Aug 28 00:48:42.356226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Auth-Token: gAAAAABjCrtpFH-MbiCvCvnlgXdH4sYbSFMbeYWsec4RtJ56PcAnFip4aMSjms7tFOE7DYNlYUlV8RrSvIQRKZNKmM0CPijzdCv738DXY4XsOIp8HY50xVbw7NtiW810P5gzWfbIrzmHM6PyzCLOo26VPmRp5VhxLLQ8_yoFryPm0Ty29eu2x6c Aug 28 00:48:42.356226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Domain-Id: None Aug 28 00:48:42.356226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Domain-Name: None Aug 28 00:48:42.356226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Identity-Status: Confirmed Aug 28 00:48:42.356226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Is-Admin-Project: True Aug 28 00:48:42.356226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Openstack-Nova-Api-Version: 2.90 Aug 28 00:48:42.356226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Project-Domain-Id: default Aug 28 00:48:42.356226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Project-Domain-Name: Default Aug 28 00:48:42.356226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Project-Id: cca4b1c3ff844aae827dd439e67820c7 Aug 28 00:48:42.356226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Project-Name: admin Aug 28 00:48:42.356226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Role: member,reader,admin Aug 28 00:48:42.356226 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Roles: member,reader,admin Aug 28 00:48:42.357874 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_cca4b1c3ff844aae827dd439e67820c7", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cca4b1c3ff844aae827dd439e67820c7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cca4b1c3ff844aae827dd439e67820c7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/cca4b1c3ff844aae827dd439e67820c7"}]}] Aug 28 00:48:42.357874 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Tenant: admin Aug 28 00:48:42.357874 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Tenant-Id: cca4b1c3ff844aae827dd439e67820c7 Aug 28 00:48:42.357874 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Tenant-Name: admin Aug 28 00:48:42.357874 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-User: admin Aug 28 00:48:42.357874 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-User-Domain-Id: default Aug 28 00:48:42.357874 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-User-Domain-Name: Default Aug 28 00:48:42.357874 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-User-Id: e5cb815a43fc4460b6ddf0175f087128 Aug 28 00:48:42.357874 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-User-Name: admin Aug 28 00:48:42.357874 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: method: > Aug 28 00:48:42.357874 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: exception: Flavor 42 could not be found. Aug 28 00:48:42.357874 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: action_args: {'id': '42'} Aug 28 00:48:42.357874 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 00:48:42.357874 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:48:42.358888 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 28 00:48:42.358888 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 28 00:48:42.358888 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 28 00:48:42.358888 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 28 00:48:42.358888 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 28 00:48:42.358888 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 28 00:48:42.358888 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 28 00:48:42.358888 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 28 00:48:42.358888 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 28 00:48:42.358888 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:48:42.358888 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 28 00:48:42.358888 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 28 00:48:42.358888 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Aug 28 00:48:42.358888 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Aug 28 00:48:42.358888 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:48:42.358888 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Aug 28 00:48:42.358888 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:48:42.358888 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:48:42.358888 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:48:42.360213 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 00:48:42.360213 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 00:48:42.360213 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 00:48:42.360213 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:48:42.360213 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 28 00:48:42.360213 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 28 00:48:42.360213 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Aug 28 00:48:42.360213 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Aug 28 00:48:42.360213 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:48:42.360213 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Aug 28 00:48:42.360213 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:48:42.360213 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:48:42.360213 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:48:42.360213 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 00:48:42.360213 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 00:48:42.360213 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Aug 28 00:48:42.360213 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Aug 28 00:48:42.360213 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-f986a2a6-ed52-4e64-a519-f232ff4f930e admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105534) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 00:48:42.361340 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-f986a2a6-ed52-4e64-a519-f232ff4f930e admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.425423 Aug 28 00:48:42.361340 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 20/39] 10.222.0.42 () {60 vars in 1190 bytes} [Sun Aug 28 00:48:41 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 427 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 28 00:48:42.367774 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-70406839-24ac-4ba3-b132-95d0b3c2fa70 admin admin] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:42.392177 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-70406839-24ac-4ba3-b132-95d0b3c2fa70 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.028099 Aug 28 00:48:42.392767 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 20/40] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Aug 28 00:48:42 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:48:42.403081 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-aac1dd2e-26a1-4284-890d-5cc5df442f64 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:42.425471 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-aac1dd2e-26a1-4284-890d-5cc5df442f64 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.026070 Aug 28 00:48:42.426011 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 21/41] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Aug 28 00:48:42 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:48:42.435921 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-3ab5cee3-d740-45f9-982a-744e292b1b36 admin admin] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:42.441971 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.wsgi [None req-3ab5cee3-d740-45f9-982a-744e292b1b36 admin admin] HTTP exception thrown: Flavor 42 could not be found. Aug 28 00:48:42.445231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-3ab5cee3-d740-45f9-982a-744e292b1b36 admin admin] Request method failure captured: Aug 28 00:48:42.445231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Aug 28 00:48:42.445231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Accept: application/json Aug 28 00:48:42.445231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Accept-Encoding: gzip, deflate Aug 28 00:48:42.445231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Connection: keep-alive Aug 28 00:48:42.445231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Content-Length: 0 Aug 28 00:48:42.445231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Host: 10.222.0.42 Aug 28 00:48:42.445231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Openstack-Api-Version: compute 2.90 Aug 28 00:48:42.445231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Openstack-System-Scope: None Aug 28 00:48:42.445231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: User-Agent: python-novaclient Aug 28 00:48:42.445231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Auth-Token: gAAAAABjCrtpFH-MbiCvCvnlgXdH4sYbSFMbeYWsec4RtJ56PcAnFip4aMSjms7tFOE7DYNlYUlV8RrSvIQRKZNKmM0CPijzdCv738DXY4XsOIp8HY50xVbw7NtiW810P5gzWfbIrzmHM6PyzCLOo26VPmRp5VhxLLQ8_yoFryPm0Ty29eu2x6c Aug 28 00:48:42.445231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Domain-Id: None Aug 28 00:48:42.445231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Domain-Name: None Aug 28 00:48:42.445231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Identity-Status: Confirmed Aug 28 00:48:42.445231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Is-Admin-Project: True Aug 28 00:48:42.445231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Openstack-Nova-Api-Version: 2.90 Aug 28 00:48:42.445231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Project-Domain-Id: default Aug 28 00:48:42.445231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Project-Domain-Name: Default Aug 28 00:48:42.445231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Project-Id: cca4b1c3ff844aae827dd439e67820c7 Aug 28 00:48:42.445231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Project-Name: admin Aug 28 00:48:42.445231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Role: member,reader,admin Aug 28 00:48:42.445231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Roles: member,reader,admin Aug 28 00:48:42.446688 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_cca4b1c3ff844aae827dd439e67820c7", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cca4b1c3ff844aae827dd439e67820c7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cca4b1c3ff844aae827dd439e67820c7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/cca4b1c3ff844aae827dd439e67820c7"}]}] Aug 28 00:48:42.446688 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Tenant: admin Aug 28 00:48:42.446688 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Tenant-Id: cca4b1c3ff844aae827dd439e67820c7 Aug 28 00:48:42.446688 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Tenant-Name: admin Aug 28 00:48:42.446688 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-User: admin Aug 28 00:48:42.446688 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-User-Domain-Id: default Aug 28 00:48:42.446688 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-User-Domain-Name: Default Aug 28 00:48:42.446688 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-User-Id: e5cb815a43fc4460b6ddf0175f087128 Aug 28 00:48:42.446688 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-User-Name: admin Aug 28 00:48:42.446688 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: method: > Aug 28 00:48:42.446688 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: exception: Flavor 42 could not be found. Aug 28 00:48:42.446688 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: action_args: {'id': '42'} Aug 28 00:48:42.446688 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 00:48:42.446688 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:48:42.447607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 28 00:48:42.447607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 28 00:48:42.447607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 28 00:48:42.447607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 28 00:48:42.447607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 28 00:48:42.447607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 28 00:48:42.447607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 28 00:48:42.447607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 28 00:48:42.447607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 28 00:48:42.447607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:48:42.447607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 28 00:48:42.447607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 28 00:48:42.447607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Aug 28 00:48:42.447607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Aug 28 00:48:42.447607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:48:42.447607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Aug 28 00:48:42.447607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:48:42.447607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:48:42.447607 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:48:42.448863 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 00:48:42.448863 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 00:48:42.448863 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 00:48:42.448863 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:48:42.448863 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 28 00:48:42.448863 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 28 00:48:42.448863 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Aug 28 00:48:42.448863 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Aug 28 00:48:42.448863 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:48:42.448863 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Aug 28 00:48:42.448863 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:48:42.448863 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:48:42.448863 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:48:42.448863 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 00:48:42.448863 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 00:48:42.448863 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Aug 28 00:48:42.448863 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Aug 28 00:48:42.448863 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-3ab5cee3-d740-45f9-982a-744e292b1b36 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 00:48:42.450549 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-3ab5cee3-d740-45f9-982a-744e292b1b36 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.014348 Aug 28 00:48:42.450549 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 21/42] 10.222.0.42 () {60 vars in 1190 bytes} [Sun Aug 28 00:48:42 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 28 00:48:43.454006 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-aac1dd2e-26a1-4284-890d-5cc5df442f64 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:43.455028 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-aac1dd2e-26a1-4284-890d-5cc5df442f64 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001488 Aug 28 00:48:43.455401 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 22/43] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Aug 28 00:48:43 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:48:44.007355 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-900ae3d9-eaca-4ef0-8f00-03e0f629f77b admin admin] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:44.012989 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.wsgi [None req-900ae3d9-eaca-4ef0-8f00-03e0f629f77b admin admin] HTTP exception thrown: Flavor 84 could not be found. Aug 28 00:48:44.014543 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-900ae3d9-eaca-4ef0-8f00-03e0f629f77b admin admin] Request method failure captured: Aug 28 00:48:44.014543 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Aug 28 00:48:44.014543 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Accept: application/json Aug 28 00:48:44.014543 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Accept-Encoding: gzip, deflate Aug 28 00:48:44.014543 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Connection: keep-alive Aug 28 00:48:44.014543 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Content-Length: 0 Aug 28 00:48:44.014543 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Host: 10.222.0.42 Aug 28 00:48:44.014543 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Openstack-Api-Version: compute 2.90 Aug 28 00:48:44.014543 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Openstack-System-Scope: None Aug 28 00:48:44.014543 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: User-Agent: python-novaclient Aug 28 00:48:44.014543 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Auth-Token: gAAAAABjCrtrktclr7KWqTs27fyN4TXOWG0dFsygELm5nomeN7z-Y6ttmEmajAkzThrYZdbKG9yTLnIbHpdEpKU3RQ9afRtOZQkgffxhK9DeOUMHDr_HRqPYzDvy3-y4E242UYdm8zEtirRyII522ukhEhJeOf17-VPl6DHBPH6wLbNBxrY4n9Q Aug 28 00:48:44.014543 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Domain-Id: None Aug 28 00:48:44.014543 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Domain-Name: None Aug 28 00:48:44.014543 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Identity-Status: Confirmed Aug 28 00:48:44.014543 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Is-Admin-Project: True Aug 28 00:48:44.014543 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Openstack-Nova-Api-Version: 2.90 Aug 28 00:48:44.014543 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Project-Domain-Id: default Aug 28 00:48:44.014543 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Project-Domain-Name: Default Aug 28 00:48:44.014543 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Project-Id: cca4b1c3ff844aae827dd439e67820c7 Aug 28 00:48:44.014543 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Project-Name: admin Aug 28 00:48:44.014543 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Role: member,reader,admin Aug 28 00:48:44.014543 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Roles: member,reader,admin Aug 28 00:48:44.016023 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_cca4b1c3ff844aae827dd439e67820c7", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cca4b1c3ff844aae827dd439e67820c7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cca4b1c3ff844aae827dd439e67820c7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/cca4b1c3ff844aae827dd439e67820c7"}]}] Aug 28 00:48:44.016023 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Tenant: admin Aug 28 00:48:44.016023 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Tenant-Id: cca4b1c3ff844aae827dd439e67820c7 Aug 28 00:48:44.016023 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Tenant-Name: admin Aug 28 00:48:44.016023 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-User: admin Aug 28 00:48:44.016023 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-User-Domain-Id: default Aug 28 00:48:44.016023 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-User-Domain-Name: Default Aug 28 00:48:44.016023 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-User-Id: e5cb815a43fc4460b6ddf0175f087128 Aug 28 00:48:44.016023 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-User-Name: admin Aug 28 00:48:44.016023 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: method: > Aug 28 00:48:44.016023 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: exception: Flavor 84 could not be found. Aug 28 00:48:44.016023 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: action_args: {'id': '84'} Aug 28 00:48:44.016023 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 00:48:44.016023 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:48:44.016896 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 28 00:48:44.016896 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 28 00:48:44.016896 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 28 00:48:44.016896 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 28 00:48:44.016896 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 28 00:48:44.016896 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 28 00:48:44.016896 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 28 00:48:44.016896 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 28 00:48:44.016896 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 28 00:48:44.016896 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:48:44.016896 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 28 00:48:44.016896 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 28 00:48:44.016896 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Aug 28 00:48:44.016896 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Aug 28 00:48:44.016896 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:48:44.016896 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Aug 28 00:48:44.016896 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:48:44.016896 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:48:44.016896 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:48:44.018185 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 00:48:44.018185 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 00:48:44.018185 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 00:48:44.018185 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:48:44.018185 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 28 00:48:44.018185 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 28 00:48:44.018185 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Aug 28 00:48:44.018185 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Aug 28 00:48:44.018185 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:48:44.018185 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Aug 28 00:48:44.018185 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:48:44.018185 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:48:44.018185 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:48:44.018185 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 00:48:44.018185 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 00:48:44.018185 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Aug 28 00:48:44.018185 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Aug 28 00:48:44.018185 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-900ae3d9-eaca-4ef0-8f00-03e0f629f77b admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 00:48:44.019416 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-900ae3d9-eaca-4ef0-8f00-03e0f629f77b admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.424313 Aug 28 00:48:44.019416 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 22/44] 10.222.0.42 () {60 vars in 1190 bytes} [Sun Aug 28 00:48:43 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 425 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 28 00:48:44.025932 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-ed36330e-51d3-4048-a00c-ccad43c89472 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:44.041537 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-ed36330e-51d3-4048-a00c-ccad43c89472 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.019356 Aug 28 00:48:44.042079 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 23/45] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Aug 28 00:48:44 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:48:44.052195 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-c73f34fe-1a84-46e5-b355-95ada3c078cf admin admin] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:44.069220 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-c73f34fe-1a84-46e5-b355-95ada3c078cf admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.020547 Aug 28 00:48:44.069761 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 23/46] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Aug 28 00:48:44 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:48:44.079884 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-95f739d3-578a-4f08-88d5-b4ea1d4bc3ed admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:44.085539 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.wsgi [None req-95f739d3-578a-4f08-88d5-b4ea1d4bc3ed admin admin] HTTP exception thrown: Flavor 84 could not be found. Aug 28 00:48:44.087063 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-95f739d3-578a-4f08-88d5-b4ea1d4bc3ed admin admin] Request method failure captured: Aug 28 00:48:44.087063 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Aug 28 00:48:44.087063 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Accept: application/json Aug 28 00:48:44.087063 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Accept-Encoding: gzip, deflate Aug 28 00:48:44.087063 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Connection: keep-alive Aug 28 00:48:44.087063 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Content-Length: 0 Aug 28 00:48:44.087063 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Host: 10.222.0.42 Aug 28 00:48:44.087063 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Openstack-Api-Version: compute 2.90 Aug 28 00:48:44.087063 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Openstack-System-Scope: None Aug 28 00:48:44.087063 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: User-Agent: python-novaclient Aug 28 00:48:44.087063 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Auth-Token: gAAAAABjCrtrktclr7KWqTs27fyN4TXOWG0dFsygELm5nomeN7z-Y6ttmEmajAkzThrYZdbKG9yTLnIbHpdEpKU3RQ9afRtOZQkgffxhK9DeOUMHDr_HRqPYzDvy3-y4E242UYdm8zEtirRyII522ukhEhJeOf17-VPl6DHBPH6wLbNBxrY4n9Q Aug 28 00:48:44.087063 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Domain-Id: None Aug 28 00:48:44.087063 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Domain-Name: None Aug 28 00:48:44.087063 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Identity-Status: Confirmed Aug 28 00:48:44.087063 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Is-Admin-Project: True Aug 28 00:48:44.087063 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Openstack-Nova-Api-Version: 2.90 Aug 28 00:48:44.087063 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Project-Domain-Id: default Aug 28 00:48:44.087063 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Project-Domain-Name: Default Aug 28 00:48:44.087063 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Project-Id: cca4b1c3ff844aae827dd439e67820c7 Aug 28 00:48:44.087063 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Project-Name: admin Aug 28 00:48:44.087063 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Role: member,reader,admin Aug 28 00:48:44.087063 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Roles: member,reader,admin Aug 28 00:48:44.088482 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_cca4b1c3ff844aae827dd439e67820c7", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cca4b1c3ff844aae827dd439e67820c7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cca4b1c3ff844aae827dd439e67820c7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/cca4b1c3ff844aae827dd439e67820c7"}]}] Aug 28 00:48:44.088482 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Tenant: admin Aug 28 00:48:44.088482 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Tenant-Id: cca4b1c3ff844aae827dd439e67820c7 Aug 28 00:48:44.088482 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Tenant-Name: admin Aug 28 00:48:44.088482 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-User: admin Aug 28 00:48:44.088482 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-User-Domain-Id: default Aug 28 00:48:44.088482 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-User-Domain-Name: Default Aug 28 00:48:44.088482 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-User-Id: e5cb815a43fc4460b6ddf0175f087128 Aug 28 00:48:44.088482 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-User-Name: admin Aug 28 00:48:44.088482 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: method: > Aug 28 00:48:44.088482 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: exception: Flavor 84 could not be found. Aug 28 00:48:44.088482 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: action_args: {'id': '84'} Aug 28 00:48:44.088482 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 00:48:44.088482 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:48:44.089420 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 28 00:48:44.089420 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 28 00:48:44.089420 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 28 00:48:44.089420 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 28 00:48:44.089420 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 28 00:48:44.089420 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 28 00:48:44.089420 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 28 00:48:44.089420 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 28 00:48:44.089420 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 28 00:48:44.089420 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:48:44.089420 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 28 00:48:44.089420 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 28 00:48:44.089420 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Aug 28 00:48:44.089420 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Aug 28 00:48:44.089420 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:48:44.089420 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Aug 28 00:48:44.089420 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:48:44.089420 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:48:44.089420 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:48:44.090674 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 00:48:44.090674 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 00:48:44.090674 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 00:48:44.090674 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:48:44.090674 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 28 00:48:44.090674 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 28 00:48:44.090674 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Aug 28 00:48:44.090674 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Aug 28 00:48:44.090674 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:48:44.090674 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Aug 28 00:48:44.090674 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:48:44.090674 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:48:44.090674 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:48:44.090674 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 00:48:44.090674 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 00:48:44.090674 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Aug 28 00:48:44.090674 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Aug 28 00:48:44.090674 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-95f739d3-578a-4f08-88d5-b4ea1d4bc3ed admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105534) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 00:48:44.092002 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-95f739d3-578a-4f08-88d5-b4ea1d4bc3ed admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.012227 Aug 28 00:48:44.092002 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 24/47] 10.222.0.42 () {60 vars in 1190 bytes} [Sun Aug 28 00:48:44 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 28 00:48:45.068031 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-c73f34fe-1a84-46e5-b355-95ada3c078cf admin admin] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:45.069151 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-c73f34fe-1a84-46e5-b355-95ada3c078cf admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001506 Aug 28 00:48:45.069549 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 24/48] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Aug 28 00:48:45 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:48:45.634123 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-1c9b53cb-5c3d-428b-b406-eb406c18d465 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:45.640038 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.wsgi [None req-1c9b53cb-5c3d-428b-b406-eb406c18d465 admin admin] HTTP exception thrown: Flavor 451 could not be found. Aug 28 00:48:45.641573 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-1c9b53cb-5c3d-428b-b406-eb406c18d465 admin admin] Request method failure captured: Aug 28 00:48:45.641573 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Aug 28 00:48:45.641573 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Accept: application/json Aug 28 00:48:45.641573 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Accept-Encoding: gzip, deflate Aug 28 00:48:45.641573 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Connection: keep-alive Aug 28 00:48:45.641573 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Content-Length: 0 Aug 28 00:48:45.641573 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Host: 10.222.0.42 Aug 28 00:48:45.641573 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Openstack-Api-Version: compute 2.90 Aug 28 00:48:45.641573 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Openstack-System-Scope: None Aug 28 00:48:45.641573 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: User-Agent: python-novaclient Aug 28 00:48:45.641573 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Auth-Token: gAAAAABjCrtshWrbruy8b1_jRKRP_2BTiEaLrNhn6P8sAIeLAYQfVR0-87ovpbsNgTA63QMs2ruhN7utVzDTpVy_6VAC03ekFSL2u-tkzmSFRhGV_AJ95fTRmcfWWQxw-Pvor2Vj7sp9tiyfjB5rIMVfxG4hytaziTd5EbQHADdcX1g5UzvwSeM Aug 28 00:48:45.641573 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Domain-Id: None Aug 28 00:48:45.641573 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Domain-Name: None Aug 28 00:48:45.641573 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Identity-Status: Confirmed Aug 28 00:48:45.641573 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Is-Admin-Project: True Aug 28 00:48:45.641573 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Openstack-Nova-Api-Version: 2.90 Aug 28 00:48:45.641573 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Project-Domain-Id: default Aug 28 00:48:45.641573 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Project-Domain-Name: Default Aug 28 00:48:45.641573 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Project-Id: cca4b1c3ff844aae827dd439e67820c7 Aug 28 00:48:45.641573 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Project-Name: admin Aug 28 00:48:45.641573 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Role: member,reader,admin Aug 28 00:48:45.641573 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Roles: member,reader,admin Aug 28 00:48:45.643183 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_cca4b1c3ff844aae827dd439e67820c7", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cca4b1c3ff844aae827dd439e67820c7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cca4b1c3ff844aae827dd439e67820c7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/cca4b1c3ff844aae827dd439e67820c7"}]}] Aug 28 00:48:45.643183 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Tenant: admin Aug 28 00:48:45.643183 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Tenant-Id: cca4b1c3ff844aae827dd439e67820c7 Aug 28 00:48:45.643183 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-Tenant-Name: admin Aug 28 00:48:45.643183 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-User: admin Aug 28 00:48:45.643183 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-User-Domain-Id: default Aug 28 00:48:45.643183 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-User-Domain-Name: Default Aug 28 00:48:45.643183 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-User-Id: e5cb815a43fc4460b6ddf0175f087128 Aug 28 00:48:45.643183 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: X-User-Name: admin Aug 28 00:48:45.643183 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: method: > Aug 28 00:48:45.643183 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: exception: Flavor 451 could not be found. Aug 28 00:48:45.643183 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: action_args: {'id': '451'} Aug 28 00:48:45.643183 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 00:48:45.643183 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:48:45.644202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 28 00:48:45.644202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 28 00:48:45.644202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 28 00:48:45.644202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 28 00:48:45.644202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 28 00:48:45.644202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 28 00:48:45.644202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 28 00:48:45.644202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 28 00:48:45.644202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 28 00:48:45.644202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:48:45.644202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 28 00:48:45.644202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 28 00:48:45.644202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Aug 28 00:48:45.644202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Aug 28 00:48:45.644202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:48:45.644202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Aug 28 00:48:45.644202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:48:45.644202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:48:45.644202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:48:45.645782 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 00:48:45.645782 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 00:48:45.645782 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 00:48:45.645782 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:48:45.645782 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 28 00:48:45.645782 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 28 00:48:45.645782 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Aug 28 00:48:45.645782 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Aug 28 00:48:45.645782 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:48:45.645782 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Aug 28 00:48:45.645782 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:48:45.645782 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:48:45.645782 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:48:45.645782 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 00:48:45.645782 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 00:48:45.645782 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Aug 28 00:48:45.645782 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: ERROR nova.api.openstack.wsgi Aug 28 00:48:45.645782 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-1c9b53cb-5c3d-428b-b406-eb406c18d465 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105534) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 00:48:45.647121 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-1c9b53cb-5c3d-428b-b406-eb406c18d465 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.433315 Aug 28 00:48:45.647121 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 25/49] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Aug 28 00:48:45 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 435 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 28 00:48:45.652709 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-29668154-ff44-4548-b83e-e446be3c8328 admin admin] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:45.667881 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-29668154-ff44-4548-b83e-e446be3c8328 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018748 Aug 28 00:48:45.668408 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 25/50] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Aug 28 00:48:45 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:48:45.678436 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-976b023f-e0bf-4033-a010-6a9bda7bc5f2 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:45.694174 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-976b023f-e0bf-4033-a010-6a9bda7bc5f2 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.019320 Aug 28 00:48:45.694771 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 26/51] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Aug 28 00:48:45 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:48:45.705145 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-c8886e3e-fe4e-41a1-b61b-91c9c6e6982b admin admin] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:45.710996 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.wsgi [None req-c8886e3e-fe4e-41a1-b61b-91c9c6e6982b admin admin] HTTP exception thrown: Flavor 451 could not be found. Aug 28 00:48:45.712504 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-c8886e3e-fe4e-41a1-b61b-91c9c6e6982b admin admin] Request method failure captured: Aug 28 00:48:45.712504 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Aug 28 00:48:45.712504 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Accept: application/json Aug 28 00:48:45.712504 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Accept-Encoding: gzip, deflate Aug 28 00:48:45.712504 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Connection: keep-alive Aug 28 00:48:45.712504 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Content-Length: 0 Aug 28 00:48:45.712504 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Host: 10.222.0.42 Aug 28 00:48:45.712504 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Openstack-Api-Version: compute 2.90 Aug 28 00:48:45.712504 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Openstack-System-Scope: None Aug 28 00:48:45.712504 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: User-Agent: python-novaclient Aug 28 00:48:45.712504 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Auth-Token: gAAAAABjCrtshWrbruy8b1_jRKRP_2BTiEaLrNhn6P8sAIeLAYQfVR0-87ovpbsNgTA63QMs2ruhN7utVzDTpVy_6VAC03ekFSL2u-tkzmSFRhGV_AJ95fTRmcfWWQxw-Pvor2Vj7sp9tiyfjB5rIMVfxG4hytaziTd5EbQHADdcX1g5UzvwSeM Aug 28 00:48:45.712504 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Domain-Id: None Aug 28 00:48:45.712504 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Domain-Name: None Aug 28 00:48:45.712504 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Identity-Status: Confirmed Aug 28 00:48:45.712504 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Is-Admin-Project: True Aug 28 00:48:45.712504 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Openstack-Nova-Api-Version: 2.90 Aug 28 00:48:45.712504 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Project-Domain-Id: default Aug 28 00:48:45.712504 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Project-Domain-Name: Default Aug 28 00:48:45.712504 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Project-Id: cca4b1c3ff844aae827dd439e67820c7 Aug 28 00:48:45.712504 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Project-Name: admin Aug 28 00:48:45.712504 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Role: member,reader,admin Aug 28 00:48:45.712504 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Roles: member,reader,admin Aug 28 00:48:45.714047 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_cca4b1c3ff844aae827dd439e67820c7", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cca4b1c3ff844aae827dd439e67820c7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/cca4b1c3ff844aae827dd439e67820c7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/cca4b1c3ff844aae827dd439e67820c7"}]}] Aug 28 00:48:45.714047 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Tenant: admin Aug 28 00:48:45.714047 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Tenant-Id: cca4b1c3ff844aae827dd439e67820c7 Aug 28 00:48:45.714047 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-Tenant-Name: admin Aug 28 00:48:45.714047 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-User: admin Aug 28 00:48:45.714047 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-User-Domain-Id: default Aug 28 00:48:45.714047 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-User-Domain-Name: Default Aug 28 00:48:45.714047 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-User-Id: e5cb815a43fc4460b6ddf0175f087128 Aug 28 00:48:45.714047 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: X-User-Name: admin Aug 28 00:48:45.714047 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: method: > Aug 28 00:48:45.714047 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: exception: Flavor 451 could not be found. Aug 28 00:48:45.714047 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: action_args: {'id': '451'} Aug 28 00:48:45.714047 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 00:48:45.714047 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:48:45.714846 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Aug 28 00:48:45.714846 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Aug 28 00:48:45.714846 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Aug 28 00:48:45.714846 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Aug 28 00:48:45.714846 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 28 00:48:45.714846 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 28 00:48:45.714846 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Aug 28 00:48:45.714846 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Aug 28 00:48:45.714846 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Aug 28 00:48:45.714846 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:48:45.714846 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Aug 28 00:48:45.714846 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Aug 28 00:48:45.714846 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Aug 28 00:48:45.714846 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Aug 28 00:48:45.714846 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:48:45.714846 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Aug 28 00:48:45.714846 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:48:45.714846 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:48:45.714846 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:48:45.715892 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 00:48:45.715892 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 00:48:45.715892 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 00:48:45.715892 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:48:45.715892 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Aug 28 00:48:45.715892 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Aug 28 00:48:45.715892 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Aug 28 00:48:45.715892 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Aug 28 00:48:45.715892 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:48:45.715892 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Aug 28 00:48:45.715892 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:48:45.715892 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:48:45.715892 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:48:45.715892 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 00:48:45.715892 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 00:48:45.715892 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Aug 28 00:48:45.715892 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: ERROR nova.api.openstack.wsgi Aug 28 00:48:45.715892 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-c8886e3e-fe4e-41a1-b61b-91c9c6e6982b admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105533) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 00:48:45.717031 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-c8886e3e-fe4e-41a1-b61b-91c9c6e6982b admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.012394 Aug 28 00:48:45.717031 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 26/52] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Aug 28 00:48:45 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Aug 28 00:48:46.695802 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-976b023f-e0bf-4033-a010-6a9bda7bc5f2 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:46.696799 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-976b023f-e0bf-4033-a010-6a9bda7bc5f2 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001439 Aug 28 00:48:46.697234 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 27/53] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Aug 28 00:48:46 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:48:47.245690 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-ee8f0c78-fb6e-4a07-92b0-e1d57c8500cb admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:48:47.268603 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-ee8f0c78-fb6e-4a07-92b0-e1d57c8500cb admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.90 time: 0.433006 Aug 28 00:48:47.269420 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 27/54] 10.222.0.42 () {64 vars in 1235 bytes} [Sun Aug 28 00:48:46 2022] POST /compute/v2.1/flavors => generated 404 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 00:48:48.250047 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-976b023f-e0bf-4033-a010-6a9bda7bc5f2 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:48.251088 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-976b023f-e0bf-4033-a010-6a9bda7bc5f2 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001478 Aug 28 00:48:48.251483 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 28/55] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Aug 28 00:48:48 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:48:48.795080 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-a170026a-d4ed-4b9b-a60c-33848ae21c5c admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:48:48.814991 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-a170026a-d4ed-4b9b-a60c-33848ae21c5c admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.90 time: 0.423327 Aug 28 00:48:48.815457 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 28/56] 10.222.0.42 () {64 vars in 1235 bytes} [Sun Aug 28 00:48:48 2022] POST /compute/v2.1/flavors => generated 405 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 00:48:49.828254 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-976b023f-e0bf-4033-a010-6a9bda7bc5f2 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:49.829390 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-976b023f-e0bf-4033-a010-6a9bda7bc5f2 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001555 Aug 28 00:48:49.829797 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 29/57] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Aug 28 00:48:49 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:48:50.401234 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-1ff6bd60-6e9e-419e-9297-2e2b741f0dfa admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:48:50.431117 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-1ff6bd60-6e9e-419e-9297-2e2b741f0dfa admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.90 time: 0.466197 Aug 28 00:48:50.432212 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 29/58] 10.222.0.42 () {64 vars in 1235 bytes} [Sun Aug 28 00:48:49 2022] POST /compute/v2.1/flavors => generated 407 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 00:48:51.413525 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-976b023f-e0bf-4033-a010-6a9bda7bc5f2 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:51.414523 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-976b023f-e0bf-4033-a010-6a9bda7bc5f2 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001465 Aug 28 00:48:51.414915 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 30/59] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Aug 28 00:48:51 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:48:51.967013 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-8c580f0c-f1e1-4cd5-a9e9-8c4130724049 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:48:51.988395 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-8c580f0c-f1e1-4cd5-a9e9-8c4130724049 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.90 time: 0.434764 Aug 28 00:48:51.988989 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 30/60] 10.222.0.42 () {64 vars in 1235 bytes} [Sun Aug 28 00:48:51 2022] POST /compute/v2.1/flavors => generated 409 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 00:48:52.983553 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-976b023f-e0bf-4033-a010-6a9bda7bc5f2 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:52.984807 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-976b023f-e0bf-4033-a010-6a9bda7bc5f2 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001623 Aug 28 00:48:52.985240 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 31/61] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Aug 28 00:48:52 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:48:53.557677 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-6f8a97a9-2d3a-43b4-bdd8-b5bf1a0f90a8 admin admin] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:48:53.578242 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-6f8a97a9-2d3a-43b4-bdd8-b5bf1a0f90a8 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.90 time: 0.457526 Aug 28 00:48:53.578812 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 31/62] 10.222.0.42 () {60 vars in 1202 bytes} [Sun Aug 28 00:48:53 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:49:47.855365 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-976b023f-e0bf-4033-a010-6a9bda7bc5f2 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:49:47.856363 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-976b023f-e0bf-4033-a010-6a9bda7bc5f2 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001453 Aug 28 00:49:47.856758 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 32/63] 10.222.0.42 () {54 vars in 931 bytes} [Sun Aug 28 00:49:47 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:49:47.867614 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG dbcounter [-] [105533] Writing DB stats nova_api:INSERT=5,nova_api:SELECT=28 {{(pid=105533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:49:48.292693 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-5ce86ef4-75fd-404e-9e2e-7c090cb7f34b admin admin] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:49:48.314353 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-5ce86ef4-75fd-404e-9e2e-7c090cb7f34b admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.451098 Aug 28 00:49:48.314933 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 32/64] 10.222.0.42 () {60 vars in 1292 bytes} [Sun Aug 28 00:49:47 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:49:48.336278 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG dbcounter [-] [105534] Writing DB stats nova_api:SELECT=14 {{(pid=105534) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:49:48.340201 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-77aa3246-5870-457f-b8a5-62e5be988326 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:49:48.348217 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-77aa3246-5870-457f-b8a5-62e5be988326 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013420 Aug 28 00:49:48.348782 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 33/65] 10.222.0.42 () {60 vars in 1283 bytes} [Sun Aug 28 00:49:48 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:49:48.360993 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-3f27ec98-33a7-4afe-a31a-cb603e3513c1 admin admin] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:49:48.368837 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-3f27ec98-33a7-4afe-a31a-cb603e3513c1 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011472 Aug 28 00:49:48.369359 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 33/66] 10.222.0.42 () {60 vars in 1286 bytes} [Sun Aug 28 00:49:48 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:49:48.381145 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-962bda17-c320-465b-bef6-3a1527ccb530 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:49:48.389051 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-962bda17-c320-465b-bef6-3a1527ccb530 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011492 Aug 28 00:49:48.389590 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 34/67] 10.222.0.42 () {60 vars in 1286 bytes} [Sun Aug 28 00:49:48 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:49:48.401661 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-88f154fb-5bba-48a1-97eb-263358f0f4b4 admin admin] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:49:48.409841 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-88f154fb-5bba-48a1-97eb-263358f0f4b4 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011762 Aug 28 00:49:48.410393 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 34/68] 10.222.0.42 () {60 vars in 1283 bytes} [Sun Aug 28 00:49:48 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:52:45.156263 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-962bda17-c320-465b-bef6-3a1527ccb530 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:52:45.157315 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-962bda17-c320-465b-bef6-3a1527ccb530 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001460 Aug 28 00:52:45.157698 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 35/69] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Aug 28 00:52:45 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:52:45.295493 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG dbcounter [-] [105533] Writing DB stats nova_api:SELECT=6 {{(pid=105533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:52:45.484603 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-d94e07bc-33a1-425b-b79b-58df744fa4b1 admin admin] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:52:45.488025 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-d94e07bc-33a1-425b-b79b-58df744fa4b1 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:52:45.488499 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-d94e07bc-33a1-425b-b79b-58df744fa4b1 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:52:45.488951 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-d94e07bc-33a1-425b-b79b-58df744fa4b1 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:52:45.489686 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-d94e07bc-33a1-425b-b79b-58df744fa4b1 admin admin] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:52:45.490123 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-d94e07bc-33a1-425b-b79b-58df744fa4b1 admin admin] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:52:45.490532 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-d94e07bc-33a1-425b-b79b-58df744fa4b1 admin admin] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:52:45.523727 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-d94e07bc-33a1-425b-b79b-58df744fa4b1 admin admin] 10.222.0.42 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.229655 Aug 28 00:52:45.524291 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 35/70] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Aug 28 00:52:45 2022] GET /compute/v2.1/os-services => generated 791 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 00:52:46.487202 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-962bda17-c320-465b-bef6-3a1527ccb530 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:52:46.488212 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-962bda17-c320-465b-bef6-3a1527ccb530 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001472 Aug 28 00:52:46.488634 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 36/71] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Aug 28 00:52:46 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:52:46.681225 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-df4b0d3c-11cc-4dd2-9273-04e459e67bd3 admin admin] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:52:46.684636 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-df4b0d3c-11cc-4dd2-9273-04e459e67bd3 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:52:46.685145 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-df4b0d3c-11cc-4dd2-9273-04e459e67bd3 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:52:46.685598 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-df4b0d3c-11cc-4dd2-9273-04e459e67bd3 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:52:46.686310 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-df4b0d3c-11cc-4dd2-9273-04e459e67bd3 admin admin] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:52:46.686752 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-df4b0d3c-11cc-4dd2-9273-04e459e67bd3 admin admin] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:52:46.687161 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-df4b0d3c-11cc-4dd2-9273-04e459e67bd3 admin admin] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:52:46.707306 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-df4b0d3c-11cc-4dd2-9273-04e459e67bd3 admin admin] 10.222.0.42 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.080053 Aug 28 00:52:46.707784 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 36/72] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Aug 28 00:52:46 2022] GET /compute/v2.1/os-services => generated 791 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 00:52:57.956919 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-962bda17-c320-465b-bef6-3a1527ccb530 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:52:57.957938 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-962bda17-c320-465b-bef6-3a1527ccb530 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001519 Aug 28 00:52:57.958362 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 37/73] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Aug 28 00:52:57 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:52:58.101375 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG dbcounter [-] [105533] Writing DB stats nova_cell0:SELECT=4 {{(pid=105533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:52:58.102109 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG dbcounter [-] [105533] Writing DB stats nova_cell1:SELECT=4 {{(pid=105533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:52:58.102699 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG dbcounter [-] [105533] Writing DB stats nova_api:SELECT=8 {{(pid=105533) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:52:58.157002 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-807c4deb-8844-479e-8833-09bcc7549929 admin admin] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:52:58.160407 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-807c4deb-8844-479e-8833-09bcc7549929 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:52:58.161051 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-807c4deb-8844-479e-8833-09bcc7549929 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:52:58.161463 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-807c4deb-8844-479e-8833-09bcc7549929 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:52:58.162231 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-807c4deb-8844-479e-8833-09bcc7549929 admin admin] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:52:58.162575 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-807c4deb-8844-479e-8833-09bcc7549929 admin admin] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:52:58.162984 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-807c4deb-8844-479e-8833-09bcc7549929 admin admin] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:52:58.188129 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-807c4deb-8844-479e-8833-09bcc7549929 admin admin] 10.222.0.42 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.90 time: 0.091187 Aug 28 00:52:58.188679 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 37/74] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Aug 28 00:52:58 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:52:59.010704 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: DEBUG nova.api.openstack.wsgi [None req-962bda17-c320-465b-bef6-3a1527ccb530 admin admin] Calling method '>' {{(pid=105534) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:52:59.011704 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: INFO nova.api.openstack.requestlog [None req-962bda17-c320-465b-bef6-3a1527ccb530 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001457 Aug 28 00:52:59.012100 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: [pid: 105534|app: 0|req: 38/75] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Aug 28 00:52:59 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:52:59.208050 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG nova.api.openstack.wsgi [None req-7769d5a4-c364-4f52-b190-a5a628a3a486 admin admin] Calling method '>' {{(pid=105533) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:52:59.211684 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-7769d5a4-c364-4f52-b190-a5a628a3a486 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:52:59.212160 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-7769d5a4-c364-4f52-b190-a5a628a3a486 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:52:59.212589 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-7769d5a4-c364-4f52-b190-a5a628a3a486 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:52:59.213339 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-7769d5a4-c364-4f52-b190-a5a628a3a486 admin admin] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:52:59.213787 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-7769d5a4-c364-4f52-b190-a5a628a3a486 admin admin] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:52:59.214203 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: DEBUG oslo_concurrency.lockutils [None req-7769d5a4-c364-4f52-b190-a5a628a3a486 admin admin] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105533) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:52:59.236127 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: INFO nova.api.openstack.requestlog [None req-7769d5a4-c364-4f52-b190-a5a628a3a486 admin admin] 10.222.0.42 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.90 time: 0.084319 Aug 28 00:52:59.236691 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: [pid: 105533|app: 0|req: 38/76] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Aug 28 00:52:59 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:53:57.317490 dv-cinder-master-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Aug 28 00:53:57.318088 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: Sun Aug 28 00:53:57 2022 - graceful shutdown triggered... Aug 28 00:53:57.318527 dv-cinder-master-3-iscsi devstack@n-api.service[105534]: Gracefully killing worker 2 (pid: 105534)... Aug 28 00:53:57.318888 dv-cinder-master-3-iscsi devstack@n-api.service[105533]: Gracefully killing worker 1 (pid: 105533)... Aug 28 00:53:58.318587 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: worker 1 buried after 1 seconds Aug 28 00:53:58.318587 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: worker 2 buried after 1 seconds Aug 28 00:53:58.318587 dv-cinder-master-3-iscsi devstack@n-api.service[105522]: goodbye to uWSGI. Aug 28 00:53:58.320223 dv-cinder-master-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Aug 28 00:53:58.320787 dv-cinder-master-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Aug 28 00:53:58.323667 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Aug 28 00:53:58.333616 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Aug 28 00:53:58.335977 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Aug 28 00:53:58 2022] *** Aug 28 00:53:58.335977 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 28 00:53:58.335977 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 28 00:53:58.335977 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: nodename: dv-cinder-master-3-iscsi Aug 28 00:53:58.335977 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: machine: x86_64 Aug 28 00:53:58.335977 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: clock source: unix Aug 28 00:53:58.335977 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: pcre jit disabled Aug 28 00:53:58.335977 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: detected number of CPU cores: 4 Aug 28 00:53:58.335977 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: current working directory: / Aug 28 00:53:58.335977 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: detected binary path: /usr/bin/uwsgi-core Aug 28 00:53:58.335977 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: your processes number limit is 47680 Aug 28 00:53:58.335977 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: your memory page size is 4096 bytes Aug 28 00:53:58.335977 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: detected max file descriptor number: 2048 Aug 28 00:53:58.335977 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: lock engine: pthread robust mutexes Aug 28 00:53:58.337031 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: thunder lock: enabled Aug 28 00:53:58.337031 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Aug 28 00:53:58.337031 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 28 00:53:58.376179 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: Python main interpreter initialized at 0x5566cc2989c0 Aug 28 00:53:58.376179 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: python threads support enabled Aug 28 00:53:58.376179 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: your server socket listen backlog is limited to 100 connections Aug 28 00:53:58.376179 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: your mercy for graceful operations on workers is 90 seconds Aug 28 00:53:58.376597 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: mapped 403077 bytes (393 KB) for 2 cores Aug 28 00:53:58.376597 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: *** Operational MODE: preforking *** Aug 28 00:53:58.376826 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: *** uWSGI is running in multiple interpreter mode *** Aug 28 00:53:58.376826 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: spawned uWSGI master process (pid: 120503) Aug 28 00:53:58.377040 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Aug 28 00:53:58.377814 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: spawned uWSGI worker 1 (pid: 120524, cores: 1) Aug 28 00:53:58.377957 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: spawned uWSGI worker 2 (pid: 120525, cores: 1) Aug 28 00:53:58.378393 dv-cinder-master-3-iscsi devstack@n-api.service[120503]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 28 00:53:59.032120 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 28 00:53:59.034741 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Aug 28 00:54:00.182439 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 28 00:54:00.182439 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: from cryptography.utils import int_from_bytes Aug 28 00:54:00.183240 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 28 00:54:00.183240 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: from cryptography.utils import int_from_bytes Aug 28 00:54:00.194943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 28 00:54:00.194943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: from cryptography.utils import int_from_bytes Aug 28 00:54:00.195577 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 28 00:54:00.195577 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: from cryptography.utils import int_from_bytes Aug 28 00:54:02.602705 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 28 00:54:02.603365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 28 00:54:02.603458 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 28 00:54:02.603813 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 28 00:54:02.604071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 28 00:54:02.604489 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.604817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.605168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.605473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.605940 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.606288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.606585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.606922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.607251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.607494 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.607792 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.608129 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.608437 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.608750 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.609116 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.609406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.609732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.610036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.610349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.610849 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.612542 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.612874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.613215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.613589 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.613880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.614207 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.614512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.614825 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.615157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.615473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.615890 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.616196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.616510 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.616816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.617285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.617605 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.617889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.618211 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.618522 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.618842 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.619158 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.619491 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.619803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.620123 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.620413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.620719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.621051 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.621369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.621656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.621972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.622202 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.622515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.622828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.623143 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.623457 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.623710 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.623951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.624271 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.624587 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.624874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.625208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.625512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.625814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.626127 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.626417 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.626719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.627019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.627337 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.627650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.627987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.628300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.628617 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.629070 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.42 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.629301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.42 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.629613 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.629954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.630267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.630581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.630880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.631198 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.631520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.631822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.632110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.632439 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.632766 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.633097 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.633347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.633668 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.633954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.634264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.634568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.634865 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.635163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.635451 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.635776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.636054 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.636362 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.636659 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.636949 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.637277 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.637636 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.637864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.638164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.638471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.638784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.639085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.639384 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.639682 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.639968 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.640288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.640642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.640895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.641228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.641519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.641835 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.642148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.642446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.642743 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.643046 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.643340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.643753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.644023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.644325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.644543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 28 00:54:02.645106 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.645270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 28 00:54:02.645424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.645424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.645746 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 28 00:54:02.645908 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.646072 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 28 00:54:02.646230 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.646376 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 28 00:54:02.646537 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.646683 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.646845 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.647009 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.647165 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.647320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.647482 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.647637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.647791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.647791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.648078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.648239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.648393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.648539 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.648704 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.648882 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.649049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.649211 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.649406 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.649566 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.649738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.649900 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.651315 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.651494 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.651644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.651803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.651957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.652122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.652301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.652460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.652616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.652785 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.652934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.653126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.653288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.653445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.653593 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.653745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.653900 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.654058 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.654058 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.654632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.654807 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.654991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.654991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.655296 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.655442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.655595 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.655748 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.655893 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.656045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.656216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.656411 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.656572 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.656727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.656880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.657054 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.657219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.657365 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.657516 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.657662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.657852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.657852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.658143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.658306 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.658452 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.658452 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.658760 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.658914 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.659070 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.659070 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.659428 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.659633 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.659795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.659941 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.660093 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.660258 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.660404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.660556 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.660734 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.662007 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.662170 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.662325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.662483 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.662629 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.662789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.662937 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.663089 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.663246 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.663448 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.663448 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.663741 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.663885 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.664039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.664193 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.665361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.665535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.665691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.665838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.665983 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.665983 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.666267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.666438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.666592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.666744 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.666890 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.667035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.667181 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.667334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.667479 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.667632 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.667776 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.667933 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.668087 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.668246 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.668391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.668553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.668709 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.668855 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.669055 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.669734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.669901 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.670058 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.670214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.670376 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.670529 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.670689 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.670835 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.671006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.671843 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.672002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.672149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.672292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.672292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.672643 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.672789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.672789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.672789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.673254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.673421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.673421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.673707 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.673907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.674058 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.674213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.674367 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.674643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.675176 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.675622 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.675785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.675938 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.675938 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.676228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.42 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.676384 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.676538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.42 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.676685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.676830 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.677084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.677251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.677407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.677564 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.677733 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.677879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.678024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.678180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.678412 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.678683 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.678852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.679002 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.679154 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.679316 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.679470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.679617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.679818 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.679976 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.680136 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.680292 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.680292 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.680594 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.680749 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.680902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.681083 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.681257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.681947 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.682113 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.682259 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.682405 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.682557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.682702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.682861 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.683020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.683173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.683327 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.683480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.683634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.683789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.683935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.684079 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.684223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.684374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.684519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.685026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.685195 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.685340 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.685493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.685840 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.686013 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.686168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.686320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.686473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.686617 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.686770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.686929 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.687082 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.687273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.687429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.687583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.687736 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.687894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.688046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.688192 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.688350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.688552 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.688712 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.688880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.689058 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.689208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.689366 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.689366 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.689687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.689881 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.690064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.690223 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.690377 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.690522 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.690666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.690810 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.690961 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.691120 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.691273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.691425 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.692084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.692247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.692403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.692562 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.692708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.692853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.693042 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.693203 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.693350 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.693507 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.693676 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.693824 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.693968 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.694169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.694353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.694518 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.694672 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.694826 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.695070 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.695231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.695390 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.695546 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.695692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.696048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.696048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.696352 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.696513 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.696660 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.696805 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.696949 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.697122 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.697268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.697428 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.697599 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.697776 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.697932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.698077 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 28 00:54:02.698222 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.698388 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.698542 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.698695 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.698850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.698850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.699142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.699293 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.699437 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.699696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.699844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.700038 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.700504 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.700651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.700814 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.701003 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.701157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.701314 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.701474 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.701620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.702289 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.702444 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.702599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.702754 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.702917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.703074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.703275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.703426 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.703580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.703734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.703955 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.704117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.704273 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.704419 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.704616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.704776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.704924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.705199 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.705359 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.705516 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.705784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.705967 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.706398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.706562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.706709 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.706854 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.707011 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.707167 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.707322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.707474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.707629 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.707784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.708026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.708183 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.708330 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.708482 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.708634 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.708779 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.708949 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.709123 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.709278 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.709424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.709576 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.709728 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.709888 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.710101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.710254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.710407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.710552 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.710696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.710849 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.710994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.710994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.711270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.711461 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.711629 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.711775 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.712460 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.712626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.712784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.712938 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.713119 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.713276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.713431 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.713585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.713731 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.713731 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.714025 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.714025 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.714312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.714475 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.714628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.714775 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.714934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.715079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.715331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.715485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.715631 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.715790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.715944 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.716097 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.716242 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.716630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.716802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.717031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.717185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.717332 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.717478 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.717640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.717794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.717948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.718094 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.718238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.718390 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.718544 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.718696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.718869 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.719023 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.719169 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.719321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.719466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.719612 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.719771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.719924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.720079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.720282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.720436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.720591 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.720738 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.720892 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.721060 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.721218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.721218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.721516 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.721671 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.721858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.722032 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.722032 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.722898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.723085 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.723233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.723400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.723555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.723709 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.723864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.724019 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.724019 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.724306 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.724465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.724610 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.724756 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.724909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.725118 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.725266 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.725546 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.725706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.725862 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.726016 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.726188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.726455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.726887 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.726991 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.727088 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.727176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.727250 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.727349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.727349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.727516 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.727602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.727688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.727780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.727880 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.727967 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.728052 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.728136 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.728220 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.728305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.728397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.728491 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.728578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.731094 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.731207 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.731307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.731417 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.731507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.731600 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.731694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.731788 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.731888 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.731975 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.731975 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.732130 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.732216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.732906 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.733095 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.733197 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.733286 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.733372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.733465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.733465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.733651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.733748 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.733748 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.733907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.733999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.734086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.734179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.734266 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.734359 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.734446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.734538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.734633 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.734727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.734727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.734885 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.734970 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.735062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.735162 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.735162 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.735369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.735464 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.735564 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.735651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.735651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.735822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.735915 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.736010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.736104 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.736199 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.736372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.736470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.736557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.736990 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.737094 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.737201 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.737298 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.737298 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.737590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.737590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.737590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.737590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.737590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.737590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.737590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.738052 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.738132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.738132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.738277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.738347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.738499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.738597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.738693 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.738788 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.738886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.738980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.738980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.738980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.738980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.739310 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.739403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.739403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.739403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.739403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.739690 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.739777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.739870 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.739964 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.739964 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.740132 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.740226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.740226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.740226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.740226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.740226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.740226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.740693 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.740780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.740780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.740780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.740780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.741168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.741265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.741265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.741418 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.741512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.741512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.741512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.741512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.741512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.741512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.741512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.742046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.742145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.742145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.742145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.742145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.742145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.742145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.743139 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.743238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.743238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.743238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.743238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.743561 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.743658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.743658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.743893 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.744004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.744004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.744004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.744265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.744411 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.744514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.744793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.745088 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.745204 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.745444 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.745693 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.745789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.746079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.746281 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.746391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.746701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.746885 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.747014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.747302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.747427 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.747608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.747918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.748011 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.748228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.748517 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.748626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.748841 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.749150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.749251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.749478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.749723 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.749831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.750081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.750313 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.750416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.750682 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.750957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.751125 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.751285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.751491 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.751597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.751895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.752066 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.752200 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.752501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.752907 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.753127 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.753220 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.753328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.753495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.753727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.753830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.754180 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.754276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.754493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.754792 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.754896 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.755095 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.755370 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.755471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.755700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.755946 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.756041 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.756313 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.756525 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.756628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.757048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.757181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.757473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.757697 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.757806 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.757949 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.758183 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.758287 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.758541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.758755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.758857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.759156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.759344 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.759440 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.759745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.759930 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.760051 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.760369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.760507 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.760672 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.761050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.761194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.761440 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.761592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.761734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.762065 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.762174 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.762475 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.762648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.763299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.763434 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.763540 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.763540 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.763715 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.763811 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.764035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.764247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.764360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.764654 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.764818 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.764985 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.765310 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.765414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.765654 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.765925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.766035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.766259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.766505 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.766604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.766888 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.767142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.767265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.767585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.767702 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.767815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.768152 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.768265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.768518 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.768789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.769022 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.769344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.769453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.769581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.769877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.769980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.770199 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.770494 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.770596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.770797 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.771084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.771191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.771427 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.771665 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.771767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.772033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.772250 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.772346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.772621 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.772821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.772925 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.773231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.773413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.773531 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.773849 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.773961 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.774176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.774464 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.774567 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.774782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.775059 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.775154 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.775389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.775628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.775725 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.775997 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.776203 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.776309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.776586 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.776816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.776910 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.777278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.777398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.777745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.777933 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.778135 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.778238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.778448 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.778585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.779366 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.779366 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.779547 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.779647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.779743 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.780019 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.780139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.780354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.780764 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.780888 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.781025 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.781208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.781351 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.781548 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.781837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.781948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.782154 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.782446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.782540 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.782749 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.783050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.783166 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.783398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.783641 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.783756 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.783950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.784320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.784429 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.784591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.784854 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.784995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.785228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.785450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.785576 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.785861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.786013 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.786165 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.786497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.786610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.786804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.787072 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.787175 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.787426 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.787650 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.788324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.788324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.788324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.788758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.788934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.789097 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.789271 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.789549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.789656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.789883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.790654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.790777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.790777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.790963 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.791060 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.791301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.791466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.791661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.791893 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.792037 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.792272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.792452 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.792565 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.792880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.793074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.793238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.793401 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.793554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.793732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.793921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.794056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.794171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.794471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.794655 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.794790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.794989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.795116 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.795306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.795453 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.795624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.795773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.795940 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.796085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.796295 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.796395 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.796609 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.796707 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.796994 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.797104 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.797313 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.797405 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.797652 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.797758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.797962 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.798063 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.798274 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.798372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.798584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.798684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.798889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.798995 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.799197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.799295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.799505 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.799602 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.799813 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.799960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.800174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.800273 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.800481 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.800568 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.800790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.800882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.801130 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.801239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.801446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.801536 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.801762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.801854 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.802070 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.802168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.802379 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.802490 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.802685 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.802784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.802993 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.803082 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.803382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.803525 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.803625 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.803843 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.803935 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.804180 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.804375 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.804638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.804794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.804951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.805149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.805311 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.805463 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.805608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.805757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.805757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.806039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.806039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.806322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.806476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.806617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.806759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.806898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.807059 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.807199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.807353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.807501 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.807723 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.807877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.808030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.808169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.808318 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.808480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.808720 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.808872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.809031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.809192 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.809192 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.809469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.809618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.809778 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.809917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.810056 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.810203 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.810359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.810498 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.810653 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.810795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.810953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.811102 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.811248 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.811387 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.811536 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.811692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.811842 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.811981 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.812120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.812275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.812416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.812556 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.812774 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.812933 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.813153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.813257 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.813346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.813440 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.813527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.813613 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.813894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.814011 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.814135 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.814228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.814441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.814625 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.814730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.814978 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.815171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.815338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.815514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.815672 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.815897 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.816069 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.816229 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.816394 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.816536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.816666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.816803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.817113 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.817351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.817478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.817706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.817826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.818006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.818170 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.818314 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.818524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.818830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.818949 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.819069 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.819299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.819417 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.819630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.819776 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.819982 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.820154 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.820300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.820444 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.820643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.820737 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.820892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.821002 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.821253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.821411 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.821598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.821737 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.821883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.822197 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.822330 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.822506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.822763 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.822902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.823056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.823198 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.823392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.823692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.823847 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.824021 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.824156 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.824325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.824476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.824656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.824809 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.824999 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.825416 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.825613 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.825712 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.825914 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.826005 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.826216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.826316 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.826540 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.826649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.826849 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.826974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.827156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.827299 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.827464 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.827623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.827774 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.828060 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.828153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.828373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.828470 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.828682 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.828942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.829079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.829176 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.829319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.829410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.829627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.829725 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.829933 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.830033 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.830255 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.830354 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.830545 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.830652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.830852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.830962 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.831366 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.831467 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.831591 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.831787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.832000 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.832098 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.832313 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.832411 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.832627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.832732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.832973 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.833072 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.833269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.833360 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.833581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.833684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.833888 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.833978 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.834193 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.834292 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.834503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.834610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.834826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.834982 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.835134 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.835306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.835433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.835627 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.835850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.835951 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.836159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.836248 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.836474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.836566 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.836792 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.836890 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.837136 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.837415 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.837518 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.837740 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.837831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.838056 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.838174 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.838375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.838486 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.838681 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.838781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.839006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.839124 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.839334 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.839448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.839658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.839775 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.839972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.840066 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.840272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.840371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.840574 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.840791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.840904 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.841198 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.841309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.841512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.841615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.841819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.841910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.842115 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.842230 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.842414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.842564 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.842721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.842880 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.843020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.843204 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.843321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.843526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.843624 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.843855 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.843954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.844186 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.844277 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.844488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.844584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.844787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.844878 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.845124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.845228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.845426 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.845517 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.845726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.845817 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.846024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.846123 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.846326 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.846424 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.846625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.846948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.847050 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.847249 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.847348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.847549 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.847649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.847849 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.847946 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.848179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.848290 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.848506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.848611 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.848818 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.848910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.849149 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.849248 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.849458 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.849558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.849775 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.849862 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.850081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.850172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.850426 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.850528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.850742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.850841 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.851054 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.851168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.851363 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.851467 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.851683 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.851782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.851988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.852090 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.852297 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.852399 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.852603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.852694 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.852909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.853035 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.853257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.853348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.853570 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.853660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.853893 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.853993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.854202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.854292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.854500 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.854589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.854819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.854919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.855125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.855215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.855437 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.855523 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.855748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.855838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.856054 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.856152 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.856375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.856472 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.856681 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.856779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.857017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.857112 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.857313 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.857412 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.857619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.857710 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.857924 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.858022 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.858240 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.858368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.858547 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.858651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.858851 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.858944 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.859165 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.859305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.859480 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.859582 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.859796 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.859911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.860119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.860235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.860421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.860515 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.860733 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.860833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.861074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.861178 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.861395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.861501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.861705 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.861802 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.862015 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.862122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.862335 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.862462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.862701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.862852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.863001 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.863112 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.863292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.863393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.863535 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120525) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 28 00:54:02.863628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.863916 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.864227 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.864536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.864883 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.865215 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.865549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.865862 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.866187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.866508 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.866827 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.867145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.867463 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.867785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.868105 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.868428 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.868745 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.869126 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.869924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.869924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.870644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.870985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.871309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.871632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.871954 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.872277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.872598 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.872922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.873279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.873599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.873921 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.874255 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.874586 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.874920 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.875262 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.875588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.875912 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.876236 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.876553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.876920 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.877261 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.877580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.877893 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.878268 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.878590 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.878905 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.879221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.879540 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.879860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.880178 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.880494 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.880812 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.881163 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.881493 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.881831 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.882164 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.882473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.882792 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.883116 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.883630 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.883952 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.884284 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.884617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.884945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.885290 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.885614 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.885946 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.886260 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.886562 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.886884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.887196 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.887505 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.887833 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.888145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.888451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.888779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.889119 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.889746 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.890099 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.890420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.890804 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.891080 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.891402 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.894936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.894936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.894936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.894936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.894936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.894936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.894936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.894936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.895610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.895610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.895610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.895610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.895610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.896013 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.896013 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.896324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.896634 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.896948 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.897276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.897587 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.897896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.898206 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.898522 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.898832 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.899145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.899453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.899778 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.900088 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.900403 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.900765 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.901113 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.901423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.901852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.902176 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.902492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.902818 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.903136 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.903457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.903813 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.904159 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.904710 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.905082 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.905411 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.905751 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.906089 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.906404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.906715 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.907026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.907359 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.907677 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.907994 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.908313 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.908631 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.908948 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.909356 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.909684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.910003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.910318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.910631 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.910949 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.911266 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.911577 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.911892 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.912212 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.912559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.912882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.913230 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.913555 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.913879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.914191 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.914505 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.914816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.915127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.915446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.915780 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.916100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.916423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.916839 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.917180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.917506 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.917834 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.918150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.918476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.918803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.919121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.919451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.919770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.920099 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.920446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.920774 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.921127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.921439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.921746 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.922053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.922368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.922692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.923008 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.923316 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.923413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO dbcounter [None req-46131d99-137f-4070-929e-a469a9f83846 None None] Registered counter for database nova_api Aug 28 00:54:02.923624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.923940 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.924248 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.924557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.924906 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.925244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.925557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.925865 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.926173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.926479 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.926789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.927098 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.927404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.927715 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.928022 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.928331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.928638 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.928946 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.929279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.929622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.929956 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.930279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.930602 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.930919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.931245 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.931559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.931884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.932207 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.932530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.932845 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.933216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.933528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.933846 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.934159 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.934480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.934790 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.935104 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.935435 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.935749 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.936058 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.936385 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.936704 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.937128 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.937453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.937769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.938102 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.938424 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.938736 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.939531 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.939867 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.940169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.940506 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.940855 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.941198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.941523 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.941844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.942176 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.942505 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.942823 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.943150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.943484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.943837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.944155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.944340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_db.sqlalchemy.engines [None req-46131d99-137f-4070-929e-a469a9f83846 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120525) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 28 00:54:02.944621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.944826 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.945186 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.945507 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.945820 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 28 00:54:02.946066 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120524) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 28 00:54:02.951053 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writer thread running {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 28 00:54:03.006898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.context [None req-46131d99-137f-4070-929e-a469a9f83846 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),0cad982d-83ea-40cf-82f0-5c4394803725(cell1) {{(pid=120525) load_cells /opt/stack/nova/nova/context.py:464}} Aug 28 00:54:03.008462 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO dbcounter [None req-bd4c9f52-4e47-4f75-be76-e355a47ec826 None None] Registered counter for database nova_api Aug 28 00:54:03.010976 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46131d99-137f-4070-929e-a469a9f83846 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:03.011449 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46131d99-137f-4070-929e-a469a9f83846 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:03.013092 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46131d99-137f-4070-929e-a469a9f83846 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:03.013925 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46131d99-137f-4070-929e-a469a9f83846 None None] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:03.014369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46131d99-137f-4070-929e-a469a9f83846 None None] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:03.016420 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46131d99-137f-4070-929e-a469a9f83846 None None] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:03.018522 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO dbcounter [None req-46131d99-137f-4070-929e-a469a9f83846 None None] Registered counter for database nova_cell0 Aug 28 00:54:03.023022 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO dbcounter [None req-46131d99-137f-4070-929e-a469a9f83846 None None] Registered counter for database nova_cell1 Aug 28 00:54:03.030169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_db.sqlalchemy.engines [None req-bd4c9f52-4e47-4f75-be76-e355a47ec826 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120524) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 28 00:54:03.033024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_db.sqlalchemy.engines [None req-46131d99-137f-4070-929e-a469a9f83846 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120525) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 28 00:54:03.034172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_db.sqlalchemy.engines [None req-46131d99-137f-4070-929e-a469a9f83846 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120525) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 28 00:54:03.038403 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writer thread running {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 28 00:54:03.040247 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writer thread running {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 28 00:54:03.042739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writer thread running {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 28 00:54:03.095228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.context [None req-bd4c9f52-4e47-4f75-be76-e355a47ec826 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),0cad982d-83ea-40cf-82f0-5c4394803725(cell1) {{(pid=120524) load_cells /opt/stack/nova/nova/context.py:464}} Aug 28 00:54:03.099882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd4c9f52-4e47-4f75-be76-e355a47ec826 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:03.100353 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd4c9f52-4e47-4f75-be76-e355a47ec826 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:03.101982 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd4c9f52-4e47-4f75-be76-e355a47ec826 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:03.102723 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd4c9f52-4e47-4f75-be76-e355a47ec826 None None] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:03.103145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd4c9f52-4e47-4f75-be76-e355a47ec826 None None] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:03.105273 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd4c9f52-4e47-4f75-be76-e355a47ec826 None None] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:03.107346 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO dbcounter [None req-bd4c9f52-4e47-4f75-be76-e355a47ec826 None None] Registered counter for database nova_cell0 Aug 28 00:54:03.112110 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO dbcounter [None req-bd4c9f52-4e47-4f75-be76-e355a47ec826 None None] Registered counter for database nova_cell1 Aug 28 00:54:03.120194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_db.sqlalchemy.engines [None req-bd4c9f52-4e47-4f75-be76-e355a47ec826 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120524) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 28 00:54:03.122205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_db.sqlalchemy.engines [None req-bd4c9f52-4e47-4f75-be76-e355a47ec826 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120524) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 28 00:54:03.129433 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writer thread running {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 28 00:54:03.131221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writer thread running {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 28 00:54:03.194831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO dbcounter [None req-46131d99-137f-4070-929e-a469a9f83846 None None] Registered counter for database nova_cell0 Aug 28 00:54:03.202356 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_db.sqlalchemy.engines [None req-46131d99-137f-4070-929e-a469a9f83846 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120525) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 28 00:54:03.207328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writer thread running {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 28 00:54:03.283868 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO dbcounter [None req-bd4c9f52-4e47-4f75-be76-e355a47ec826 None None] Registered counter for database nova_cell0 Aug 28 00:54:03.292499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_db.sqlalchemy.engines [None req-bd4c9f52-4e47-4f75-be76-e355a47ec826 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120524) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 28 00:54:03.297982 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writer thread running {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 28 00:54:03.528298 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: WARNING keystonemiddleware.auth_token [None req-46131d99-137f-4070-929e-a469a9f83846 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 28 00:54:03.535684 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: WARNING keystonemiddleware.auth_token [None req-46131d99-137f-4070-929e-a469a9f83846 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 28 00:54:03.641519 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: WARNING keystonemiddleware.auth_token [None req-bd4c9f52-4e47-4f75-be76-e355a47ec826 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 28 00:54:03.648908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: WARNING keystonemiddleware.auth_token [None req-bd4c9f52-4e47-4f75-be76-e355a47ec826 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 28 00:54:03.884278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: WARNING keystonemiddleware.auth_token [None req-46131d99-137f-4070-929e-a469a9f83846 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 28 00:54:03.890696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: WARNING keystonemiddleware.auth_token [None req-46131d99-137f-4070-929e-a469a9f83846 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 28 00:54:03.896291 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5566cc2989c0 pid: 120525 (default app) Aug 28 00:54:04.001609 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: WARNING keystonemiddleware.auth_token [None req-bd4c9f52-4e47-4f75-be76-e355a47ec826 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Aug 28 00:54:04.007823 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: WARNING keystonemiddleware.auth_token [None req-bd4c9f52-4e47-4f75-be76-e355a47ec826 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Aug 28 00:54:04.013276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5566cc2989c0 pid: 120524 (default app) Aug 28 00:54:39.235701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=3 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:54:39.237034 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=3 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:54:39.237964 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=3 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:54:39.239638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=3 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:54:40.576691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-73ea46e1-a561-457c-bb77-24c82be9370f tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Action: 'create', calling method: .version_select at 0x7f0e6f89b940>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1374583581"}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:54:43.301083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-73ea46e1-a561-457c-bb77-24c82be9370f tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 4.074781 Aug 28 00:54:43.301726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1/1] 10.222.0.42 () {60 vars in 1155 bytes} [Sun Aug 28 00:54:39 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 4078 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Aug 28 00:54:43.320178 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=3 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:54:43.321543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=3 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:54:43.322383 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=3 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:54:43.325044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=3 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:54:43.633163 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cda225d1-9e83-4daa-994e-41eb78199c85 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e6f808d30>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:44.084217 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cda225d1-9e83-4daa-994e-41eb78199c85 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/5b7d8f79-89de-4cfd-9d3b-7630b7661554" status: 200 len: 345 microversion: 2.1 time: 0.771058 Aug 28 00:54:44.084829 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1/2] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Aug 28 00:54:43 2022] GET /compute/v2.1/os-volumes/5b7d8f79-89de-4cfd-9d3b-7630b7661554 => generated 345 bytes in 773 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:54:45.098576 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-50dd6686-263b-442e-aadf-045c21661b8e tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e6f69ae50>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:45.200057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-50dd6686-263b-442e-aadf-045c21661b8e tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/5b7d8f79-89de-4cfd-9d3b-7630b7661554" status: 200 len: 346 microversion: 2.1 time: 0.105586 Aug 28 00:54:45.200618 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Aug 28 00:54:45 2022] GET /compute/v2.1/os-volumes/5b7d8f79-89de-4cfd-9d3b-7630b7661554 => generated 346 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:54:45.225884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-89147d98-9154-4aae-ad94-87dfd893052d tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Action: 'create', calling method: .version_select at 0x7f0e6f6dddc0>, body: {"snapshot": {"volume_id": "5b7d8f79-89de-4cfd-9d3b-7630b7661554", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-1753141856"}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:54:45.586690 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-89147d98-9154-4aae-ad94-87dfd893052d tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.375840 Aug 28 00:54:45.587482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 2/4] 10.222.0.42 () {60 vars in 1162 bytes} [Sun Aug 28 00:54:45 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:54:45.605887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-682988be-9300-4452-8497-9eaf5080f6f2 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e6f6bc550>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:45.668947 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-682988be-9300-4452-8497-9eaf5080f6f2 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349" status: 200 len: 287 microversion: 2.1 time: 0.072248 Aug 28 00:54:45.669531 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 3/5] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Aug 28 00:54:45 2022] GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349 => generated 287 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:54:45.689763 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e590914b-d66b-4653-87dd-377d92369995 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e6f6a3b80>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:45.762153 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e590914b-d66b-4653-87dd-377d92369995 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.084228 Aug 28 00:54:45.762719 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 3/6] 10.222.0.42 () {58 vars in 1140 bytes} [Sun Aug 28 00:54:45 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:54:45.773736 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-68a91c3f-85a0-4d98-854f-73ca3e8a9adb tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e6f6bcca0>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:45.836919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-68a91c3f-85a0-4d98-854f-73ca3e8a9adb tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349" status: 200 len: 287 microversion: 2.1 time: 0.066993 Aug 28 00:54:45.837884 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Aug 28 00:54:45 2022] GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349 => generated 287 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:54:46.516510 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2dd12f31-7570-4b38-afec-e399ae028d27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-384999750", "imageRef": "2c2585b2-bde5-4652-aed7-9f53b1bd5d94", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:54:46.705585 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.network.neutron [None req-2dd12f31-7570-4b38-afec-e399ae028d27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] validate_networks() for [('92aee389-59bb-4ec3-9e65-fe3bcb195f1a', None, None, None, None, None)] {{(pid=120524) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 28 00:54:46.851570 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-109ca75a-d47f-4dbb-a7ff-397b1f2fc122 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e6f6bcb80>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:46.916529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-109ca75a-d47f-4dbb-a7ff-397b1f2fc122 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349" status: 200 len: 287 microversion: 2.1 time: 0.069935 Aug 28 00:54:46.917538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 5/8] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Aug 28 00:54:46 2022] GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349 => generated 287 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:54:47.929280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-42271104-2114-4137-a61a-31a065eff70c tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e6f6bc430>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:48.062790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-42271104-2114-4137-a61a-31a065eff70c tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349" status: 200 len: 287 microversion: 2.1 time: 0.135463 Aug 28 00:54:48.063848 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 6/9] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Aug 28 00:54:47 2022] GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349 => generated 287 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:54:48.633808 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-2dd12f31-7570-4b38-afec-e399ae028d27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Flavor limits 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 28 00:54:48.635740 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-2dd12f31-7570-4b38-afec-e399ae028d27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Image limits 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 28 00:54:48.636588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-2dd12f31-7570-4b38-afec-e399ae028d27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Flavor pref 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 28 00:54:48.637225 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-2dd12f31-7570-4b38-afec-e399ae028d27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Image pref 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 28 00:54:48.637802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-2dd12f31-7570-4b38-afec-e399ae028d27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 28 00:54:48.662476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2dd12f31-7570-4b38-afec-e399ae028d27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:48.662971 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2dd12f31-7570-4b38-afec-e399ae028d27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:48.663408 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2dd12f31-7570-4b38-afec-e399ae028d27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:48.665931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2dd12f31-7570-4b38-afec-e399ae028d27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:48.666585 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2dd12f31-7570-4b38-afec-e399ae028d27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:48.667178 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2dd12f31-7570-4b38-afec-e399ae028d27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:48.714250 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-2dd12f31-7570-4b38-afec-e399ae028d27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Getting quotas for project 6f3acee4af62483ab1f2cbf3a47b09e1. Resources: {'ram', 'instances', 'cores'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 00:54:48.729131 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-2dd12f31-7570-4b38-afec-e399ae028d27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Getting quotas for user 1ab2fcb3759045c7a076e5c8222a421d and project 6f3acee4af62483ab1f2cbf3a47b09e1. Resources: {'ram', 'instances', 'cores'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 00:54:48.748879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2dd12f31-7570-4b38-afec-e399ae028d27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Going to run 1 instances... {{(pid=120524) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 28 00:54:48.754689 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2dd12f31-7570-4b38-afec-e399ae028d27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2c2585b2-bde5-4652-aed7-9f53b1bd5d94',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120524) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 28 00:54:48.961304 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2dd12f31-7570-4b38-afec-e399ae028d27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.552487 Aug 28 00:54:48.962570 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 4/10] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Aug 28 00:54:46 2022] POST /compute/v2.1/servers => generated 383 bytes in 2555 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 28 00:54:48.975730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-08d6dcba-3519-45d9-a8c9-ef3d5563f19c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:48.976796 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-08d6dcba-3519-45d9-a8c9-ef3d5563f19c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:54:49.078190 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fcb9f5e8-b989-4855-9d8d-3ad8fcbc0ff0 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e6f5baa60>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:49.193851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fcb9f5e8-b989-4855-9d8d-3ad8fcbc0ff0 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349" status: 200 len: 287 microversion: 2.1 time: 0.121779 Aug 28 00:54:49.194444 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 5/11] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Aug 28 00:54:49 2022] GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349 => generated 287 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:54:49.232917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-08d6dcba-3519-45d9-a8c9-ef3d5563f19c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:54:49.278373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-08d6dcba-3519-45d9-a8c9-ef3d5563f19c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1221 microversion: 2.1 time: 0.307994 Aug 28 00:54:49.278971 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 7/12] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:54:48 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1221 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:54:49.375470 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-15107f5f-4687-4a96-a786-d6eb0f040b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1567649210", "imageRef": "2c2585b2-bde5-4652-aed7-9f53b1bd5d94", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:54:49.525311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.network.neutron [None req-15107f5f-4687-4a96-a786-d6eb0f040b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] validate_networks() for [('92aee389-59bb-4ec3-9e65-fe3bcb195f1a', None, None, None, None, None)] {{(pid=120524) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 28 00:54:50.207732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-020ef095-b434-4b2d-abac-18129fdad6c6 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e6f6bc790>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:50.271858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-020ef095-b434-4b2d-abac-18129fdad6c6 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349" status: 200 len: 287 microversion: 2.1 time: 0.068728 Aug 28 00:54:50.272428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 8/13] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Aug 28 00:54:50 2022] GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349 => generated 287 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:54:51.120257 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-15107f5f-4687-4a96-a786-d6eb0f040b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Flavor limits 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 28 00:54:51.120834 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-15107f5f-4687-4a96-a786-d6eb0f040b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Image limits 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 28 00:54:51.121894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-15107f5f-4687-4a96-a786-d6eb0f040b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Flavor pref 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 28 00:54:51.122316 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-15107f5f-4687-4a96-a786-d6eb0f040b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Image pref 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 28 00:54:51.122738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-15107f5f-4687-4a96-a786-d6eb0f040b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 28 00:54:51.144469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-15107f5f-4687-4a96-a786-d6eb0f040b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:51.145439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-15107f5f-4687-4a96-a786-d6eb0f040b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:51.150059 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-15107f5f-4687-4a96-a786-d6eb0f040b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:51.159905 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-15107f5f-4687-4a96-a786-d6eb0f040b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:51.160431 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-15107f5f-4687-4a96-a786-d6eb0f040b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:51.160844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-15107f5f-4687-4a96-a786-d6eb0f040b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:51.244481 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-15107f5f-4687-4a96-a786-d6eb0f040b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Getting quotas for project 6f3acee4af62483ab1f2cbf3a47b09e1. Resources: {'ram', 'instances', 'cores'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 00:54:51.255899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-15107f5f-4687-4a96-a786-d6eb0f040b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Getting quotas for user 1ab2fcb3759045c7a076e5c8222a421d and project 6f3acee4af62483ab1f2cbf3a47b09e1. Resources: {'ram', 'instances', 'cores'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 00:54:51.281909 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-15107f5f-4687-4a96-a786-d6eb0f040b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Going to run 1 instances... {{(pid=120524) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 28 00:54:51.284194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-15107f5f-4687-4a96-a786-d6eb0f040b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2c2585b2-bde5-4652-aed7-9f53b1bd5d94',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120524) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 28 00:54:51.286006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8008b683-a951-4f92-8ab3-924f5e0312dd tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e6f608e50>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:51.347237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8008b683-a951-4f92-8ab3-924f5e0312dd tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349" status: 200 len: 287 microversion: 2.1 time: 0.066310 Aug 28 00:54:51.348058 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 9/14] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Aug 28 00:54:51 2022] GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349 => generated 287 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:54:51.400014 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-15107f5f-4687-4a96-a786-d6eb0f040b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.028844 Aug 28 00:54:51.403748 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 6/15] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Aug 28 00:54:49 2022] POST /compute/v2.1/servers => generated 383 bytes in 2033 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 28 00:54:51.414332 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9dba53e4-ab07-47a4-b617-75ddbf3f421d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:51.415103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9dba53e4-ab07-47a4-b617-75ddbf3f421d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:54:51.543703 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9dba53e4-ab07-47a4-b617-75ddbf3f421d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:54:51.566392 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9dba53e4-ab07-47a4-b617-75ddbf3f421d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1222 microversion: 2.1 time: 0.155734 Aug 28 00:54:51.567072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 10/16] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:54:51 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1222 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:54:52.360351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-62b8d103-ca53-4e6d-a735-b80309b6a600 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e6ed55550>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:52.467855 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-62b8d103-ca53-4e6d-a735-b80309b6a600 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349" status: 200 len: 288 microversion: 2.1 time: 0.111642 Aug 28 00:54:52.468736 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 7/17] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Aug 28 00:54:52 2022] GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349 => generated 288 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:54:52.480829 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0a2d8a1d-a12b-4808-9ca9-4c3097d2f061 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e6f6173a0>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:52.588628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3d1ee646-54f4-468a-a009-a9294a6db035 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:52.589637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3d1ee646-54f4-468a-a009-a9294a6db035 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:54:52.642726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0a2d8a1d-a12b-4808-9ca9-4c3097d2f061 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "DELETE /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349" status: 202 len: 0 microversion: 2.1 time: 0.166105 Aug 28 00:54:52.643300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 11/18] 10.222.0.42 () {58 vars in 1254 bytes} [Sun Aug 28 00:54:52 2022] DELETE /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349 => generated 0 bytes in 167 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 00:54:52.656851 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5c6b46b7-206c-4a9e-b1ef-1d9006edca3f tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e6f617a60>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:52.757392 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5c6b46b7-206c-4a9e-b1ef-1d9006edca3f tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349" status: 200 len: 287 microversion: 2.1 time: 0.109413 Aug 28 00:54:52.757952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 12/19] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Aug 28 00:54:52 2022] GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349 => generated 287 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:54:52.843588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3d1ee646-54f4-468a-a009-a9294a6db035 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:54:52.875658 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3d1ee646-54f4-468a-a009-a9294a6db035 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1222 microversion: 2.1 time: 0.289690 Aug 28 00:54:52.876869 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 8/20] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:54:52 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1222 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:54:53.633591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6ddbe936-f029-4664-8232-1ecb354f3628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1234765925", "imageRef": "2c2585b2-bde5-4652-aed7-9f53b1bd5d94", "flavorRef": "452", "networks": [{"uuid": "0f715ce2-4336-47fc-aabf-469c0d6da07a"}]}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:54:53.770109 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0fd356e0-560c-43d9-8ed0-93162b91afe5 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e6ed55040>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:53.883618 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.network.neutron [None req-6ddbe936-f029-4664-8232-1ecb354f3628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] validate_networks() for [('0f715ce2-4336-47fc-aabf-469c0d6da07a', None, None, None, None, None)] {{(pid=120525) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 28 00:54:53.885743 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0fd356e0-560c-43d9-8ed0-93162b91afe5 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349" status: 200 len: 287 microversion: 2.1 time: 0.119485 Aug 28 00:54:53.886298 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 9/21] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Aug 28 00:54:53 2022] GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349 => generated 287 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:54:53.903227 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-aa79a529-1a36-4784-af15-e3a135b169e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:53.905185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-aa79a529-1a36-4784-af15-e3a135b169e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:54:53.940546 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aa79a529-1a36-4784-af15-e3a135b169e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:53.941248 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aa79a529-1a36-4784-af15-e3a135b169e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:53.945299 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aa79a529-1a36-4784-af15-e3a135b169e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:53.946093 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aa79a529-1a36-4784-af15-e3a135b169e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:53.946566 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aa79a529-1a36-4784-af15-e3a135b169e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:53.946987 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aa79a529-1a36-4784-af15-e3a135b169e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:54.317605 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-aa79a529-1a36-4784-af15-e3a135b169e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:54:54.354387 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-aa79a529-1a36-4784-af15-e3a135b169e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1226 microversion: 2.1 time: 0.458211 Aug 28 00:54:54.355772 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 10/22] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:54:53 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1226 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:54:54.476933 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-86f1928d-0255-456e-b2e6-fc15316165f2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Action: 'create', calling method: .version_select at 0x7f0e6ecea550>, body: {"keypair": {"name": "tempest-keypair-1836615983", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHGF9opNDfVjqy0Qxb7oZ0d3SUimgN2X1imeMfWYbGMvqsed+jNPGDvga3N83B0ZLc1jjlWv1Ocvet46maQLCoiR7DuXJRW71JYgv7SfL0WopPr/Nl9c7dq1ojVm0Ki2Eg=="}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:54:54.529355 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-86f1928d-0255-456e-b2e6-fc15316165f2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Getting quotas for project 95df81e14a8549fca0851234f4a00b89. Resources: {'key_pairs'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 00:54:54.534837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-86f1928d-0255-456e-b2e6-fc15316165f2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Getting quotas for user d209174d0db04b3584556824c6d0a3d4 and project 95df81e14a8549fca0851234f4a00b89. Resources: {'key_pairs'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 00:54:54.592591 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-86f1928d-0255-456e-b2e6-fc15316165f2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.234822 Aug 28 00:54:54.594412 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 11/23] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Aug 28 00:54:54 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:54:54.898159 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-14d62861-27b6-4461-8ce8-119cd2028a20 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e6ecf90d0>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:54.964895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-14d62861-27b6-4461-8ce8-119cd2028a20 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349" status: 200 len: 287 microversion: 2.1 time: 0.070717 Aug 28 00:54:54.965938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 12/24] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Aug 28 00:54:54 2022] GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349 => generated 287 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:54:55.379400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-359bc05f-759d-466c-a23b-5f88a9ae15e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:55.380220 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-359bc05f-759d-466c-a23b-5f88a9ae15e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:54:55.388311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-359bc05f-759d-466c-a23b-5f88a9ae15e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:55.389212 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-359bc05f-759d-466c-a23b-5f88a9ae15e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:55.391276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-359bc05f-759d-466c-a23b-5f88a9ae15e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:55.392871 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-359bc05f-759d-466c-a23b-5f88a9ae15e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:55.394054 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-359bc05f-759d-466c-a23b-5f88a9ae15e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:55.394636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-359bc05f-759d-466c-a23b-5f88a9ae15e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:55.525306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-359bc05f-759d-466c-a23b-5f88a9ae15e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:54:55.544583 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-359bc05f-759d-466c-a23b-5f88a9ae15e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1226 microversion: 2.1 time: 0.170801 Aug 28 00:54:55.548398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 13/25] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:54:55 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1226 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:54:55.664776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-6ddbe936-f029-4664-8232-1ecb354f3628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Flavor limits 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 28 00:54:55.665415 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-6ddbe936-f029-4664-8232-1ecb354f3628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Image limits 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 28 00:54:55.665839 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-6ddbe936-f029-4664-8232-1ecb354f3628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Flavor pref 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 28 00:54:55.666247 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-6ddbe936-f029-4664-8232-1ecb354f3628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Image pref 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 28 00:54:55.666654 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-6ddbe936-f029-4664-8232-1ecb354f3628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 28 00:54:55.694182 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ddbe936-f029-4664-8232-1ecb354f3628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:55.694712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ddbe936-f029-4664-8232-1ecb354f3628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:55.695153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ddbe936-f029-4664-8232-1ecb354f3628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:55.695947 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ddbe936-f029-4664-8232-1ecb354f3628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:55.700843 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ddbe936-f029-4664-8232-1ecb354f3628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:55.701551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ddbe936-f029-4664-8232-1ecb354f3628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:55.754925 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-6ddbe936-f029-4664-8232-1ecb354f3628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Getting quotas for project d36dd9ee223f496ebc7b6de74b92227a. Resources: {'ram', 'instances', 'cores'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 00:54:55.765347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-6ddbe936-f029-4664-8232-1ecb354f3628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Getting quotas for user dd49314743b14266a6a646ffd4d26698 and project d36dd9ee223f496ebc7b6de74b92227a. Resources: {'ram', 'instances', 'cores'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 00:54:55.811017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6ddbe936-f029-4664-8232-1ecb354f3628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Going to run 1 instances... {{(pid=120525) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 28 00:54:55.821068 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6ddbe936-f029-4664-8232-1ecb354f3628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2c2585b2-bde5-4652-aed7-9f53b1bd5d94',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120525) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 28 00:54:55.981192 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1d3df54c-4e08-4890-bf0b-680c3286ccc1 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e6ecf94c0>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:56.001320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6ddbe936-f029-4664-8232-1ecb354f3628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.470393 Aug 28 00:54:56.006020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 13/26] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Aug 28 00:54:53 2022] POST /compute/v2.1/servers => generated 380 bytes in 2487 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 28 00:54:56.021264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-13ab04e1-32c5-45f0-9f51-2a509119fec2 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:56.021959 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-13ab04e1-32c5-45f0-9f51-2a509119fec2 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:54:56.092186 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1d3df54c-4e08-4890-bf0b-680c3286ccc1 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349" status: 200 len: 287 microversion: 2.1 time: 0.117652 Aug 28 00:54:56.092764 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 14/27] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Aug 28 00:54:55 2022] GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349 => generated 287 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:54:56.297175 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-13ab04e1-32c5-45f0-9f51-2a509119fec2 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:54:56.316840 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-13ab04e1-32c5-45f0-9f51-2a509119fec2 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1215 microversion: 2.1 time: 0.302293 Aug 28 00:54:56.333018 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 14/28] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:54:56 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1215 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:54:56.568441 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9eb7290f-e3f8-4fc6-a1dc-d666e30ec83c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:56.569128 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9eb7290f-e3f8-4fc6-a1dc-d666e30ec83c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:54:56.583492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9eb7290f-e3f8-4fc6-a1dc-d666e30ec83c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:56.583985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9eb7290f-e3f8-4fc6-a1dc-d666e30ec83c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:56.584419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9eb7290f-e3f8-4fc6-a1dc-d666e30ec83c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:56.601625 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9eb7290f-e3f8-4fc6-a1dc-d666e30ec83c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:56.602295 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9eb7290f-e3f8-4fc6-a1dc-d666e30ec83c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:56.602745 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9eb7290f-e3f8-4fc6-a1dc-d666e30ec83c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:56.991743 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9eb7290f-e3f8-4fc6-a1dc-d666e30ec83c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:54:56.999934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9eb7290f-e3f8-4fc6-a1dc-d666e30ec83c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1218 microversion: 2.1 time: 0.435208 Aug 28 00:54:57.007254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 15/29] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:54:56 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1218 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:54:57.103649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2dce1257-8480-49f6-9be7-bd736c678fa3 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e6ed05b80>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:57.167117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2dce1257-8480-49f6-9be7-bd736c678fa3 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349" status: 200 len: 287 microversion: 2.1 time: 0.067251 Aug 28 00:54:57.167871 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 15/30] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Aug 28 00:54:57 2022] GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349 => generated 287 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:54:57.346396 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-34da79ee-6b09-421c-a573-0f60a183a6ed tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:57.347130 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-34da79ee-6b09-421c-a573-0f60a183a6ed tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:54:57.386674 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-34da79ee-6b09-421c-a573-0f60a183a6ed tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:57.388347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-34da79ee-6b09-421c-a573-0f60a183a6ed tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:57.390419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-34da79ee-6b09-421c-a573-0f60a183a6ed tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:57.393810 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-34da79ee-6b09-421c-a573-0f60a183a6ed tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:57.395126 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-34da79ee-6b09-421c-a573-0f60a183a6ed tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:57.399366 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-34da79ee-6b09-421c-a573-0f60a183a6ed tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:57.814870 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-34da79ee-6b09-421c-a573-0f60a183a6ed tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:54:57.831811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-34da79ee-6b09-421c-a573-0f60a183a6ed tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1211 microversion: 2.1 time: 0.495394 Aug 28 00:54:57.842150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 16/31] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:54:57 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1211 bytes in 507 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:54:58.021854 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1ea703a1-07f5-431d-b834-b7e63bb2bed1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:58.022635 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1ea703a1-07f5-431d-b834-b7e63bb2bed1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:54:58.038621 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ea703a1-07f5-431d-b834-b7e63bb2bed1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:58.041623 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ea703a1-07f5-431d-b834-b7e63bb2bed1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:58.042214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ea703a1-07f5-431d-b834-b7e63bb2bed1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:58.044280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ea703a1-07f5-431d-b834-b7e63bb2bed1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:58.047703 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ea703a1-07f5-431d-b834-b7e63bb2bed1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:58.049331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ea703a1-07f5-431d-b834-b7e63bb2bed1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:58.180650 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f72ffdf2-3d41-4eef-904d-f28eb93a41f0 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e6ec833a0>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:58.238398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-f72ffdf2-3d41-4eef-904d-f28eb93a41f0 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] HTTP exception thrown: Snapshot 2d2542cc-6d20-448f-a7ca-eed2dafeb349 could not be found. Aug 28 00:54:58.243199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f72ffdf2-3d41-4eef-904d-f28eb93a41f0 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Request method failure captured: Aug 28 00:54:58.243199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349 HTTP/1.1 Aug 28 00:54:58.243199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 00:54:58.243199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 00:54:58.243199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 00:54:58.243199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 00:54:58.243199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 00:54:58.243199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 00:54:58.243199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 00:54:58.243199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 00:54:58.243199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCrzPi1vubP_Bcx_QX_wsKOT42eEwNnnuXc4cC6ZypEV9locCcwosJTNEmFnMt-z9BKEtizzMGm4YsAwyQPhxqB7U0lUrJCGZqwgdEhWhIfTkkVLVLdffh1RigaCc2BpsqY1q6Gv3RYrg5hAPRe1mb1hWPaoP-r2RqX632pE7KcZVMHw Aug 28 00:54:58.243199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 00:54:58.243199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 00:54:58.243199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 00:54:58.243199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 00:54:58.243199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 00:54:58.243199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 00:54:58.243199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: c22d5ee0e87f478995d585f2b567542a Aug 28 00:54:58.243199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1155394448 Aug 28 00:54:58.243199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 00:54:58.243199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 00:54:58.244476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_c22d5ee0e87f478995d585f2b567542a", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/c22d5ee0e87f478995d585f2b567542a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/c22d5ee0e87f478995d585f2b567542a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/c22d5ee0e87f478995d585f2b567542a"}]}] Aug 28 00:54:58.244476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1155394448 Aug 28 00:54:58.244476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: c22d5ee0e87f478995d585f2b567542a Aug 28 00:54:58.244476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1155394448 Aug 28 00:54:58.244476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-VolumesSnapshotsTestJSON-1155394448-project Aug 28 00:54:58.244476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 00:54:58.244476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 00:54:58.244476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: ab1089ce15ea4dcb8052e0e647af4945 Aug 28 00:54:58.244476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1155394448-project Aug 28 00:54:58.244476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: Aug 28 00:54:58.244476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Snapshot 2d2542cc-6d20-448f-a7ca-eed2dafeb349 could not be found. Aug 28 00:54:58.244476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'id': '2d2542cc-6d20-448f-a7ca-eed2dafeb349'} Aug 28 00:54:58.244476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 00:54:58.245575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:54:58.245575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Aug 28 00:54:58.245575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Aug 28 00:54:58.245575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Aug 28 00:54:58.245575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Aug 28 00:54:58.245575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Aug 28 00:54:58.245575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Aug 28 00:54:58.245575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 28 00:54:58.245575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 28 00:54:58.245575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 28 00:54:58.245575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 28 00:54:58.245575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 00:54:58.245575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 00:54:58.245575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 00:54:58.245575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 00:54:58.245575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 2d2542cc-6d20-448f-a7ca-eed2dafeb349 could not be found. (HTTP 404) (Request-ID: req-af197974-da4e-40d6-b1c4-bd6f780a8004) Aug 28 00:54:58.245575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:54:58.245575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:54:58.246492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:54:58.246492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:54:58.246492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Aug 28 00:54:58.246492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Aug 28 00:54:58.246492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 00:54:58.246492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 00:54:58.246492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Aug 28 00:54:58.246492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Aug 28 00:54:58.246492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 28 00:54:58.246492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 28 00:54:58.246492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Aug 28 00:54:58.246492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Aug 28 00:54:58.246492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Aug 28 00:54:58.246492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Aug 28 00:54:58.246492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Aug 28 00:54:58.246492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Aug 28 00:54:58.246492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 28 00:54:58.246492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 28 00:54:58.246492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 28 00:54:58.247420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 28 00:54:58.247420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 00:54:58.247420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 00:54:58.247420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 00:54:58.247420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 00:54:58.247420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 2d2542cc-6d20-448f-a7ca-eed2dafeb349 could not be found. Aug 28 00:54:58.247420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:54:58.247420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:54:58.247420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:54:58.247420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:54:58.247420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:54:58.247420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:54:58.247420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 00:54:58.247420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 00:54:58.247420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 28 00:54:58.247420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 28 00:54:58.247420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 00:54:58.247420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:54:58.247420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Aug 28 00:54:58.247420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 00:54:58.251606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 2d2542cc-6d20-448f-a7ca-eed2dafeb349 could not be found. Aug 28 00:54:58.251606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:54:58.251606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:54:58.251606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:54:58.251606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:54:58.251606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:54:58.251606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:54:58.251606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 00:54:58.251606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 00:54:58.251606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 2d2542cc-6d20-448f-a7ca-eed2dafeb349 could not be found. Aug 28 00:54:58.251606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:54:58.251606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f72ffdf2-3d41-4eef-904d-f28eb93a41f0 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Returning 404 to user: Snapshot 2d2542cc-6d20-448f-a7ca-eed2dafeb349 could not be found. {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 00:54:58.251606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f72ffdf2-3d41-4eef-904d-f28eb93a41f0 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349" status: 404 len: 111 microversion: 2.1 time: 0.069319 Aug 28 00:54:58.254383 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 17/32] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Aug 28 00:54:58 2022] GET /compute/v2.1/os-snapshots/2d2542cc-6d20-448f-a7ca-eed2dafeb349 => generated 111 bytes in 79 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 00:54:58.278735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-495898b3-a443-4a80-b21a-9f491e6a2b28 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e700284c0>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:58.660285 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-495898b3-a443-4a80-b21a-9f491e6a2b28 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/5b7d8f79-89de-4cfd-9d3b-7630b7661554" status: 202 len: 0 microversion: 2.1 time: 0.398444 Aug 28 00:54:58.660862 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 18/33] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Aug 28 00:54:58 2022] DELETE /compute/v2.1/os-volumes/5b7d8f79-89de-4cfd-9d3b-7630b7661554 => generated 0 bytes in 406 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 00:54:58.671908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-70c197c2-4065-4068-909f-4d9008848e5a tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e6ec5e160>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:58.884351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-70c197c2-4065-4068-909f-4d9008848e5a tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/5b7d8f79-89de-4cfd-9d3b-7630b7661554" status: 200 len: 345 microversion: 2.1 time: 0.218973 Aug 28 00:54:58.885702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1ea703a1-07f5-431d-b834-b7e63bb2bed1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:54:58.893062 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 19/34] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Aug 28 00:54:58 2022] GET /compute/v2.1/os-volumes/5b7d8f79-89de-4cfd-9d3b-7630b7661554 => generated 345 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:54:58.901095 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-94b792ae-60ec-42b7-9e29-a729c33bb977 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:58.904493 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-94b792ae-60ec-42b7-9e29-a729c33bb977 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:54:58.918665 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94b792ae-60ec-42b7-9e29-a729c33bb977 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:58.920205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94b792ae-60ec-42b7-9e29-a729c33bb977 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:58.923899 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1ea703a1-07f5-431d-b834-b7e63bb2bed1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1274 microversion: 2.1 time: 0.906789 Aug 28 00:54:58.924221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94b792ae-60ec-42b7-9e29-a729c33bb977 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:58.924790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 16/35] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:54:58 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1274 bytes in 908 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:54:58.934789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94b792ae-60ec-42b7-9e29-a729c33bb977 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:58.935223 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94b792ae-60ec-42b7-9e29-a729c33bb977 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:58.941044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94b792ae-60ec-42b7-9e29-a729c33bb977 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:59.095124 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-94b792ae-60ec-42b7-9e29-a729c33bb977 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:54:59.108794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-94b792ae-60ec-42b7-9e29-a729c33bb977 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1275 microversion: 2.1 time: 0.211532 Aug 28 00:54:59.111313 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 20/36] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:54:58 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1275 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:54:59.669650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-83be7a88-f257-4b6b-849c-448e26253208 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Action: 'create', calling method: .version_select at 0x7f0e6ec85af0>, body: {"keypair": {"name": "tempest-keypair-1316453355", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGS+zWgAQ86pvhW9lXLQ6AGrdXN/a2aOsl8Y6YUX5U6XTE1tWUTVR5KxqR39XxfwgqzLMuFQhN5tF0c9KD773GVl3KNZciZU2kSpPx3nFVzdjy/NXfWCJ/pdb1HmbdPwCQ=="}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:54:59.700400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-83be7a88-f257-4b6b-849c-448e26253208 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Getting quotas for project 19f069b9ad7a4a5984fa947ad755db0a. Resources: {'key_pairs'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 00:54:59.704739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-83be7a88-f257-4b6b-849c-448e26253208 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Getting quotas for user 4894ef8b630e4bc084799c26096f7c66 and project 19f069b9ad7a4a5984fa947ad755db0a. Resources: {'key_pairs'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 00:54:59.762101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-83be7a88-f257-4b6b-849c-448e26253208 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.208833 Aug 28 00:54:59.771208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 17/37] 10.222.0.42 () {62 vars in 1200 bytes} [Sun Aug 28 00:54:59 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 219 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Aug 28 00:54:59.899562 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1c46a300-2835-42c4-87c6-862809ae6ad9 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e700281f0>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:59.947529 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-1c46a300-2835-42c4-87c6-862809ae6ad9 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] HTTP exception thrown: Volume 5b7d8f79-89de-4cfd-9d3b-7630b7661554 could not be found. Aug 28 00:54:59.949829 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bf0da45d-92d1-4787-ae48-eb962b852e91 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:54:59.950471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bf0da45d-92d1-4787-ae48-eb962b852e91 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:54:59.951074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1c46a300-2835-42c4-87c6-862809ae6ad9 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Request method failure captured: Aug 28 00:54:59.951074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: GET /compute/v2.1/os-volumes/5b7d8f79-89de-4cfd-9d3b-7630b7661554 HTTP/1.1 Aug 28 00:54:59.951074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 00:54:59.951074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 00:54:59.951074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 00:54:59.951074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 00:54:59.951074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 00:54:59.951074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 00:54:59.951074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 00:54:59.951074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 00:54:59.951074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCrzPi1vubP_Bcx_QX_wsKOT42eEwNnnuXc4cC6ZypEV9locCcwosJTNEmFnMt-z9BKEtizzMGm4YsAwyQPhxqB7U0lUrJCGZqwgdEhWhIfTkkVLVLdffh1RigaCc2BpsqY1q6Gv3RYrg5hAPRe1mb1hWPaoP-r2RqX632pE7KcZVMHw Aug 28 00:54:59.951074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 00:54:59.951074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 00:54:59.951074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 00:54:59.951074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 00:54:59.951074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 00:54:59.951074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 00:54:59.951074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: c22d5ee0e87f478995d585f2b567542a Aug 28 00:54:59.951074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1155394448 Aug 28 00:54:59.951074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 00:54:59.951074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 00:54:59.952485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_c22d5ee0e87f478995d585f2b567542a", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/c22d5ee0e87f478995d585f2b567542a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/c22d5ee0e87f478995d585f2b567542a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/c22d5ee0e87f478995d585f2b567542a"}]}] Aug 28 00:54:59.952485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1155394448 Aug 28 00:54:59.952485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: c22d5ee0e87f478995d585f2b567542a Aug 28 00:54:59.952485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1155394448 Aug 28 00:54:59.952485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-VolumesSnapshotsTestJSON-1155394448-project Aug 28 00:54:59.952485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 00:54:59.952485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 00:54:59.952485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: ab1089ce15ea4dcb8052e0e647af4945 Aug 28 00:54:59.952485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1155394448-project Aug 28 00:54:59.952485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: Aug 28 00:54:59.952485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Volume 5b7d8f79-89de-4cfd-9d3b-7630b7661554 could not be found. Aug 28 00:54:59.952485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'id': '5b7d8f79-89de-4cfd-9d3b-7630b7661554'} Aug 28 00:54:59.952485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 00:54:59.953536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:54:59.953536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 00:54:59.953536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 00:54:59.953536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 28 00:54:59.953536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 28 00:54:59.953536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 28 00:54:59.953536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 28 00:54:59.953536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 28 00:54:59.953536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 28 00:54:59.953536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 28 00:54:59.953536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 28 00:54:59.953536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 00:54:59.953536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 00:54:59.953536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 00:54:59.953536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 00:54:59.953536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 5b7d8f79-89de-4cfd-9d3b-7630b7661554 could not be found. (HTTP 404) (Request-ID: req-fba02272-1d30-4924-84e3-f20d20cc9bd2) Aug 28 00:54:59.953536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:54:59.953536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:54:59.953536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:54:59.954785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:54:59.954785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 28 00:54:59.954785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 28 00:54:59.954785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 00:54:59.954785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 00:54:59.954785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 28 00:54:59.954785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 28 00:54:59.954785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 28 00:54:59.954785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 28 00:54:59.954785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 00:54:59.954785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 00:54:59.954785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 28 00:54:59.954785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 28 00:54:59.954785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 28 00:54:59.954785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 28 00:54:59.954785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 28 00:54:59.954785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 28 00:54:59.954785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 28 00:54:59.954785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 28 00:54:59.955926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 00:54:59.955926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 00:54:59.955926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 00:54:59.955926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 00:54:59.955926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 5b7d8f79-89de-4cfd-9d3b-7630b7661554 could not be found. Aug 28 00:54:59.955926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:54:59.955926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:54:59.955926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:54:59.955926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:54:59.955926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:54:59.955926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:54:59.955926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 00:54:59.955926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 00:54:59.955926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 28 00:54:59.955926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 28 00:54:59.955926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 00:54:59.955926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:54:59.955926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 28 00:54:59.955926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 00:54:59.957125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 5b7d8f79-89de-4cfd-9d3b-7630b7661554 could not be found. Aug 28 00:54:59.957125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:54:59.957125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:54:59.957125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:54:59.957125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:54:59.957125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:54:59.957125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:54:59.957125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 00:54:59.957125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 00:54:59.957125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 5b7d8f79-89de-4cfd-9d3b-7630b7661554 could not be found. Aug 28 00:54:59.957125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:54:59.958890 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1c46a300-2835-42c4-87c6-862809ae6ad9 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Returning 404 to user: Volume 5b7d8f79-89de-4cfd-9d3b-7630b7661554 could not be found. {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 00:54:59.960699 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1c46a300-2835-42c4-87c6-862809ae6ad9 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/5b7d8f79-89de-4cfd-9d3b-7630b7661554" status: 404 len: 109 microversion: 2.1 time: 0.065624 Aug 28 00:54:59.961816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 21/38] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Aug 28 00:54:59 2022] GET /compute/v2.1/os-volumes/5b7d8f79-89de-4cfd-9d3b-7630b7661554 => generated 109 bytes in 67 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 00:54:59.968081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf0da45d-92d1-4787-ae48-eb962b852e91 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:59.969069 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf0da45d-92d1-4787-ae48-eb962b852e91 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:59.969759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf0da45d-92d1-4787-ae48-eb962b852e91 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:59.971995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf0da45d-92d1-4787-ae48-eb962b852e91 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:54:59.972939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf0da45d-92d1-4787-ae48-eb962b852e91 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:54:59.973686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf0da45d-92d1-4787-ae48-eb962b852e91 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:54:59.982312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-71963323-3d36-4db4-865a-0fbbde0abd34 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e70028310>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:00.051662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-71963323-3d36-4db4-865a-0fbbde0abd34 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] HTTP exception thrown: Volume 5b7d8f79-89de-4cfd-9d3b-7630b7661554 could not be found. Aug 28 00:55:00.054121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-71963323-3d36-4db4-865a-0fbbde0abd34 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Request method failure captured: Aug 28 00:55:00.054121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: DELETE /compute/v2.1/os-volumes/5b7d8f79-89de-4cfd-9d3b-7630b7661554 HTTP/1.1 Aug 28 00:55:00.054121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 00:55:00.054121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 00:55:00.054121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 00:55:00.054121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 00:55:00.054121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 00:55:00.054121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 00:55:00.054121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 00:55:00.054121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 00:55:00.054121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCrzPi1vubP_Bcx_QX_wsKOT42eEwNnnuXc4cC6ZypEV9locCcwosJTNEmFnMt-z9BKEtizzMGm4YsAwyQPhxqB7U0lUrJCGZqwgdEhWhIfTkkVLVLdffh1RigaCc2BpsqY1q6Gv3RYrg5hAPRe1mb1hWPaoP-r2RqX632pE7KcZVMHw Aug 28 00:55:00.054121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 00:55:00.054121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 00:55:00.054121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 00:55:00.054121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 00:55:00.054121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 00:55:00.054121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 00:55:00.054121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: c22d5ee0e87f478995d585f2b567542a Aug 28 00:55:00.054121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1155394448 Aug 28 00:55:00.054121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 00:55:00.054121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 00:55:00.055592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_c22d5ee0e87f478995d585f2b567542a", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/c22d5ee0e87f478995d585f2b567542a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/c22d5ee0e87f478995d585f2b567542a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/c22d5ee0e87f478995d585f2b567542a"}]}] Aug 28 00:55:00.055592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1155394448 Aug 28 00:55:00.055592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: c22d5ee0e87f478995d585f2b567542a Aug 28 00:55:00.055592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1155394448 Aug 28 00:55:00.055592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-VolumesSnapshotsTestJSON-1155394448-project Aug 28 00:55:00.055592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 00:55:00.055592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 00:55:00.055592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: ab1089ce15ea4dcb8052e0e647af4945 Aug 28 00:55:00.055592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1155394448-project Aug 28 00:55:00.055592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: Aug 28 00:55:00.055592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Volume 5b7d8f79-89de-4cfd-9d3b-7630b7661554 could not be found. Aug 28 00:55:00.055592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'id': '5b7d8f79-89de-4cfd-9d3b-7630b7661554'} Aug 28 00:55:00.055592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 00:55:00.056374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:55:00.056374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 00:55:00.056374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 00:55:00.056374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 28 00:55:00.056374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 28 00:55:00.056374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 28 00:55:00.056374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 28 00:55:00.056374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 28 00:55:00.056374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 28 00:55:00.056374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 28 00:55:00.056374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 28 00:55:00.056374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 00:55:00.056374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 00:55:00.056374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 00:55:00.056374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 00:55:00.056374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 5b7d8f79-89de-4cfd-9d3b-7630b7661554 could not be found. (HTTP 404) (Request-ID: req-ed0c1f4b-09a5-40ee-916f-c07d2f7a9205) Aug 28 00:55:00.056374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:55:00.056374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:55:00.056374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:55:00.057732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:55:00.057732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Aug 28 00:55:00.057732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Aug 28 00:55:00.057732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 00:55:00.057732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 00:55:00.057732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 28 00:55:00.057732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 28 00:55:00.057732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 28 00:55:00.057732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 28 00:55:00.057732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 00:55:00.057732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 00:55:00.057732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 28 00:55:00.057732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 28 00:55:00.057732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 28 00:55:00.057732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 28 00:55:00.057732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 28 00:55:00.057732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 28 00:55:00.057732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 28 00:55:00.057732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 28 00:55:00.058985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 00:55:00.058985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 00:55:00.058985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 00:55:00.058985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 00:55:00.058985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 5b7d8f79-89de-4cfd-9d3b-7630b7661554 could not be found. Aug 28 00:55:00.058985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:55:00.058985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:55:00.058985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:55:00.058985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:55:00.058985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:55:00.058985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:55:00.058985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 00:55:00.058985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 00:55:00.058985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 28 00:55:00.058985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 28 00:55:00.058985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 00:55:00.058985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:55:00.058985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Aug 28 00:55:00.058985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 00:55:00.060180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 5b7d8f79-89de-4cfd-9d3b-7630b7661554 could not be found. Aug 28 00:55:00.060180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:55:00.060180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:55:00.060180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:55:00.060180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:55:00.060180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:55:00.060180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:55:00.060180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 00:55:00.060180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 00:55:00.060180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 5b7d8f79-89de-4cfd-9d3b-7630b7661554 could not be found. Aug 28 00:55:00.060180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:55:00.060180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-71963323-3d36-4db4-865a-0fbbde0abd34 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Returning 404 to user: Volume 5b7d8f79-89de-4cfd-9d3b-7630b7661554 could not be found. {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 00:55:00.060180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-71963323-3d36-4db4-865a-0fbbde0abd34 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/5b7d8f79-89de-4cfd-9d3b-7630b7661554" status: 404 len: 109 microversion: 2.1 time: 0.088300 Aug 28 00:55:00.060180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 22/39] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Aug 28 00:54:59 2022] DELETE /compute/v2.1/os-volumes/5b7d8f79-89de-4cfd-9d3b-7630b7661554 => generated 109 bytes in 90 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 28 00:55:00.065297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b82f13b9-3bc0-40b9-ab33-305a33e23277 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Calling method '.version_select at 0x7f0e6ec0f1f0>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:00.156690 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-b82f13b9-3bc0-40b9-ab33-305a33e23277 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] HTTP exception thrown: Volume 5b7d8f79-89de-4cfd-9d3b-7630b7661554 could not be found. Aug 28 00:55:00.159057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b82f13b9-3bc0-40b9-ab33-305a33e23277 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Request method failure captured: Aug 28 00:55:00.159057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: GET /compute/v2.1/os-volumes/5b7d8f79-89de-4cfd-9d3b-7630b7661554 HTTP/1.1 Aug 28 00:55:00.159057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 00:55:00.159057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 00:55:00.159057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 00:55:00.159057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 00:55:00.159057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 00:55:00.159057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 00:55:00.159057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 00:55:00.159057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 00:55:00.159057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCrzPi1vubP_Bcx_QX_wsKOT42eEwNnnuXc4cC6ZypEV9locCcwosJTNEmFnMt-z9BKEtizzMGm4YsAwyQPhxqB7U0lUrJCGZqwgdEhWhIfTkkVLVLdffh1RigaCc2BpsqY1q6Gv3RYrg5hAPRe1mb1hWPaoP-r2RqX632pE7KcZVMHw Aug 28 00:55:00.159057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 00:55:00.159057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 00:55:00.159057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 00:55:00.159057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 00:55:00.159057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 00:55:00.159057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 00:55:00.159057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: c22d5ee0e87f478995d585f2b567542a Aug 28 00:55:00.159057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1155394448 Aug 28 00:55:00.159057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 00:55:00.159057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 00:55:00.160619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_c22d5ee0e87f478995d585f2b567542a", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/c22d5ee0e87f478995d585f2b567542a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/c22d5ee0e87f478995d585f2b567542a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/c22d5ee0e87f478995d585f2b567542a"}]}] Aug 28 00:55:00.160619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1155394448 Aug 28 00:55:00.160619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: c22d5ee0e87f478995d585f2b567542a Aug 28 00:55:00.160619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1155394448 Aug 28 00:55:00.160619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-VolumesSnapshotsTestJSON-1155394448-project Aug 28 00:55:00.160619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 00:55:00.160619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 00:55:00.160619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: ab1089ce15ea4dcb8052e0e647af4945 Aug 28 00:55:00.160619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1155394448-project Aug 28 00:55:00.160619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: Aug 28 00:55:00.160619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Volume 5b7d8f79-89de-4cfd-9d3b-7630b7661554 could not be found. Aug 28 00:55:00.160619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'id': '5b7d8f79-89de-4cfd-9d3b-7630b7661554'} Aug 28 00:55:00.160619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 00:55:00.162391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:55:00.162391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 00:55:00.162391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 00:55:00.162391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 28 00:55:00.162391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 28 00:55:00.162391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 28 00:55:00.162391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 28 00:55:00.162391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 28 00:55:00.162391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 28 00:55:00.162391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 28 00:55:00.162391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 28 00:55:00.162391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 00:55:00.162391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 00:55:00.162391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 00:55:00.162391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 00:55:00.162391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 5b7d8f79-89de-4cfd-9d3b-7630b7661554 could not be found. (HTTP 404) (Request-ID: req-004a03d6-0956-4539-8603-296ca4bd6aa7) Aug 28 00:55:00.162391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:55:00.162391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:55:00.162391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:55:00.165397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:55:00.165397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 28 00:55:00.165397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 28 00:55:00.165397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 00:55:00.165397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 00:55:00.165397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 28 00:55:00.165397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 28 00:55:00.165397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 28 00:55:00.165397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 28 00:55:00.165397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 00:55:00.165397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 00:55:00.165397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 28 00:55:00.165397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 28 00:55:00.165397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 28 00:55:00.165397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 28 00:55:00.165397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 28 00:55:00.165397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 28 00:55:00.165397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 28 00:55:00.165397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 28 00:55:00.177899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 00:55:00.177899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 00:55:00.177899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 00:55:00.177899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 00:55:00.177899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 5b7d8f79-89de-4cfd-9d3b-7630b7661554 could not be found. Aug 28 00:55:00.177899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:55:00.177899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:55:00.177899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:55:00.177899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:55:00.177899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:55:00.177899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:55:00.177899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 00:55:00.177899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 00:55:00.177899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 28 00:55:00.177899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 28 00:55:00.177899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 00:55:00.177899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:55:00.177899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 28 00:55:00.177899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 00:55:00.182041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 5b7d8f79-89de-4cfd-9d3b-7630b7661554 could not be found. Aug 28 00:55:00.182041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:55:00.182041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:55:00.182041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:55:00.182041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:55:00.182041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:55:00.182041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:55:00.182041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 00:55:00.182041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 00:55:00.182041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 5b7d8f79-89de-4cfd-9d3b-7630b7661554 could not be found. Aug 28 00:55:00.182041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:55:00.182041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b82f13b9-3bc0-40b9-ab33-305a33e23277 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] Returning 404 to user: Volume 5b7d8f79-89de-4cfd-9d3b-7630b7661554 could not be found. {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 00:55:00.182041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b82f13b9-3bc0-40b9-ab33-305a33e23277 tempest-VolumesSnapshotsTestJSON-1155394448 tempest-VolumesSnapshotsTestJSON-1155394448-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/5b7d8f79-89de-4cfd-9d3b-7630b7661554" status: 404 len: 109 microversion: 2.1 time: 0.099596 Aug 28 00:55:00.182041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 23/40] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Aug 28 00:55:00 2022] GET /compute/v2.1/os-volumes/5b7d8f79-89de-4cfd-9d3b-7630b7661554 => generated 109 bytes in 101 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 00:55:00.185847 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1331cfaf-3994-421f-99da-fbb49b968c7a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:00.191897 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1331cfaf-3994-421f-99da-fbb49b968c7a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:00.231083 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1331cfaf-3994-421f-99da-fbb49b968c7a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:00.233039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1331cfaf-3994-421f-99da-fbb49b968c7a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:00.237463 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1331cfaf-3994-421f-99da-fbb49b968c7a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:00.238940 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1331cfaf-3994-421f-99da-fbb49b968c7a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:00.240191 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1331cfaf-3994-421f-99da-fbb49b968c7a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:00.242043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1331cfaf-3994-421f-99da-fbb49b968c7a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:00.532905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bf0da45d-92d1-4787-ae48-eb962b852e91 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:00.548110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bf0da45d-92d1-4787-ae48-eb962b852e91 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1280 microversion: 2.1 time: 0.600770 Aug 28 00:55:00.568193 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 18/41] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:54:59 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1280 bytes in 624 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:00.587334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b714f608-d804-42a3-9387-b9d53a4902ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1680969307", "imageRef": "2c2585b2-bde5-4652-aed7-9f53b1bd5d94", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "e3dfdb02-2ba5-4b22-9a96-a212c53e9d60"}], "security_groups": [{"name": "tempest-securitygroup--1970598943"}], "key_name": "tempest-keypair-1836615983", "user_data": "}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:55:00.733666 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1331cfaf-3994-421f-99da-fbb49b968c7a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:00.744187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1331cfaf-3994-421f-99da-fbb49b968c7a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1273 microversion: 2.1 time: 0.581213 Aug 28 00:55:00.750866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 24/42] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:00 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1273 bytes in 589 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:01.571648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-aaa3d01e-4407-4d3b-aec7-df19a25aa9a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:01.572352 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-aaa3d01e-4407-4d3b-aec7-df19a25aa9a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:01.583797 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=4 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:01.586095 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aaa3d01e-4407-4d3b-aec7-df19a25aa9a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:01.586753 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aaa3d01e-4407-4d3b-aec7-df19a25aa9a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:01.587193 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aaa3d01e-4407-4d3b-aec7-df19a25aa9a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:01.592133 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aaa3d01e-4407-4d3b-aec7-df19a25aa9a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:01.592908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aaa3d01e-4407-4d3b-aec7-df19a25aa9a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:01.593639 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aaa3d01e-4407-4d3b-aec7-df19a25aa9a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:01.747428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.network.neutron [None req-b714f608-d804-42a3-9387-b9d53a4902ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] validate_networks() for [('e3dfdb02-2ba5-4b22-9a96-a212c53e9d60', None, None, None, None, None)] {{(pid=120525) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 28 00:55:02.017855 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-aaa3d01e-4407-4d3b-aec7-df19a25aa9a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:02.034463 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-aaa3d01e-4407-4d3b-aec7-df19a25aa9a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1280 microversion: 2.1 time: 0.467062 Aug 28 00:55:02.035418 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 25/43] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:01 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1280 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:55:02.076400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cea131e6-9728-427f-a8ed-3ba86c587acf tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:02.076400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cea131e6-9728-427f-a8ed-3ba86c587acf tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:02.076400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cea131e6-9728-427f-a8ed-3ba86c587acf tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:02.076400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cea131e6-9728-427f-a8ed-3ba86c587acf tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:02.077533 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cea131e6-9728-427f-a8ed-3ba86c587acf tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:02.077533 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cea131e6-9728-427f-a8ed-3ba86c587acf tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:02.077533 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cea131e6-9728-427f-a8ed-3ba86c587acf tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:02.077533 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cea131e6-9728-427f-a8ed-3ba86c587acf tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:02.795272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-cea131e6-9728-427f-a8ed-3ba86c587acf tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:02.804677 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cea131e6-9728-427f-a8ed-3ba86c587acf tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1273 microversion: 2.1 time: 0.768081 Aug 28 00:55:02.805359 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 26/44] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:02 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1273 bytes in 769 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:03.156413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-eb66c07a-5a0d-4124-8380-112845be1bff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Action: 'create', calling method: .version_select at 0x7f0e6ecaa9d0>, body: {"keypair": {"name": "tempest-keypair-1424971404", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBL0T4GxBl8iKRXT7m5iOotAPHFBEC+fW1hV8DGO3DrXbxO4muc45AQkp1Fw0dIoHFVGaxIXT1nxdKxt8qguJOJmWzfBdGrE2M5SaDBPIOIAPjV565AqdRwYYRXhvzxMZ5A=="}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:55:03.219910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-eb66c07a-5a0d-4124-8380-112845be1bff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Getting quotas for project 2348c1ddbec1476bacb194214794015d. Resources: {'key_pairs'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 00:55:03.232123 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-eb66c07a-5a0d-4124-8380-112845be1bff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Getting quotas for user 3de5d68f67784e33bdcd19a9cdfed744 and project 2348c1ddbec1476bacb194214794015d. Resources: {'key_pairs'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 00:55:03.300643 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-eb66c07a-5a0d-4124-8380-112845be1bff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.494036 Aug 28 00:55:03.301356 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 27/45] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Aug 28 00:55:02 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 495 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:55:03.312527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-965cf0c1-2a01-4dd7-b370-0b32ff561b78 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:03.317404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-965cf0c1-2a01-4dd7-b370-0b32ff561b78 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:03.333106 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-965cf0c1-2a01-4dd7-b370-0b32ff561b78 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:03.333984 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-965cf0c1-2a01-4dd7-b370-0b32ff561b78 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:03.334682 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-965cf0c1-2a01-4dd7-b370-0b32ff561b78 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:03.337654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-965cf0c1-2a01-4dd7-b370-0b32ff561b78 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:03.338419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-965cf0c1-2a01-4dd7-b370-0b32ff561b78 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:03.340542 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-965cf0c1-2a01-4dd7-b370-0b32ff561b78 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:04.149314 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-965cf0c1-2a01-4dd7-b370-0b32ff561b78 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:04.158735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-965cf0c1-2a01-4dd7-b370-0b32ff561b78 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1280 microversion: 2.1 time: 0.855814 Aug 28 00:55:04.175329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 28/46] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:03 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1280 bytes in 872 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:04.182842 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b25e5c77-2b95-42cf-a821-d45f1b826628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:04.183465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b25e5c77-2b95-42cf-a821-d45f1b826628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:04.232213 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b25e5c77-2b95-42cf-a821-d45f1b826628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:04.232676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b25e5c77-2b95-42cf-a821-d45f1b826628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:04.233160 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b25e5c77-2b95-42cf-a821-d45f1b826628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:04.233887 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b25e5c77-2b95-42cf-a821-d45f1b826628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:04.245403 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b25e5c77-2b95-42cf-a821-d45f1b826628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:04.245961 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b25e5c77-2b95-42cf-a821-d45f1b826628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:05.701873 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b25e5c77-2b95-42cf-a821-d45f1b826628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:05.719944 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b25e5c77-2b95-42cf-a821-d45f1b826628 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1273 microversion: 2.1 time: 1.537672 Aug 28 00:55:05.720643 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 29/47] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:04 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1273 bytes in 1545 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:05.721422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=2 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:05.740911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-eb32c19f-2c20-485d-a1ef-7fbad5b2b9f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:05.742298 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-eb32c19f-2c20-485d-a1ef-7fbad5b2b9f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:05.768279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb32c19f-2c20-485d-a1ef-7fbad5b2b9f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:05.769482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb32c19f-2c20-485d-a1ef-7fbad5b2b9f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:05.780338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb32c19f-2c20-485d-a1ef-7fbad5b2b9f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:05.785676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb32c19f-2c20-485d-a1ef-7fbad5b2b9f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:05.790590 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb32c19f-2c20-485d-a1ef-7fbad5b2b9f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:05.791405 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb32c19f-2c20-485d-a1ef-7fbad5b2b9f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:06.348610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-eb32c19f-2c20-485d-a1ef-7fbad5b2b9f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:06.360869 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-eb32c19f-2c20-485d-a1ef-7fbad5b2b9f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1280 microversion: 2.1 time: 0.637582 Aug 28 00:55:06.361498 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 30/48] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:05 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1280 bytes in 639 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:06.395232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-77f2fd93-13fe-4822-8a89-ecf625934330 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-671807053", "imageRef": "2c2585b2-bde5-4652-aed7-9f53b1bd5d94", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "4c439816-86e3-4522-87f6-42fdb101eeb5"}], "security_groups": [{"name": "tempest-securitygroup--38818691"}], "key_name": "tempest-keypair-1316453355", "user_data": "}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:55:06.762502 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-b714f608-d804-42a3-9387-b9d53a4902ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Flavor limits 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 28 00:55:06.763100 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-b714f608-d804-42a3-9387-b9d53a4902ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Image limits 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 28 00:55:06.763724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-b714f608-d804-42a3-9387-b9d53a4902ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Flavor pref 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 28 00:55:06.763827 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-b714f608-d804-42a3-9387-b9d53a4902ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Image pref 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 28 00:55:06.764171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-b714f608-d804-42a3-9387-b9d53a4902ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 28 00:55:06.784515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b714f608-d804-42a3-9387-b9d53a4902ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:06.785040 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b714f608-d804-42a3-9387-b9d53a4902ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:06.796160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b714f608-d804-42a3-9387-b9d53a4902ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:06.798125 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b714f608-d804-42a3-9387-b9d53a4902ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:06.798616 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b714f608-d804-42a3-9387-b9d53a4902ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:06.799037 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b714f608-d804-42a3-9387-b9d53a4902ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:06.902004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-b714f608-d804-42a3-9387-b9d53a4902ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Getting quotas for project 95df81e14a8549fca0851234f4a00b89. Resources: {'ram', 'instances', 'cores'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 00:55:06.923119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-b714f608-d804-42a3-9387-b9d53a4902ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Getting quotas for user d209174d0db04b3584556824c6d0a3d4 and project 95df81e14a8549fca0851234f4a00b89. Resources: {'ram', 'instances', 'cores'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 00:55:06.974934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b714f608-d804-42a3-9387-b9d53a4902ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Going to run 1 instances... {{(pid=120525) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 28 00:55:06.983255 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b714f608-d804-42a3-9387-b9d53a4902ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2c2585b2-bde5-4652-aed7-9f53b1bd5d94',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120525) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 28 00:55:07.148476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b714f608-d804-42a3-9387-b9d53a4902ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 6.565865 Aug 28 00:55:07.149767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 19/49] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Aug 28 00:55:00 2022] POST /compute/v2.1/servers => generated 402 bytes in 6581 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 28 00:55:07.164849 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-152547d9-2f24-460a-a3da-26774cb7fef6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:07.174180 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-152547d9-2f24-460a-a3da-26774cb7fef6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:07.184725 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-152547d9-2f24-460a-a3da-26774cb7fef6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:07.193759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-152547d9-2f24-460a-a3da-26774cb7fef6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:07.234577 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-152547d9-2f24-460a-a3da-26774cb7fef6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:07.235822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-152547d9-2f24-460a-a3da-26774cb7fef6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:07.236804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-152547d9-2f24-460a-a3da-26774cb7fef6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:07.259205 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-152547d9-2f24-460a-a3da-26774cb7fef6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:07.936780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-152547d9-2f24-460a-a3da-26774cb7fef6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:07.985909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-152547d9-2f24-460a-a3da-26774cb7fef6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1273 microversion: 2.1 time: 0.834430 Aug 28 00:55:07.990642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 20/50] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:07 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1273 bytes in 840 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:08.039434 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ab0d7765-09ff-4c3e-ab71-399037220287 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:08.040523 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ab0d7765-09ff-4c3e-ab71-399037220287 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:08.271866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.network.neutron [None req-77f2fd93-13fe-4822-8a89-ecf625934330 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] validate_networks() for [('4c439816-86e3-4522-87f6-42fdb101eeb5', None, None, None, None, None)] {{(pid=120524) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 28 00:55:08.558923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ab0d7765-09ff-4c3e-ab71-399037220287 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:08.585773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ab0d7765-09ff-4c3e-ab71-399037220287 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1242 microversion: 2.1 time: 0.592457 Aug 28 00:55:08.589584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 21/51] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:07 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1242 bytes in 598 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:08.597672 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-93e4d6c3-c8cc-4963-9c6d-3094524e302c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:08.598730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-93e4d6c3-c8cc-4963-9c6d-3094524e302c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:08.612678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93e4d6c3-c8cc-4963-9c6d-3094524e302c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:08.616011 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93e4d6c3-c8cc-4963-9c6d-3094524e302c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:08.616011 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93e4d6c3-c8cc-4963-9c6d-3094524e302c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:08.617487 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93e4d6c3-c8cc-4963-9c6d-3094524e302c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:08.618210 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93e4d6c3-c8cc-4963-9c6d-3094524e302c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:08.619000 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93e4d6c3-c8cc-4963-9c6d-3094524e302c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:10.298101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-93e4d6c3-c8cc-4963-9c6d-3094524e302c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:10.328382 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-93e4d6c3-c8cc-4963-9c6d-3094524e302c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 1.731931 Aug 28 00:55:10.335200 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 22/52] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:08 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 1745 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:10.394112 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-691288b2-1a05-474e-b5ea-b74fc64711d6 tempest-VolumesTestJSON-1713501982 tempest-VolumesTestJSON-1713501982-project] Calling method '.version_select at 0x7f0e6ec610d0>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:10.557863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-691288b2-1a05-474e-b5ea-b74fc64711d6 tempest-VolumesTestJSON-1713501982 tempest-VolumesTestJSON-1713501982-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/f9716280-d6f4-4dfe-b1ea-ed4ff1e6da91" status: 200 len: 351 microversion: 2.1 time: 0.205918 Aug 28 00:55:10.559481 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 23/53] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Aug 28 00:55:10 2022] GET /compute/v2.1/os-volumes/f9716280-d6f4-4dfe-b1ea-ed4ff1e6da91 => generated 351 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:55:10.597878 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7eafd1b1-a9aa-4669-9e26-ceb910527115 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:10.608068 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7eafd1b1-a9aa-4669-9e26-ceb910527115 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:10.627348 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=64,nova_api:INSERT=7 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:10.705068 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7eafd1b1-a9aa-4669-9e26-ceb910527115 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:10.710477 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7eafd1b1-a9aa-4669-9e26-ceb910527115 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:10.719471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7eafd1b1-a9aa-4669-9e26-ceb910527115 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:10.720794 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7eafd1b1-a9aa-4669-9e26-ceb910527115 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:10.738977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7eafd1b1-a9aa-4669-9e26-ceb910527115 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:10.744922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7eafd1b1-a9aa-4669-9e26-ceb910527115 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.021s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:12.941686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7eafd1b1-a9aa-4669-9e26-ceb910527115 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:12.952416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7eafd1b1-a9aa-4669-9e26-ceb910527115 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 2.391139 Aug 28 00:55:12.964234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 24/54] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:10 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 2404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:12.983224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4961b899-4139-4ca9-98ec-88741b61055b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:12.986590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4961b899-4139-4ca9-98ec-88741b61055b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:12.996777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4961b899-4139-4ca9-98ec-88741b61055b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:12.998042 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4961b899-4139-4ca9-98ec-88741b61055b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:12.998776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4961b899-4139-4ca9-98ec-88741b61055b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:13.000859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4961b899-4139-4ca9-98ec-88741b61055b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:13.002845 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4961b899-4139-4ca9-98ec-88741b61055b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:13.004917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4961b899-4139-4ca9-98ec-88741b61055b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:13.141439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-77f2fd93-13fe-4822-8a89-ecf625934330 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Flavor limits 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 28 00:55:13.141987 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-77f2fd93-13fe-4822-8a89-ecf625934330 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Image limits 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 28 00:55:13.142467 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-77f2fd93-13fe-4822-8a89-ecf625934330 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Flavor pref 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 28 00:55:13.142941 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-77f2fd93-13fe-4822-8a89-ecf625934330 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Image pref 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 28 00:55:13.143429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-77f2fd93-13fe-4822-8a89-ecf625934330 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 28 00:55:13.164732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-77f2fd93-13fe-4822-8a89-ecf625934330 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:13.165559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-77f2fd93-13fe-4822-8a89-ecf625934330 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:13.166195 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-77f2fd93-13fe-4822-8a89-ecf625934330 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:13.167059 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-77f2fd93-13fe-4822-8a89-ecf625934330 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:13.167578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-77f2fd93-13fe-4822-8a89-ecf625934330 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:13.171965 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-77f2fd93-13fe-4822-8a89-ecf625934330 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:13.209307 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-77f2fd93-13fe-4822-8a89-ecf625934330 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Getting quotas for project 19f069b9ad7a4a5984fa947ad755db0a. Resources: {'ram', 'instances', 'cores'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 00:55:13.221560 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-77f2fd93-13fe-4822-8a89-ecf625934330 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Getting quotas for user 4894ef8b630e4bc084799c26096f7c66 and project 19f069b9ad7a4a5984fa947ad755db0a. Resources: {'ram', 'instances', 'cores'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 00:55:13.251379 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-77f2fd93-13fe-4822-8a89-ecf625934330 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Going to run 1 instances... {{(pid=120524) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 28 00:55:13.253917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-77f2fd93-13fe-4822-8a89-ecf625934330 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2c2585b2-bde5-4652-aed7-9f53b1bd5d94',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120524) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 28 00:55:13.319351 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4961b899-4139-4ca9-98ec-88741b61055b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:13.331290 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4961b899-4139-4ca9-98ec-88741b61055b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1300 microversion: 2.1 time: 0.364335 Aug 28 00:55:13.350756 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=89,nova_api:INSERT=11 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:13.359864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 25/55] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:12 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1300 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:13.371761 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8f31c55c-4b24-4ba1-836a-c621e8f0f6f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1923584688", "imageRef": "2c2585b2-bde5-4652-aed7-9f53b1bd5d94", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "9173759e-b079-4264-9516-c7ce5b7af7ce"}], "security_groups": [{"name": "tempest-securitygroup--2023293624"}], "key_name": "tempest-keypair-1424971404", "user_data": "}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:55:13.428494 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-77f2fd93-13fe-4822-8a89-ecf625934330 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.20 time: 7.061581 Aug 28 00:55:13.429246 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 31/56] 10.222.0.42 () {62 vars in 1188 bytes} [Sun Aug 28 00:55:06 2022] POST /compute/v2.1/servers => generated 400 bytes in 7067 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Aug 28 00:55:13.446896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-66b7389c-c8dd-4725-bcf3-0a173fd25dcf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:13.447498 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-66b7389c-c8dd-4725-bcf3-0a173fd25dcf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:13.474182 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-66b7389c-c8dd-4725-bcf3-0a173fd25dcf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:13.480996 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-66b7389c-c8dd-4725-bcf3-0a173fd25dcf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:13.481611 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-66b7389c-c8dd-4725-bcf3-0a173fd25dcf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:13.482514 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-66b7389c-c8dd-4725-bcf3-0a173fd25dcf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:13.486254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-66b7389c-c8dd-4725-bcf3-0a173fd25dcf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:13.486702 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-66b7389c-c8dd-4725-bcf3-0a173fd25dcf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:14.661505 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.network.neutron [None req-8f31c55c-4b24-4ba1-836a-c621e8f0f6f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] validate_networks() for [('9173759e-b079-4264-9516-c7ce5b7af7ce', None, None, None, None, None)] {{(pid=120525) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 28 00:55:14.906782 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-66b7389c-c8dd-4725-bcf3-0a173fd25dcf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:14.924803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=77 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:14.932786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-66b7389c-c8dd-4725-bcf3-0a173fd25dcf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 1.496612 Aug 28 00:55:14.950738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 32/57] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:13 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 1505 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:14.973967 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b1c62838-4561-43ea-ba5e-bad1586b2c6f tempest-VolumesTestJSON-1713501982 tempest-VolumesTestJSON-1713501982-project] Calling method '.version_select at 0x7f0e70028550>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:15.150255 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b1c62838-4561-43ea-ba5e-bad1586b2c6f tempest-VolumesTestJSON-1713501982 tempest-VolumesTestJSON-1713501982-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/f3d5dbe9-9c57-47d1-96c5-4f4b28e92de5" status: 200 len: 351 microversion: 2.1 time: 0.191234 Aug 28 00:55:15.161601 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 33/58] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Aug 28 00:55:14 2022] GET /compute/v2.1/os-volumes/f3d5dbe9-9c57-47d1-96c5-4f4b28e92de5 => generated 351 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:55:15.167270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-43ce270a-01b0-437e-a424-ee69c4cc53b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:15.167898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-43ce270a-01b0-437e-a424-ee69c4cc53b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:15.182559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43ce270a-01b0-437e-a424-ee69c4cc53b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:15.183062 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43ce270a-01b0-437e-a424-ee69c4cc53b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:15.183499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43ce270a-01b0-437e-a424-ee69c4cc53b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:15.184211 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43ce270a-01b0-437e-a424-ee69c4cc53b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:15.184666 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43ce270a-01b0-437e-a424-ee69c4cc53b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:15.185116 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43ce270a-01b0-437e-a424-ee69c4cc53b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:15.584259 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-43ce270a-01b0-437e-a424-ee69c4cc53b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:15.596847 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-43ce270a-01b0-437e-a424-ee69c4cc53b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:15.599665 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-43ce270a-01b0-437e-a424-ee69c4cc53b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:15.618486 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-43ce270a-01b0-437e-a424-ee69c4cc53b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1277 microversion: 2.20 time: 0.455603 Aug 28 00:55:15.630746 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 34/59] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:15 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1277 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:55:15.641421 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d8309e41-9fe4-43ef-b038-f42cb7c2e94f tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:15.646405 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d8309e41-9fe4-43ef-b038-f42cb7c2e94f tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:15.678469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d8309e41-9fe4-43ef-b038-f42cb7c2e94f tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:15.683216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d8309e41-9fe4-43ef-b038-f42cb7c2e94f tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:15.685362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d8309e41-9fe4-43ef-b038-f42cb7c2e94f tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:15.693789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d8309e41-9fe4-43ef-b038-f42cb7c2e94f tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:15.695214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d8309e41-9fe4-43ef-b038-f42cb7c2e94f tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:15.696847 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d8309e41-9fe4-43ef-b038-f42cb7c2e94f tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:16.704518 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d8309e41-9fe4-43ef-b038-f42cb7c2e94f tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:16.723492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d8309e41-9fe4-43ef-b038-f42cb7c2e94f tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 1.091354 Aug 28 00:55:16.738637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 35/60] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:15 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 1107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:16.743753 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-08684a65-4924-4f7b-a5b7-5289d7b11f88 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:16.759168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-08684a65-4924-4f7b-a5b7-5289d7b11f88 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:16.775579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08684a65-4924-4f7b-a5b7-5289d7b11f88 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:16.776347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08684a65-4924-4f7b-a5b7-5289d7b11f88 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:16.777630 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08684a65-4924-4f7b-a5b7-5289d7b11f88 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:16.780759 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08684a65-4924-4f7b-a5b7-5289d7b11f88 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:16.784007 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08684a65-4924-4f7b-a5b7-5289d7b11f88 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:16.784626 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08684a65-4924-4f7b-a5b7-5289d7b11f88 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:16.856252 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=2 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:17.145308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-08684a65-4924-4f7b-a5b7-5289d7b11f88 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:17.155505 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-08684a65-4924-4f7b-a5b7-5289d7b11f88 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.415377 Aug 28 00:55:17.173292 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 36/61] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:16 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:17.187719 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e0027caa-ada1-4968-b822-a7ede5a8e342 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:17.193687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e0027caa-ada1-4968-b822-a7ede5a8e342 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:17.238010 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e0027caa-ada1-4968-b822-a7ede5a8e342 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:17.245461 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e0027caa-ada1-4968-b822-a7ede5a8e342 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:17.246857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e0027caa-ada1-4968-b822-a7ede5a8e342 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:17.251772 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e0027caa-ada1-4968-b822-a7ede5a8e342 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:17.254268 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e0027caa-ada1-4968-b822-a7ede5a8e342 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:17.254985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e0027caa-ada1-4968-b822-a7ede5a8e342 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:17.901780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-8f31c55c-4b24-4ba1-836a-c621e8f0f6f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Flavor limits 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 28 00:55:17.904525 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-8f31c55c-4b24-4ba1-836a-c621e8f0f6f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Image limits 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 28 00:55:17.909182 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-8f31c55c-4b24-4ba1-836a-c621e8f0f6f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Flavor pref 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 28 00:55:17.909619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-8f31c55c-4b24-4ba1-836a-c621e8f0f6f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Image pref 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 28 00:55:17.913239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-8f31c55c-4b24-4ba1-836a-c621e8f0f6f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 28 00:55:17.922790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=25 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:17.928269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f31c55c-4b24-4ba1-836a-c621e8f0f6f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:17.928773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f31c55c-4b24-4ba1-836a-c621e8f0f6f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:17.929257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f31c55c-4b24-4ba1-836a-c621e8f0f6f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:17.934367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f31c55c-4b24-4ba1-836a-c621e8f0f6f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:17.934901 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f31c55c-4b24-4ba1-836a-c621e8f0f6f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:17.935353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f31c55c-4b24-4ba1-836a-c621e8f0f6f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:17.959220 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-8f31c55c-4b24-4ba1-836a-c621e8f0f6f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Getting quotas for project 2348c1ddbec1476bacb194214794015d. Resources: {'ram', 'instances', 'cores'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 00:55:17.964200 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-8f31c55c-4b24-4ba1-836a-c621e8f0f6f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Getting quotas for user 3de5d68f67784e33bdcd19a9cdfed744 and project 2348c1ddbec1476bacb194214794015d. Resources: {'ram', 'instances', 'cores'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 00:55:17.980701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8f31c55c-4b24-4ba1-836a-c621e8f0f6f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Going to run 1 instances... {{(pid=120525) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 28 00:55:17.985270 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8f31c55c-4b24-4ba1-836a-c621e8f0f6f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2c2585b2-bde5-4652-aed7-9f53b1bd5d94',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120525) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 28 00:55:18.045533 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e0027caa-ada1-4968-b822-a7ede5a8e342 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:18.056571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e0027caa-ada1-4968-b822-a7ede5a8e342 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.880735 Aug 28 00:55:18.066847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8f31c55c-4b24-4ba1-836a-c621e8f0f6f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 4.705726 Aug 28 00:55:18.070527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 26/62] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Aug 28 00:55:13 2022] POST /compute/v2.1/servers => generated 402 bytes in 4710 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 28 00:55:18.071948 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 37/63] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:17 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 898 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:18.083474 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-41029c42-ebb0-4e8a-b995-7fc8dd5cd1b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:18.085320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-41029c42-ebb0-4e8a-b995-7fc8dd5cd1b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:18.103200 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41029c42-ebb0-4e8a-b995-7fc8dd5cd1b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:18.103697 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41029c42-ebb0-4e8a-b995-7fc8dd5cd1b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:18.104123 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41029c42-ebb0-4e8a-b995-7fc8dd5cd1b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:18.104823 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41029c42-ebb0-4e8a-b995-7fc8dd5cd1b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:18.105291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41029c42-ebb0-4e8a-b995-7fc8dd5cd1b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:18.105701 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41029c42-ebb0-4e8a-b995-7fc8dd5cd1b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:18.405337 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-41029c42-ebb0-4e8a-b995-7fc8dd5cd1b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:18.422225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-401ce4fe-3df8-4b8d-8811-82701d924efa req-7e3cfc95-f3c3-43a1-a907-6a698be54296 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dff36aeb-f0c7-41f0-8ecc-9924adf8289c", "tag": "072b8279-6084-43c1-8f0e-c1a705ec9b99"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:55:18.430718 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-41029c42-ebb0-4e8a-b995-7fc8dd5cd1b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:18.432810 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-41029c42-ebb0-4e8a-b995-7fc8dd5cd1b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:18.441921 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-41029c42-ebb0-4e8a-b995-7fc8dd5cd1b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1339 microversion: 2.20 time: 0.368250 Aug 28 00:55:18.443934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 38/64] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:18 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1339 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:55:18.460897 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-401ce4fe-3df8-4b8d-8811-82701d924efa req-7e3cfc95-f3c3-43a1-a907-6a698be54296 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:18.461506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-401ce4fe-3df8-4b8d-8811-82701d924efa req-7e3cfc95-f3c3-43a1-a907-6a698be54296 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:18.462904 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-401ce4fe-3df8-4b8d-8811-82701d924efa req-7e3cfc95-f3c3-43a1-a907-6a698be54296 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:18.503970 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-401ce4fe-3df8-4b8d-8811-82701d924efa req-7e3cfc95-f3c3-43a1-a907-6a698be54296 service nova] Creating event network-changed:072b8279-6084-43c1-8f0e-c1a705ec9b99 for instance dff36aeb-f0c7-41f0-8ecc-9924adf8289c on hv-cinder-maste Aug 28 00:55:18.509559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-401ce4fe-3df8-4b8d-8811-82701d924efa req-7e3cfc95-f3c3-43a1-a907-6a698be54296 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120525) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:293}} Aug 28 00:55:18.510233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-401ce4fe-3df8-4b8d-8811-82701d924efa req-7e3cfc95-f3c3-43a1-a907-6a698be54296 service nova] Acquired lock "compute-rpcapi-router" {{(pid=120525) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:296}} Aug 28 00:55:18.511558 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-efaaed96-b5e1-42f5-8cf7-5cd2b8147d75 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:18.512236 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-efaaed96-b5e1-42f5-8cf7-5cd2b8147d75 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:18.513218 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-efaaed96-b5e1-42f5-8cf7-5cd2b8147d75 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:18.514312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-efaaed96-b5e1-42f5-8cf7-5cd2b8147d75 None None] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:18.514852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-efaaed96-b5e1-42f5-8cf7-5cd2b8147d75 None None] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:18.515353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-efaaed96-b5e1-42f5-8cf7-5cd2b8147d75 None None] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:18.542883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.compute.rpcapi [None req-efaaed96-b5e1-42f5-8cf7-5cd2b8147d75 None None] Automatically selected compute RPC version 6.0 from minimum service version 63 Aug 28 00:55:18.553895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-efaaed96-b5e1-42f5-8cf7-5cd2b8147d75 None None] Releasing lock "compute-rpcapi-router" {{(pid=120525) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:314}} Aug 28 00:55:18.573341 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-efaaed96-b5e1-42f5-8cf7-5cd2b8147d75 None None] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.500779 Aug 28 00:55:18.575050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 27/65] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:55:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:55:18.581576 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e92a40e3-687a-49ad-955b-dd095ac35341 tempest-VolumesTestJSON-1713501982 tempest-VolumesTestJSON-1713501982-project] Calling method '.version_select at 0x7f0e6ec31670>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:18.672453 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e92a40e3-687a-49ad-955b-dd095ac35341 tempest-VolumesTestJSON-1713501982 tempest-VolumesTestJSON-1713501982-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/45ae8145-beb0-4034-867d-3754205ba464" status: 200 len: 350 microversion: 2.1 time: 0.095741 Aug 28 00:55:18.680725 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 28/66] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Aug 28 00:55:18 2022] GET /compute/v2.1/os-volumes/45ae8145-beb0-4034-867d-3754205ba464 => generated 350 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:55:18.686404 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-91a41992-8f25-4f84-b3ff-cee5f9a0abf4 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:18.687592 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-91a41992-8f25-4f84-b3ff-cee5f9a0abf4 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:18.702950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91a41992-8f25-4f84-b3ff-cee5f9a0abf4 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:18.703865 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91a41992-8f25-4f84-b3ff-cee5f9a0abf4 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:18.704501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91a41992-8f25-4f84-b3ff-cee5f9a0abf4 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:18.705947 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91a41992-8f25-4f84-b3ff-cee5f9a0abf4 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:18.707991 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91a41992-8f25-4f84-b3ff-cee5f9a0abf4 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:18.709618 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91a41992-8f25-4f84-b3ff-cee5f9a0abf4 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:18.736796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-957e27e9-8188-4817-9b30-372452d4e718 req-7f71cf3a-ad18-4877-ac27-bbc52c7bb390 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f5f7fe76-9095-4d8d-967f-be0009648de9", "tag": "ff741c2a-17ad-4c15-a6a0-687d9dfcad8b"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:55:18.764109 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-957e27e9-8188-4817-9b30-372452d4e718 req-7f71cf3a-ad18-4877-ac27-bbc52c7bb390 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:18.768290 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-957e27e9-8188-4817-9b30-372452d4e718 req-7f71cf3a-ad18-4877-ac27-bbc52c7bb390 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:18.769740 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-957e27e9-8188-4817-9b30-372452d4e718 req-7f71cf3a-ad18-4877-ac27-bbc52c7bb390 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:18.818268 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-957e27e9-8188-4817-9b30-372452d4e718 req-7f71cf3a-ad18-4877-ac27-bbc52c7bb390 service nova] Creating event network-changed:ff741c2a-17ad-4c15-a6a0-687d9dfcad8b for instance f5f7fe76-9095-4d8d-967f-be0009648de9 on hv-cinder-maste Aug 28 00:55:18.819039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-957e27e9-8188-4817-9b30-372452d4e718 req-7f71cf3a-ad18-4877-ac27-bbc52c7bb390 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120524) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:293}} Aug 28 00:55:18.819562 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-957e27e9-8188-4817-9b30-372452d4e718 req-7f71cf3a-ad18-4877-ac27-bbc52c7bb390 service nova] Acquired lock "compute-rpcapi-router" {{(pid=120524) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:296}} Aug 28 00:55:18.820768 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49a6e850-2155-4a03-9fa6-0ee7b6f7b1ad None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:18.821405 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49a6e850-2155-4a03-9fa6-0ee7b6f7b1ad None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:18.821979 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49a6e850-2155-4a03-9fa6-0ee7b6f7b1ad None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:18.833981 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49a6e850-2155-4a03-9fa6-0ee7b6f7b1ad None None] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:18.834711 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49a6e850-2155-4a03-9fa6-0ee7b6f7b1ad None None] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:18.835315 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49a6e850-2155-4a03-9fa6-0ee7b6f7b1ad None None] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:18.848006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.compute.rpcapi [None req-49a6e850-2155-4a03-9fa6-0ee7b6f7b1ad None None] Automatically selected compute RPC version 6.0 from minimum service version 63 Aug 28 00:55:18.850782 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49a6e850-2155-4a03-9fa6-0ee7b6f7b1ad None None] Releasing lock "compute-rpcapi-router" {{(pid=120524) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:314}} Aug 28 00:55:18.882610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-49a6e850-2155-4a03-9fa6-0ee7b6f7b1ad None None] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.435815 Aug 28 00:55:18.884266 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 39/67] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:55:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:55:18.898020 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6eb0dabe-6011-4ab4-a7ff-cc238cc4b644 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:18.898755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6eb0dabe-6011-4ab4-a7ff-cc238cc4b644 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:18.912704 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6eb0dabe-6011-4ab4-a7ff-cc238cc4b644 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:18.913223 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6eb0dabe-6011-4ab4-a7ff-cc238cc4b644 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:18.913652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6eb0dabe-6011-4ab4-a7ff-cc238cc4b644 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:18.917541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6eb0dabe-6011-4ab4-a7ff-cc238cc4b644 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:18.921689 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6eb0dabe-6011-4ab4-a7ff-cc238cc4b644 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:18.922410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6eb0dabe-6011-4ab4-a7ff-cc238cc4b644 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:19.039535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6eb0dabe-6011-4ab4-a7ff-cc238cc4b644 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:19.049437 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6eb0dabe-6011-4ab4-a7ff-cc238cc4b644 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1242 microversion: 2.1 time: 0.163100 Aug 28 00:55:19.051045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 40/68] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:18 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1242 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:19.056393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bacc4219-9ab2-4474-a1f4-3df08fd2ced9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:19.057388 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bacc4219-9ab2-4474-a1f4-3df08fd2ced9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:19.069128 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bacc4219-9ab2-4474-a1f4-3df08fd2ced9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:19.069621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bacc4219-9ab2-4474-a1f4-3df08fd2ced9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:19.070054 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bacc4219-9ab2-4474-a1f4-3df08fd2ced9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:19.070762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bacc4219-9ab2-4474-a1f4-3df08fd2ced9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:19.071241 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bacc4219-9ab2-4474-a1f4-3df08fd2ced9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:19.071856 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bacc4219-9ab2-4474-a1f4-3df08fd2ced9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:19.283630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-91a41992-8f25-4f84-b3ff-cee5f9a0abf4 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:19.295139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-91a41992-8f25-4f84-b3ff-cee5f9a0abf4 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.613008 Aug 28 00:55:19.308336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 29/69] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:18 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 627 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:19.313228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-81438baa-8188-4311-8fc3-f7e8731aa653 tempest-VolumesTestJSON-1713501982 tempest-VolumesTestJSON-1713501982-project] Calling method '.version_select at 0x7f0e6ec31430>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:19.411235 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-81438baa-8188-4311-8fc3-f7e8731aa653 tempest-VolumesTestJSON-1713501982 tempest-VolumesTestJSON-1713501982-project] 10.222.0.42 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.101332 Aug 28 00:55:19.412781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 30/70] 10.222.0.42 () {58 vars in 1134 bytes} [Sun Aug 28 00:55:19 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:19.420948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-48f68526-0c59-4142-bec0-02e0f912d2f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:19.422268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-48f68526-0c59-4142-bec0-02e0f912d2f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:19.429939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48f68526-0c59-4142-bec0-02e0f912d2f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:19.430547 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48f68526-0c59-4142-bec0-02e0f912d2f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:19.431030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48f68526-0c59-4142-bec0-02e0f912d2f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:19.432221 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48f68526-0c59-4142-bec0-02e0f912d2f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:19.432758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48f68526-0c59-4142-bec0-02e0f912d2f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:19.433400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48f68526-0c59-4142-bec0-02e0f912d2f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:19.450322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bacc4219-9ab2-4474-a1f4-3df08fd2ced9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:19.462571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bacc4219-9ab2-4474-a1f4-3df08fd2ced9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.409899 Aug 28 00:55:19.463739 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 41/71] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:19 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:19.470757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ac19f374-c507-434c-a650-08bf9a622ba4 tempest-VolumesTestJSON-1713501982 tempest-VolumesTestJSON-1713501982-project] Calling method '.version_select at 0x7f0e6ec0fca0>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:19.570395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ac19f374-c507-434c-a650-08bf9a622ba4 tempest-VolumesTestJSON-1713501982 tempest-VolumesTestJSON-1713501982-project] 10.222.0.42 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.104983 Aug 28 00:55:19.570969 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 42/72] 10.222.0.42 () {58 vars in 1149 bytes} [Sun Aug 28 00:55:19 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 694 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:55:19.576435 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-65d7b9b9-9ebc-413e-83ed-7e8bf3e74344 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:19.577261 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-65d7b9b9-9ebc-413e-83ed-7e8bf3e74344 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:19.587002 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-65d7b9b9-9ebc-413e-83ed-7e8bf3e74344 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:19.587799 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-65d7b9b9-9ebc-413e-83ed-7e8bf3e74344 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:19.588506 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-65d7b9b9-9ebc-413e-83ed-7e8bf3e74344 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:19.589692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-65d7b9b9-9ebc-413e-83ed-7e8bf3e74344 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:19.590193 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-65d7b9b9-9ebc-413e-83ed-7e8bf3e74344 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:19.590699 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-65d7b9b9-9ebc-413e-83ed-7e8bf3e74344 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:19.636418 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-48f68526-0c59-4142-bec0-02e0f912d2f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:19.645887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-48f68526-0c59-4142-bec0-02e0f912d2f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.229919 Aug 28 00:55:19.645887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 31/73] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:19 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:19.650301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5d89576d-ced4-49dd-ad07-16b6d9553be0 tempest-VolumesTestJSON-1713501982 tempest-VolumesTestJSON-1713501982-project] Calling method '.version_select at 0x7f0e6ec6de50>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:19.673503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-65d7b9b9-9ebc-413e-83ed-7e8bf3e74344 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:19.685479 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-65d7b9b9-9ebc-413e-83ed-7e8bf3e74344 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:19.688499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-65d7b9b9-9ebc-413e-83ed-7e8bf3e74344 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:19.690912 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-65d7b9b9-9ebc-413e-83ed-7e8bf3e74344 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1339 microversion: 2.20 time: 0.118392 Aug 28 00:55:19.692008 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 43/74] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:19 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1339 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:55:19.743336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5d89576d-ced4-49dd-ad07-16b6d9553be0 tempest-VolumesTestJSON-1713501982 tempest-VolumesTestJSON-1713501982-project] 10.222.0.42 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.096669 Aug 28 00:55:19.753187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 32/75] 10.222.0.42 () {58 vars in 1134 bytes} [Sun Aug 28 00:55:19 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:19.758946 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9e0fef94-ecdf-474b-99e0-c63b96be0383 tempest-VolumesTestJSON-1713501982 tempest-VolumesTestJSON-1713501982-project] Calling method '.version_select at 0x7f0e6ec83670>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:19.869333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9e0fef94-ecdf-474b-99e0-c63b96be0383 tempest-VolumesTestJSON-1713501982 tempest-VolumesTestJSON-1713501982-project] 10.222.0.42 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.114401 Aug 28 00:55:19.873076 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 44/76] 10.222.0.42 () {58 vars in 1167 bytes} [Sun Aug 28 00:55:19 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:55:19.883512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-67c1f581-365c-4b55-8cf0-4c16573e7de4 tempest-VolumesTestJSON-1713501982 tempest-VolumesTestJSON-1713501982-project] Calling method '.version_select at 0x7f0e6ec61310>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:19.974184 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-67c1f581-365c-4b55-8cf0-4c16573e7de4 tempest-VolumesTestJSON-1713501982 tempest-VolumesTestJSON-1713501982-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.094456 Aug 28 00:55:19.974766 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 33/77] 10.222.0.42 () {58 vars in 1170 bytes} [Sun Aug 28 00:55:19 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 694 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:55:19.988266 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-69762b5d-6a10-44d9-80b2-5ab7af957328 tempest-VolumesTestJSON-1713501982 tempest-VolumesTestJSON-1713501982-project] Calling method '.version_select at 0x7f0e6ebfeee0>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:20.075722 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1bb1f58e-4917-4104-8aa4-ed060f5250eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:20.076645 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1bb1f58e-4917-4104-8aa4-ed060f5250eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:20.084869 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1bb1f58e-4917-4104-8aa4-ed060f5250eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:20.085540 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1bb1f58e-4917-4104-8aa4-ed060f5250eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:20.086001 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1bb1f58e-4917-4104-8aa4-ed060f5250eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:20.087035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1bb1f58e-4917-4104-8aa4-ed060f5250eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:20.087583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1bb1f58e-4917-4104-8aa4-ed060f5250eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:20.088010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1bb1f58e-4917-4104-8aa4-ed060f5250eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:20.095321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-69762b5d-6a10-44d9-80b2-5ab7af957328 tempest-VolumesTestJSON-1713501982 tempest-VolumesTestJSON-1713501982-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.110852 Aug 28 00:55:20.096012 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 45/78] 10.222.0.42 () {58 vars in 1155 bytes} [Sun Aug 28 00:55:19 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:20.127948 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-32bd8e1a-dbb5-447c-965d-ed319ac23edf tempest-VolumesTestJSON-1713501982 tempest-VolumesTestJSON-1713501982-project] Calling method '.version_select at 0x7f0e6ebfd160>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:20.177968 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1bb1f58e-4917-4104-8aa4-ed060f5250eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:20.187707 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1bb1f58e-4917-4104-8aa4-ed060f5250eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1242 microversion: 2.1 time: 0.120295 Aug 28 00:55:20.197248 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 34/79] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:20 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1242 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:55:20.217602 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-32bd8e1a-dbb5-447c-965d-ed319ac23edf tempest-VolumesTestJSON-1713501982 tempest-VolumesTestJSON-1713501982-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.111197 Aug 28 00:55:20.218450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 46/80] 10.222.0.42 () {58 vars in 1188 bytes} [Sun Aug 28 00:55:20 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:55:20.232677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e0a3c63b-f6c4-4b5d-974c-68ece61f0a93 tempest-VolumesTestJSON-1713501982 tempest-VolumesTestJSON-1713501982-project] Calling method '.version_select at 0x7f0e6ec615e0>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:20.314390 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e0a3c63b-f6c4-4b5d-974c-68ece61f0a93 tempest-VolumesTestJSON-1713501982 tempest-VolumesTestJSON-1713501982-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.085954 Aug 28 00:55:20.315255 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 35/81] 10.222.0.42 () {58 vars in 1155 bytes} [Sun Aug 28 00:55:20 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:20.316464 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-55655dd7-7489-4492-8bae-aa4ced6d5a40 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:20.317144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-55655dd7-7489-4492-8bae-aa4ced6d5a40 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:20.323191 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-55655dd7-7489-4492-8bae-aa4ced6d5a40 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:20.323714 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-55655dd7-7489-4492-8bae-aa4ced6d5a40 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:20.324147 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-55655dd7-7489-4492-8bae-aa4ced6d5a40 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:20.325187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-55655dd7-7489-4492-8bae-aa4ced6d5a40 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:20.325636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-55655dd7-7489-4492-8bae-aa4ced6d5a40 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:20.326233 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-55655dd7-7489-4492-8bae-aa4ced6d5a40 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:20.485228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-718762e3-0245-45b2-9ad2-3548edc50ed7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:20.485918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-718762e3-0245-45b2-9ad2-3548edc50ed7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:20.487573 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-55655dd7-7489-4492-8bae-aa4ced6d5a40 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:20.495876 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-718762e3-0245-45b2-9ad2-3548edc50ed7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:20.496546 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-718762e3-0245-45b2-9ad2-3548edc50ed7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:20.496677 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-55655dd7-7489-4492-8bae-aa4ced6d5a40 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.184660 Aug 28 00:55:20.497174 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 47/82] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:20 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:55:20.497622 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-718762e3-0245-45b2-9ad2-3548edc50ed7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:20.498417 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-718762e3-0245-45b2-9ad2-3548edc50ed7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:20.498865 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-718762e3-0245-45b2-9ad2-3548edc50ed7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:20.499281 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-718762e3-0245-45b2-9ad2-3548edc50ed7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:20.504476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=65 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:20.675661 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e8570a90-eda4-48ca-a057-69459333ea07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:20.676729 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e8570a90-eda4-48ca-a057-69459333ea07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:20.680848 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-718762e3-0245-45b2-9ad2-3548edc50ed7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:20.686685 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8570a90-eda4-48ca-a057-69459333ea07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:20.687126 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8570a90-eda4-48ca-a057-69459333ea07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:20.687551 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8570a90-eda4-48ca-a057-69459333ea07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:20.688268 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8570a90-eda4-48ca-a057-69459333ea07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:20.688706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8570a90-eda4-48ca-a057-69459333ea07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:20.689157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8570a90-eda4-48ca-a057-69459333ea07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:20.694082 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-718762e3-0245-45b2-9ad2-3548edc50ed7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.212833 Aug 28 00:55:20.695082 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 36/83] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:20 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:20.718917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1a3f2c8b-2fc2-45e0-be68-17c987b9ad3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:20.719636 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1a3f2c8b-2fc2-45e0-be68-17c987b9ad3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:20.728016 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a3f2c8b-2fc2-45e0-be68-17c987b9ad3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:20.729066 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a3f2c8b-2fc2-45e0-be68-17c987b9ad3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:20.729963 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a3f2c8b-2fc2-45e0-be68-17c987b9ad3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:20.731035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a3f2c8b-2fc2-45e0-be68-17c987b9ad3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:20.732007 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a3f2c8b-2fc2-45e0-be68-17c987b9ad3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:20.732993 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a3f2c8b-2fc2-45e0-be68-17c987b9ad3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:20.891451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e8570a90-eda4-48ca-a057-69459333ea07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:20.899503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e8570a90-eda4-48ca-a057-69459333ea07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.228154 Aug 28 00:55:20.900086 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 48/84] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:20 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:55:20.931339 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-364b40a5-4f17-47e6-8b89-215800af2164 req-0698a828-f20b-48ce-9072-b95346ea075b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4d664b12-2edd-46e0-bcf3-4d94eb576bcd", "tag": "ffe6cab9-475d-45bb-9156-c9785ed386fd"}, {"name": "network-changed", "server_uuid": "460a7822-a544-4b66-9361-79ac7d118804", "tag": "b01cc365-e4c3-4f15-b660-e7eb82f6fcf4"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:55:20.934193 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1a3f2c8b-2fc2-45e0-be68-17c987b9ad3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:20.943234 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-364b40a5-4f17-47e6-8b89-215800af2164 req-0698a828-f20b-48ce-9072-b95346ea075b service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:20.943713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-364b40a5-4f17-47e6-8b89-215800af2164 req-0698a828-f20b-48ce-9072-b95346ea075b service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:20.944143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-364b40a5-4f17-47e6-8b89-215800af2164 req-0698a828-f20b-48ce-9072-b95346ea075b service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:20.946448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1a3f2c8b-2fc2-45e0-be68-17c987b9ad3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:20.948433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1a3f2c8b-2fc2-45e0-be68-17c987b9ad3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:20.949956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1a3f2c8b-2fc2-45e0-be68-17c987b9ad3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.234840 Aug 28 00:55:20.950504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 37/85] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:20 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:55:20.969609 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-364b40a5-4f17-47e6-8b89-215800af2164 req-0698a828-f20b-48ce-9072-b95346ea075b service nova] Creating event network-changed:ffe6cab9-475d-45bb-9156-c9785ed386fd for instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd on hv-cinder-maste Aug 28 00:55:20.970312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-364b40a5-4f17-47e6-8b89-215800af2164 req-0698a828-f20b-48ce-9072-b95346ea075b service nova] Creating event network-changed:b01cc365-e4c3-4f15-b660-e7eb82f6fcf4 for instance 460a7822-a544-4b66-9361-79ac7d118804 on hv-cinder-maste Aug 28 00:55:20.978352 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-364b40a5-4f17-47e6-8b89-215800af2164 req-0698a828-f20b-48ce-9072-b95346ea075b service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.051399 Aug 28 00:55:20.979207 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 49/86] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:55:20 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:55:21.210204 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-799e0777-eb63-4205-857a-c00185fe362d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:21.210807 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-799e0777-eb63-4205-857a-c00185fe362d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:21.216972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-799e0777-eb63-4205-857a-c00185fe362d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:21.217517 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-799e0777-eb63-4205-857a-c00185fe362d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:21.217981 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-799e0777-eb63-4205-857a-c00185fe362d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:21.218678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-799e0777-eb63-4205-857a-c00185fe362d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:21.219116 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-799e0777-eb63-4205-857a-c00185fe362d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:21.219605 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-799e0777-eb63-4205-857a-c00185fe362d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:21.281422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-799e0777-eb63-4205-857a-c00185fe362d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:21.289241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-799e0777-eb63-4205-857a-c00185fe362d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1242 microversion: 2.1 time: 0.082898 Aug 28 00:55:21.289814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 38/87] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:21 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1242 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:21.520853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-78fda981-1793-491a-a7f7-894fa0b81268 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:21.521709 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-78fda981-1793-491a-a7f7-894fa0b81268 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:21.528128 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78fda981-1793-491a-a7f7-894fa0b81268 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:21.528635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78fda981-1793-491a-a7f7-894fa0b81268 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:21.529209 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78fda981-1793-491a-a7f7-894fa0b81268 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:21.529890 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78fda981-1793-491a-a7f7-894fa0b81268 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:21.530340 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78fda981-1793-491a-a7f7-894fa0b81268 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:21.530752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78fda981-1793-491a-a7f7-894fa0b81268 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:21.676179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-78fda981-1793-491a-a7f7-894fa0b81268 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:21.685588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-78fda981-1793-491a-a7f7-894fa0b81268 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.168731 Aug 28 00:55:21.686255 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 50/88] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:21 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:21.727585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2bce5ed1-e41a-4c9e-b3ef-2a125cc80d8e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:21.727585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2bce5ed1-e41a-4c9e-b3ef-2a125cc80d8e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:21.727585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2bce5ed1-e41a-4c9e-b3ef-2a125cc80d8e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:21.727585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2bce5ed1-e41a-4c9e-b3ef-2a125cc80d8e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:21.728303 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2bce5ed1-e41a-4c9e-b3ef-2a125cc80d8e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:21.728303 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2bce5ed1-e41a-4c9e-b3ef-2a125cc80d8e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:21.728303 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2bce5ed1-e41a-4c9e-b3ef-2a125cc80d8e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:21.728303 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2bce5ed1-e41a-4c9e-b3ef-2a125cc80d8e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:21.866256 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2bce5ed1-e41a-4c9e-b3ef-2a125cc80d8e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:21.874376 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2bce5ed1-e41a-4c9e-b3ef-2a125cc80d8e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.162283 Aug 28 00:55:21.874971 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 39/89] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:21 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:21.919847 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5e56d498-6757-4594-b840-6bb27cecbe2f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:21.920380 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5e56d498-6757-4594-b840-6bb27cecbe2f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:21.926368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e56d498-6757-4594-b840-6bb27cecbe2f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:21.926872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e56d498-6757-4594-b840-6bb27cecbe2f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:21.927298 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e56d498-6757-4594-b840-6bb27cecbe2f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:21.927976 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e56d498-6757-4594-b840-6bb27cecbe2f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:21.928427 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e56d498-6757-4594-b840-6bb27cecbe2f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:21.928841 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e56d498-6757-4594-b840-6bb27cecbe2f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:21.971932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-007a5e16-7d0d-47f0-b972-2b7732e5ed12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:21.972514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-007a5e16-7d0d-47f0-b972-2b7732e5ed12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:21.978673 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-007a5e16-7d0d-47f0-b972-2b7732e5ed12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:21.979158 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-007a5e16-7d0d-47f0-b972-2b7732e5ed12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:21.979586 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-007a5e16-7d0d-47f0-b972-2b7732e5ed12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:21.980277 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-007a5e16-7d0d-47f0-b972-2b7732e5ed12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:21.980721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-007a5e16-7d0d-47f0-b972-2b7732e5ed12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:21.981188 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-007a5e16-7d0d-47f0-b972-2b7732e5ed12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:22.313948 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5e56d498-6757-4594-b840-6bb27cecbe2f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:22.322800 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5e56d498-6757-4594-b840-6bb27cecbe2f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.406842 Aug 28 00:55:22.323376 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 51/90] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:21 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:22.328268 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9e44801c-3f8e-4f81-a3a2-937e8297fd36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:22.329363 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9e44801c-3f8e-4f81-a3a2-937e8297fd36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:22.335974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e44801c-3f8e-4f81-a3a2-937e8297fd36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:22.336449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e44801c-3f8e-4f81-a3a2-937e8297fd36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:22.337116 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e44801c-3f8e-4f81-a3a2-937e8297fd36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:22.338124 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e44801c-3f8e-4f81-a3a2-937e8297fd36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:22.338938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e44801c-3f8e-4f81-a3a2-937e8297fd36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:22.339364 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e44801c-3f8e-4f81-a3a2-937e8297fd36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:22.361409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-007a5e16-7d0d-47f0-b972-2b7732e5ed12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:22.369699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-007a5e16-7d0d-47f0-b972-2b7732e5ed12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:22.371111 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-007a5e16-7d0d-47f0-b972-2b7732e5ed12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:22.372190 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-007a5e16-7d0d-47f0-b972-2b7732e5ed12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.404369 Aug 28 00:55:22.373399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 40/91] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:21 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:55:22.436643 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9e44801c-3f8e-4f81-a3a2-937e8297fd36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:22.445426 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9e44801c-3f8e-4f81-a3a2-937e8297fd36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1290 microversion: 2.1 time: 0.120872 Aug 28 00:55:22.445948 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 52/92] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:22 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1290 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:22.706934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5f21c777-79b2-4508-918c-bfb56e3ab92a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:22.707491 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5f21c777-79b2-4508-918c-bfb56e3ab92a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:22.713528 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f21c777-79b2-4508-918c-bfb56e3ab92a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:22.713967 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f21c777-79b2-4508-918c-bfb56e3ab92a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:22.714403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f21c777-79b2-4508-918c-bfb56e3ab92a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:22.715106 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f21c777-79b2-4508-918c-bfb56e3ab92a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:22.715553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f21c777-79b2-4508-918c-bfb56e3ab92a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:22.716020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f21c777-79b2-4508-918c-bfb56e3ab92a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:22.895241 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2df6c832-3898-43c9-bb5c-876ab928d6bc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:22.896386 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2df6c832-3898-43c9-bb5c-876ab928d6bc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:22.903915 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2df6c832-3898-43c9-bb5c-876ab928d6bc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:22.904397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2df6c832-3898-43c9-bb5c-876ab928d6bc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:22.905079 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2df6c832-3898-43c9-bb5c-876ab928d6bc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:22.905771 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2df6c832-3898-43c9-bb5c-876ab928d6bc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:22.906216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2df6c832-3898-43c9-bb5c-876ab928d6bc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:22.906658 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2df6c832-3898-43c9-bb5c-876ab928d6bc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:22.914209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5f21c777-79b2-4508-918c-bfb56e3ab92a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:22.923462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5f21c777-79b2-4508-918c-bfb56e3ab92a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.220354 Aug 28 00:55:22.924044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 41/93] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:22 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:23.008018 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-b11f4640-08cd-4917-ac3d-98ac0eee06de req-e19e1522-1957-4c67-9b31-3020bda9c4a3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dd61123b-7a96-4c9e-936c-5794a4fdd7df", "tag": "522d4a1b-32bc-4106-afb0-5995aecc53c0"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:55:23.026879 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-b11f4640-08cd-4917-ac3d-98ac0eee06de req-e19e1522-1957-4c67-9b31-3020bda9c4a3 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:23.027611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-b11f4640-08cd-4917-ac3d-98ac0eee06de req-e19e1522-1957-4c67-9b31-3020bda9c4a3 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:23.028493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-b11f4640-08cd-4917-ac3d-98ac0eee06de req-e19e1522-1957-4c67-9b31-3020bda9c4a3 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:23.041370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-b11f4640-08cd-4917-ac3d-98ac0eee06de req-e19e1522-1957-4c67-9b31-3020bda9c4a3 service nova] Creating event network-changed:522d4a1b-32bc-4106-afb0-5995aecc53c0 for instance dd61123b-7a96-4c9e-936c-5794a4fdd7df on hv-cinder-maste Aug 28 00:55:23.046519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-b11f4640-08cd-4917-ac3d-98ac0eee06de req-e19e1522-1957-4c67-9b31-3020bda9c4a3 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043837 Aug 28 00:55:23.047350 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 42/94] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:55:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:55:23.083256 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2df6c832-3898-43c9-bb5c-876ab928d6bc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:23.091551 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2df6c832-3898-43c9-bb5c-876ab928d6bc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.200698 Aug 28 00:55:23.092124 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 53/95] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:22 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:55:23.345422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-960e93df-8101-40ed-b7db-f242d1eb4b6a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:23.346127 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-960e93df-8101-40ed-b7db-f242d1eb4b6a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:23.352684 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-960e93df-8101-40ed-b7db-f242d1eb4b6a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:23.353198 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-960e93df-8101-40ed-b7db-f242d1eb4b6a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:23.353627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-960e93df-8101-40ed-b7db-f242d1eb4b6a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:23.354321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-960e93df-8101-40ed-b7db-f242d1eb4b6a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:23.354756 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-960e93df-8101-40ed-b7db-f242d1eb4b6a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:23.355171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-960e93df-8101-40ed-b7db-f242d1eb4b6a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:23.395289 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0f103086-c2dd-4c06-aeda-7ebf3abf38ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:23.395865 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0f103086-c2dd-4c06-aeda-7ebf3abf38ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:23.398010 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=28 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:23.403558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f103086-c2dd-4c06-aeda-7ebf3abf38ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:23.404044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f103086-c2dd-4c06-aeda-7ebf3abf38ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:23.404467 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f103086-c2dd-4c06-aeda-7ebf3abf38ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:23.405186 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f103086-c2dd-4c06-aeda-7ebf3abf38ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:23.405633 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f103086-c2dd-4c06-aeda-7ebf3abf38ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:23.406047 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f103086-c2dd-4c06-aeda-7ebf3abf38ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:23.538010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-960e93df-8101-40ed-b7db-f242d1eb4b6a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:23.545833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-960e93df-8101-40ed-b7db-f242d1eb4b6a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.204665 Aug 28 00:55:23.546415 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 43/96] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:23 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:23.559069 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0308289c-b483-4e28-81ec-7a667692f771 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:23.559069 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0308289c-b483-4e28-81ec-7a667692f771 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:23.559069 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0308289c-b483-4e28-81ec-7a667692f771 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:23.559069 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0308289c-b483-4e28-81ec-7a667692f771 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:23.559501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0308289c-b483-4e28-81ec-7a667692f771 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:23.559501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0308289c-b483-4e28-81ec-7a667692f771 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:23.559886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0308289c-b483-4e28-81ec-7a667692f771 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:23.560311 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0308289c-b483-4e28-81ec-7a667692f771 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:23.567136 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0f103086-c2dd-4c06-aeda-7ebf3abf38ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:23.576818 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0f103086-c2dd-4c06-aeda-7ebf3abf38ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:23.578329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0f103086-c2dd-4c06-aeda-7ebf3abf38ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:23.579564 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0f103086-c2dd-4c06-aeda-7ebf3abf38ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.188331 Aug 28 00:55:23.580643 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 54/97] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:23 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:55:23.630840 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0308289c-b483-4e28-81ec-7a667692f771 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:23.638480 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0308289c-b483-4e28-81ec-7a667692f771 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1290 microversion: 2.1 time: 0.091100 Aug 28 00:55:23.639049 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 44/98] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:23 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1290 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:23.945733 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-61353cf4-943f-4610-ac9b-572c5adee34c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:23.946327 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-61353cf4-943f-4610-ac9b-572c5adee34c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:23.952631 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-61353cf4-943f-4610-ac9b-572c5adee34c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:23.953137 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-61353cf4-943f-4610-ac9b-572c5adee34c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:23.953560 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-61353cf4-943f-4610-ac9b-572c5adee34c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:23.954238 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-61353cf4-943f-4610-ac9b-572c5adee34c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:23.954673 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-61353cf4-943f-4610-ac9b-572c5adee34c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:23.955067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-61353cf4-943f-4610-ac9b-572c5adee34c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:23.970226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:24.096826 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-61353cf4-943f-4610-ac9b-572c5adee34c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:24.105062 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-61353cf4-943f-4610-ac9b-572c5adee34c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.163082 Aug 28 00:55:24.105610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 55/99] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:23 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:24.112342 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9135a006-e245-4380-9a6e-0a2fec504244 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:24.113885 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9135a006-e245-4380-9a6e-0a2fec504244 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:24.121712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9135a006-e245-4380-9a6e-0a2fec504244 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:24.122491 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9135a006-e245-4380-9a6e-0a2fec504244 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:24.122596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9135a006-e245-4380-9a6e-0a2fec504244 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:24.123359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9135a006-e245-4380-9a6e-0a2fec504244 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:24.123799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9135a006-e245-4380-9a6e-0a2fec504244 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:24.124230 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9135a006-e245-4380-9a6e-0a2fec504244 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:24.264607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9135a006-e245-4380-9a6e-0a2fec504244 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:24.273231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9135a006-e245-4380-9a6e-0a2fec504244 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.165287 Aug 28 00:55:24.273811 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 45/100] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:24 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:24.566987 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-73f1dd79-4397-49be-bcd6-77df479e1c6f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:24.567594 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-73f1dd79-4397-49be-bcd6-77df479e1c6f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:24.576585 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73f1dd79-4397-49be-bcd6-77df479e1c6f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:24.577239 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73f1dd79-4397-49be-bcd6-77df479e1c6f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:24.577788 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73f1dd79-4397-49be-bcd6-77df479e1c6f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:24.578803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73f1dd79-4397-49be-bcd6-77df479e1c6f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:24.579250 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73f1dd79-4397-49be-bcd6-77df479e1c6f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:24.579670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73f1dd79-4397-49be-bcd6-77df479e1c6f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:24.602908 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-57f9ea1d-76f1-4ab5-b5e9-a682b7f7ae63 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:24.603507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-57f9ea1d-76f1-4ab5-b5e9-a682b7f7ae63 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:24.609401 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-57f9ea1d-76f1-4ab5-b5e9-a682b7f7ae63 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:24.609945 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-57f9ea1d-76f1-4ab5-b5e9-a682b7f7ae63 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:24.610341 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-57f9ea1d-76f1-4ab5-b5e9-a682b7f7ae63 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:24.611048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-57f9ea1d-76f1-4ab5-b5e9-a682b7f7ae63 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:24.611488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-57f9ea1d-76f1-4ab5-b5e9-a682b7f7ae63 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:24.611894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-57f9ea1d-76f1-4ab5-b5e9-a682b7f7ae63 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:24.761646 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-73f1dd79-4397-49be-bcd6-77df479e1c6f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:24.770202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-73f1dd79-4397-49be-bcd6-77df479e1c6f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.207047 Aug 28 00:55:24.770931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 56/101] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:24 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:24.775403 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ade683a5-cf7f-4068-8f0c-07f5b206a791 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:24.776075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ade683a5-cf7f-4068-8f0c-07f5b206a791 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:24.782361 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ade683a5-cf7f-4068-8f0c-07f5b206a791 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:24.783097 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ade683a5-cf7f-4068-8f0c-07f5b206a791 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:24.783640 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ade683a5-cf7f-4068-8f0c-07f5b206a791 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:24.784361 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ade683a5-cf7f-4068-8f0c-07f5b206a791 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:24.784810 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ade683a5-cf7f-4068-8f0c-07f5b206a791 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:24.785231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ade683a5-cf7f-4068-8f0c-07f5b206a791 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:24.788390 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-57f9ea1d-76f1-4ab5-b5e9-a682b7f7ae63 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:24.796708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-57f9ea1d-76f1-4ab5-b5e9-a682b7f7ae63 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:24.798334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-57f9ea1d-76f1-4ab5-b5e9-a682b7f7ae63 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:24.799485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-57f9ea1d-76f1-4ab5-b5e9-a682b7f7ae63 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.200554 Aug 28 00:55:24.800318 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 46/102] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:24 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:55:24.846847 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ade683a5-cf7f-4068-8f0c-07f5b206a791 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:24.854397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ade683a5-cf7f-4068-8f0c-07f5b206a791 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1290 microversion: 2.1 time: 0.082500 Aug 28 00:55:24.854968 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 57/103] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:24 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1290 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:25.126780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f05055d8-688f-465e-8a3e-e1eda8649148 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:25.127434 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f05055d8-688f-465e-8a3e-e1eda8649148 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:25.134611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f05055d8-688f-465e-8a3e-e1eda8649148 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:25.135187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f05055d8-688f-465e-8a3e-e1eda8649148 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:25.135577 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f05055d8-688f-465e-8a3e-e1eda8649148 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:25.136340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f05055d8-688f-465e-8a3e-e1eda8649148 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:25.136818 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f05055d8-688f-465e-8a3e-e1eda8649148 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:25.137303 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f05055d8-688f-465e-8a3e-e1eda8649148 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:25.269171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f05055d8-688f-465e-8a3e-e1eda8649148 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:25.280810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f05055d8-688f-465e-8a3e-e1eda8649148 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.157727 Aug 28 00:55:25.281768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 47/104] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:25 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:25.298006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7ee5808e-e8b5-4d87-aaf1-ce0a42c41cef tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:25.298692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7ee5808e-e8b5-4d87-aaf1-ce0a42c41cef tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:25.305334 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ee5808e-e8b5-4d87-aaf1-ce0a42c41cef tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:25.305818 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ee5808e-e8b5-4d87-aaf1-ce0a42c41cef tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:25.306253 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ee5808e-e8b5-4d87-aaf1-ce0a42c41cef tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:25.306945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ee5808e-e8b5-4d87-aaf1-ce0a42c41cef tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:25.307393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ee5808e-e8b5-4d87-aaf1-ce0a42c41cef tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:25.307811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ee5808e-e8b5-4d87-aaf1-ce0a42c41cef tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:25.441093 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7ee5808e-e8b5-4d87-aaf1-ce0a42c41cef tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:25.452053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7ee5808e-e8b5-4d87-aaf1-ce0a42c41cef tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.160110 Aug 28 00:55:25.452672 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 58/105] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:25 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:25.794261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ee5a8c73-1cb3-4b6e-a2cf-7c8e4dd112c6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:25.794924 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ee5a8c73-1cb3-4b6e-a2cf-7c8e4dd112c6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:25.801474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ee5a8c73-1cb3-4b6e-a2cf-7c8e4dd112c6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:25.801992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ee5a8c73-1cb3-4b6e-a2cf-7c8e4dd112c6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:25.802432 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ee5a8c73-1cb3-4b6e-a2cf-7c8e4dd112c6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:25.803120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ee5a8c73-1cb3-4b6e-a2cf-7c8e4dd112c6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:25.803562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ee5a8c73-1cb3-4b6e-a2cf-7c8e4dd112c6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:25.804100 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ee5a8c73-1cb3-4b6e-a2cf-7c8e4dd112c6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:25.822653 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fd2a049d-5136-4b46-a718-a041bc1aade7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:25.823378 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fd2a049d-5136-4b46-a718-a041bc1aade7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:25.830477 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd2a049d-5136-4b46-a718-a041bc1aade7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:25.831053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd2a049d-5136-4b46-a718-a041bc1aade7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:25.831538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd2a049d-5136-4b46-a718-a041bc1aade7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:25.832349 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd2a049d-5136-4b46-a718-a041bc1aade7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:25.832812 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd2a049d-5136-4b46-a718-a041bc1aade7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:25.833267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd2a049d-5136-4b46-a718-a041bc1aade7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:26.000765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ee5a8c73-1cb3-4b6e-a2cf-7c8e4dd112c6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:26.004181 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fd2a049d-5136-4b46-a718-a041bc1aade7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:26.009007 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ee5a8c73-1cb3-4b6e-a2cf-7c8e4dd112c6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.220310 Aug 28 00:55:26.009623 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 48/106] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:25 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:26.012353 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fd2a049d-5136-4b46-a718-a041bc1aade7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:26.013874 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fd2a049d-5136-4b46-a718-a041bc1aade7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:26.015006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fd2a049d-5136-4b46-a718-a041bc1aade7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.196294 Aug 28 00:55:26.015611 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 59/107] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:25 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:55:26.019118 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6d63358d-e9bf-4b88-a97d-2fe87aed0e64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:26.019741 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6d63358d-e9bf-4b88-a97d-2fe87aed0e64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:26.026036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d63358d-e9bf-4b88-a97d-2fe87aed0e64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:26.026542 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d63358d-e9bf-4b88-a97d-2fe87aed0e64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:26.026971 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d63358d-e9bf-4b88-a97d-2fe87aed0e64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:26.027643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d63358d-e9bf-4b88-a97d-2fe87aed0e64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:26.028088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d63358d-e9bf-4b88-a97d-2fe87aed0e64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:26.028592 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d63358d-e9bf-4b88-a97d-2fe87aed0e64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:26.092577 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6d63358d-e9bf-4b88-a97d-2fe87aed0e64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:26.103687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6d63358d-e9bf-4b88-a97d-2fe87aed0e64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1290 microversion: 2.1 time: 0.088362 Aug 28 00:55:26.104343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 49/108] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:26 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1290 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:26.309094 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1f31c8a0-d349-4b0e-8910-9f3f5a0eda65 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:26.309623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1f31c8a0-d349-4b0e-8910-9f3f5a0eda65 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:26.316297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1f31c8a0-d349-4b0e-8910-9f3f5a0eda65 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:26.316769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1f31c8a0-d349-4b0e-8910-9f3f5a0eda65 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:26.317218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1f31c8a0-d349-4b0e-8910-9f3f5a0eda65 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:26.317901 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1f31c8a0-d349-4b0e-8910-9f3f5a0eda65 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:26.318339 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1f31c8a0-d349-4b0e-8910-9f3f5a0eda65 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:26.318749 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1f31c8a0-d349-4b0e-8910-9f3f5a0eda65 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:26.459445 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1f31c8a0-d349-4b0e-8910-9f3f5a0eda65 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:26.467573 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1f31c8a0-d349-4b0e-8910-9f3f5a0eda65 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.162338 Aug 28 00:55:26.468457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 60/109] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:26 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:26.486002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-dd2a7dc4-bcac-40de-90ac-12240c480a87 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:26.486590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-dd2a7dc4-bcac-40de-90ac-12240c480a87 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:26.492625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dd2a7dc4-bcac-40de-90ac-12240c480a87 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:26.493135 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dd2a7dc4-bcac-40de-90ac-12240c480a87 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:26.493557 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dd2a7dc4-bcac-40de-90ac-12240c480a87 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:26.494233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dd2a7dc4-bcac-40de-90ac-12240c480a87 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:26.494675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dd2a7dc4-bcac-40de-90ac-12240c480a87 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:26.495082 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dd2a7dc4-bcac-40de-90ac-12240c480a87 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:26.631576 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dd2a7dc4-bcac-40de-90ac-12240c480a87 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:26.642239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-dd2a7dc4-bcac-40de-90ac-12240c480a87 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.160163 Aug 28 00:55:26.642821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 50/110] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:26 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:27.031260 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2714b956-af75-4887-b4c4-6d2e648abe19 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:27.032057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2714b956-af75-4887-b4c4-6d2e648abe19 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:27.038766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2714b956-af75-4887-b4c4-6d2e648abe19 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:27.039256 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2714b956-af75-4887-b4c4-6d2e648abe19 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:27.039680 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2714b956-af75-4887-b4c4-6d2e648abe19 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:27.040357 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2714b956-af75-4887-b4c4-6d2e648abe19 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:27.040825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2714b956-af75-4887-b4c4-6d2e648abe19 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:27.041276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2714b956-af75-4887-b4c4-6d2e648abe19 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:27.043828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-80093a6e-97c7-4ed6-87f3-e989188ea4f4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:27.044538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-80093a6e-97c7-4ed6-87f3-e989188ea4f4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:27.051985 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-80093a6e-97c7-4ed6-87f3-e989188ea4f4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:27.052463 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-80093a6e-97c7-4ed6-87f3-e989188ea4f4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:27.052884 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-80093a6e-97c7-4ed6-87f3-e989188ea4f4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:27.053602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-80093a6e-97c7-4ed6-87f3-e989188ea4f4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:27.054041 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-80093a6e-97c7-4ed6-87f3-e989188ea4f4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:27.054545 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-80093a6e-97c7-4ed6-87f3-e989188ea4f4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:27.392431 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2714b956-af75-4887-b4c4-6d2e648abe19 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:27.400120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-80093a6e-97c7-4ed6-87f3-e989188ea4f4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:27.401188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2714b956-af75-4887-b4c4-6d2e648abe19 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.374638 Aug 28 00:55:27.403259 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 61/111] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:27 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:27.407114 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5537ab40-9742-45f2-9b33-8d99b1833745 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:27.407877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5537ab40-9742-45f2-9b33-8d99b1833745 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:27.414663 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5537ab40-9742-45f2-9b33-8d99b1833745 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:27.415155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5537ab40-9742-45f2-9b33-8d99b1833745 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:27.415587 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5537ab40-9742-45f2-9b33-8d99b1833745 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:27.416280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5537ab40-9742-45f2-9b33-8d99b1833745 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:27.416860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5537ab40-9742-45f2-9b33-8d99b1833745 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:27.417643 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5537ab40-9742-45f2-9b33-8d99b1833745 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:27.418604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-80093a6e-97c7-4ed6-87f3-e989188ea4f4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:27.420297 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-80093a6e-97c7-4ed6-87f3-e989188ea4f4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:27.421542 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-80093a6e-97c7-4ed6-87f3-e989188ea4f4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.382004 Aug 28 00:55:27.422669 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 51/112] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:27 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:55:27.485395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5537ab40-9742-45f2-9b33-8d99b1833745 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:27.490288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-45400e1b-f4d6-444c-8e1b-b3981cab79ce tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:27.491935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-45400e1b-f4d6-444c-8e1b-b3981cab79ce tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:27.498560 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5537ab40-9742-45f2-9b33-8d99b1833745 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1290 microversion: 2.1 time: 0.094959 Aug 28 00:55:27.500138 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 62/113] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:27 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1290 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:27.501476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45400e1b-f4d6-444c-8e1b-b3981cab79ce tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:27.502244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45400e1b-f4d6-444c-8e1b-b3981cab79ce tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:27.502739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45400e1b-f4d6-444c-8e1b-b3981cab79ce tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:27.503712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45400e1b-f4d6-444c-8e1b-b3981cab79ce tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:27.504423 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45400e1b-f4d6-444c-8e1b-b3981cab79ce tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:27.505078 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45400e1b-f4d6-444c-8e1b-b3981cab79ce tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:27.652185 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-45400e1b-f4d6-444c-8e1b-b3981cab79ce tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:27.661259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-45400e1b-f4d6-444c-8e1b-b3981cab79ce tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.174878 Aug 28 00:55:27.661821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 52/114] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:27 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:55:27.667811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1708dd7f-1170-4fa5-af2e-1387af1cc7f5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:27.668415 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1708dd7f-1170-4fa5-af2e-1387af1cc7f5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:27.678306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1708dd7f-1170-4fa5-af2e-1387af1cc7f5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:27.678787 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1708dd7f-1170-4fa5-af2e-1387af1cc7f5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:27.679214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1708dd7f-1170-4fa5-af2e-1387af1cc7f5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:27.679983 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1708dd7f-1170-4fa5-af2e-1387af1cc7f5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:27.680365 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1708dd7f-1170-4fa5-af2e-1387af1cc7f5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:27.680777 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1708dd7f-1170-4fa5-af2e-1387af1cc7f5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:27.825284 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1708dd7f-1170-4fa5-af2e-1387af1cc7f5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:27.833725 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1708dd7f-1170-4fa5-af2e-1387af1cc7f5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.169759 Aug 28 00:55:27.834302 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 63/115] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:27 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:28.423224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-071b30fd-8597-4431-948c-4445f0a260e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:28.423852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-071b30fd-8597-4431-948c-4445f0a260e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:28.425881 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=5 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:28.431415 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-071b30fd-8597-4431-948c-4445f0a260e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:28.431919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-071b30fd-8597-4431-948c-4445f0a260e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:28.432330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-071b30fd-8597-4431-948c-4445f0a260e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:28.433130 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-071b30fd-8597-4431-948c-4445f0a260e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:28.433570 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-071b30fd-8597-4431-948c-4445f0a260e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:28.433957 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-071b30fd-8597-4431-948c-4445f0a260e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:28.443884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ada9cfc8-1722-4745-87e1-a2f12fd67154 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:28.444488 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ada9cfc8-1722-4745-87e1-a2f12fd67154 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:28.451221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ada9cfc8-1722-4745-87e1-a2f12fd67154 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:28.451634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:28.452316 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ada9cfc8-1722-4745-87e1-a2f12fd67154 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:28.452761 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ada9cfc8-1722-4745-87e1-a2f12fd67154 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:28.453482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ada9cfc8-1722-4745-87e1-a2f12fd67154 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:28.454056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ada9cfc8-1722-4745-87e1-a2f12fd67154 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:28.454391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ada9cfc8-1722-4745-87e1-a2f12fd67154 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:28.538491 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=4 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:28.630768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-071b30fd-8597-4431-948c-4445f0a260e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:28.640757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-071b30fd-8597-4431-948c-4445f0a260e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.221525 Aug 28 00:55:28.641814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 53/116] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:28 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:28.647946 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-53aad257-9fe9-4d70-aa46-f5bbc20ba52a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:28.648856 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-53aad257-9fe9-4d70-aa46-f5bbc20ba52a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:28.653009 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ada9cfc8-1722-4745-87e1-a2f12fd67154 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:28.659861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-53aad257-9fe9-4d70-aa46-f5bbc20ba52a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:28.660822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-53aad257-9fe9-4d70-aa46-f5bbc20ba52a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:28.661560 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-53aad257-9fe9-4d70-aa46-f5bbc20ba52a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:28.662415 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ada9cfc8-1722-4745-87e1-a2f12fd67154 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:28.662577 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-53aad257-9fe9-4d70-aa46-f5bbc20ba52a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:28.663341 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-53aad257-9fe9-4d70-aa46-f5bbc20ba52a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:28.663849 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-53aad257-9fe9-4d70-aa46-f5bbc20ba52a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:28.664161 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ada9cfc8-1722-4745-87e1-a2f12fd67154 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:28.665439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ada9cfc8-1722-4745-87e1-a2f12fd67154 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.225445 Aug 28 00:55:28.666208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 64/117] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:28 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:55:28.685492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-88859d6c-ced3-4174-b308-e6a2bd1b2e62 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:28.686459 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-88859d6c-ced3-4174-b308-e6a2bd1b2e62 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:28.692794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88859d6c-ced3-4174-b308-e6a2bd1b2e62 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:28.693374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88859d6c-ced3-4174-b308-e6a2bd1b2e62 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:28.693839 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88859d6c-ced3-4174-b308-e6a2bd1b2e62 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:28.694830 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88859d6c-ced3-4174-b308-e6a2bd1b2e62 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:28.695531 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88859d6c-ced3-4174-b308-e6a2bd1b2e62 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:28.695988 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88859d6c-ced3-4174-b308-e6a2bd1b2e62 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:28.741221 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-53aad257-9fe9-4d70-aa46-f5bbc20ba52a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:28.751496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-53aad257-9fe9-4d70-aa46-f5bbc20ba52a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1290 microversion: 2.1 time: 0.108496 Aug 28 00:55:28.752082 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 54/118] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:28 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1290 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:28.845763 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-88859d6c-ced3-4174-b308-e6a2bd1b2e62 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:28.848095 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=4 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:28.853719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2e9a4160-76de-4482-8abc-317f5cc79bde tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:28.854333 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2e9a4160-76de-4482-8abc-317f5cc79bde tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:28.871195 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2e9a4160-76de-4482-8abc-317f5cc79bde tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:28.872295 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-88859d6c-ced3-4174-b308-e6a2bd1b2e62 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.190510 Aug 28 00:55:28.872848 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2e9a4160-76de-4482-8abc-317f5cc79bde tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:28.873494 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2e9a4160-76de-4482-8abc-317f5cc79bde tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:28.886733 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 65/119] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:28 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:28.887030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2e9a4160-76de-4482-8abc-317f5cc79bde tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:28.887030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2e9a4160-76de-4482-8abc-317f5cc79bde tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:28.887030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2e9a4160-76de-4482-8abc-317f5cc79bde tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:29.033142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2e9a4160-76de-4482-8abc-317f5cc79bde tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:29.033340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2e9a4160-76de-4482-8abc-317f5cc79bde tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.181586 Aug 28 00:55:29.037449 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 55/120] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:28 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:55:29.662855 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2408a1ae-68ff-4302-8110-77f23358d088 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:29.663466 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2408a1ae-68ff-4302-8110-77f23358d088 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:29.670975 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2408a1ae-68ff-4302-8110-77f23358d088 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:29.671978 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2408a1ae-68ff-4302-8110-77f23358d088 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:29.672471 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2408a1ae-68ff-4302-8110-77f23358d088 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:29.673444 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2408a1ae-68ff-4302-8110-77f23358d088 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:29.673884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2408a1ae-68ff-4302-8110-77f23358d088 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:29.674293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2408a1ae-68ff-4302-8110-77f23358d088 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:29.690168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5f3f9bab-08cc-4b33-b626-51e2a147d6ed tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:29.693385 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5f3f9bab-08cc-4b33-b626-51e2a147d6ed tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:29.705406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f3f9bab-08cc-4b33-b626-51e2a147d6ed tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:29.706341 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f3f9bab-08cc-4b33-b626-51e2a147d6ed tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:29.707828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f3f9bab-08cc-4b33-b626-51e2a147d6ed tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:29.708668 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f3f9bab-08cc-4b33-b626-51e2a147d6ed tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:29.709218 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f3f9bab-08cc-4b33-b626-51e2a147d6ed tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:29.710046 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f3f9bab-08cc-4b33-b626-51e2a147d6ed tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:29.906438 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2408a1ae-68ff-4302-8110-77f23358d088 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:29.921486 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2408a1ae-68ff-4302-8110-77f23358d088 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.262607 Aug 28 00:55:29.922789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 66/121] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:29 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:55:29.928811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cc9d5ed5-fde2-4d02-8c32-b3bef392c7af tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:29.929887 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cc9d5ed5-fde2-4d02-8c32-b3bef392c7af tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:29.937508 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc9d5ed5-fde2-4d02-8c32-b3bef392c7af tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:29.938316 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc9d5ed5-fde2-4d02-8c32-b3bef392c7af tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:29.938939 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc9d5ed5-fde2-4d02-8c32-b3bef392c7af tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:29.939849 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc9d5ed5-fde2-4d02-8c32-b3bef392c7af tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:29.940581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc9d5ed5-fde2-4d02-8c32-b3bef392c7af tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:29.941301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc9d5ed5-fde2-4d02-8c32-b3bef392c7af tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:29.953118 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5f3f9bab-08cc-4b33-b626-51e2a147d6ed tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:29.963325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5f3f9bab-08cc-4b33-b626-51e2a147d6ed tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:29.965662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5f3f9bab-08cc-4b33-b626-51e2a147d6ed tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:29.967080 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5f3f9bab-08cc-4b33-b626-51e2a147d6ed tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.282859 Aug 28 00:55:29.967854 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 56/122] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:29 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:55:29.983751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a16687ad-c2bb-4e28-a004-3137ed3fd36b tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:29.984714 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a16687ad-c2bb-4e28-a004-3137ed3fd36b tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:29.991194 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a16687ad-c2bb-4e28-a004-3137ed3fd36b tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:29.992254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a16687ad-c2bb-4e28-a004-3137ed3fd36b tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:29.992691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a16687ad-c2bb-4e28-a004-3137ed3fd36b tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:29.993634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a16687ad-c2bb-4e28-a004-3137ed3fd36b tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:29.994105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a16687ad-c2bb-4e28-a004-3137ed3fd36b tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:29.994521 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a16687ad-c2bb-4e28-a004-3137ed3fd36b tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:30.010362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-cc9d5ed5-fde2-4d02-8c32-b3bef392c7af tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:30.018436 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cc9d5ed5-fde2-4d02-8c32-b3bef392c7af tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1290 microversion: 2.1 time: 0.094447 Aug 28 00:55:30.019235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 67/123] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:29 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1290 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:30.054949 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ce387b9d-eebb-43ac-84e6-e87bf8213c7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:30.055687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ce387b9d-eebb-43ac-84e6-e87bf8213c7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:30.062464 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce387b9d-eebb-43ac-84e6-e87bf8213c7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:30.064016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce387b9d-eebb-43ac-84e6-e87bf8213c7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:30.065050 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce387b9d-eebb-43ac-84e6-e87bf8213c7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:30.065947 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce387b9d-eebb-43ac-84e6-e87bf8213c7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:30.066708 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce387b9d-eebb-43ac-84e6-e87bf8213c7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:30.067130 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce387b9d-eebb-43ac-84e6-e87bf8213c7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:30.174560 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a16687ad-c2bb-4e28-a004-3137ed3fd36b tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:30.185887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a16687ad-c2bb-4e28-a004-3137ed3fd36b tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.207794 Aug 28 00:55:30.186745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 57/124] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:29 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:30.251821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ce387b9d-eebb-43ac-84e6-e87bf8213c7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:30.262251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ce387b9d-eebb-43ac-84e6-e87bf8213c7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.212015 Aug 28 00:55:30.270069 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 68/125] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:30 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:55:30.947715 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e7d45e47-bad2-4568-9c05-0e6fdd7ae9c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:30.948470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e7d45e47-bad2-4568-9c05-0e6fdd7ae9c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:30.956146 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e7d45e47-bad2-4568-9c05-0e6fdd7ae9c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:30.956723 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e7d45e47-bad2-4568-9c05-0e6fdd7ae9c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:30.957667 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e7d45e47-bad2-4568-9c05-0e6fdd7ae9c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:30.958702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e7d45e47-bad2-4568-9c05-0e6fdd7ae9c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:30.959474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e7d45e47-bad2-4568-9c05-0e6fdd7ae9c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:30.960148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e7d45e47-bad2-4568-9c05-0e6fdd7ae9c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:30.988908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8ef1b44e-7103-4778-a6cb-15bf9639f405 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:30.993007 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8ef1b44e-7103-4778-a6cb-15bf9639f405 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:31.005724 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ef1b44e-7103-4778-a6cb-15bf9639f405 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:31.006225 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ef1b44e-7103-4778-a6cb-15bf9639f405 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:31.006735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ef1b44e-7103-4778-a6cb-15bf9639f405 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:31.007400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ef1b44e-7103-4778-a6cb-15bf9639f405 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:31.007850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ef1b44e-7103-4778-a6cb-15bf9639f405 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:31.008310 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ef1b44e-7103-4778-a6cb-15bf9639f405 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:31.214935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e7d45e47-bad2-4568-9c05-0e6fdd7ae9c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:31.225922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e7d45e47-bad2-4568-9c05-0e6fdd7ae9c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.281547 Aug 28 00:55:31.239846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 58/126] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:30 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:31.239846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-23833792-9c5c-4c98-abf0-11649e92f3cd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:31.239846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-23833792-9c5c-4c98-abf0-11649e92f3cd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:31.244393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8ef1b44e-7103-4778-a6cb-15bf9639f405 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:31.254177 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8ef1b44e-7103-4778-a6cb-15bf9639f405 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:31.254891 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23833792-9c5c-4c98-abf0-11649e92f3cd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:31.255604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23833792-9c5c-4c98-abf0-11649e92f3cd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:31.256262 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8ef1b44e-7103-4778-a6cb-15bf9639f405 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:31.256380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23833792-9c5c-4c98-abf0-11649e92f3cd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:31.257222 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23833792-9c5c-4c98-abf0-11649e92f3cd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:31.257435 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8ef1b44e-7103-4778-a6cb-15bf9639f405 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.272384 Aug 28 00:55:31.257822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23833792-9c5c-4c98-abf0-11649e92f3cd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:31.265103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23833792-9c5c-4c98-abf0-11649e92f3cd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:31.265273 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 69/127] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:30 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:55:31.269073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-53da728e-ea0a-4931-945d-5df732f5d2e0 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:31.269726 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-53da728e-ea0a-4931-945d-5df732f5d2e0 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:31.279754 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53da728e-ea0a-4931-945d-5df732f5d2e0 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:31.280295 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53da728e-ea0a-4931-945d-5df732f5d2e0 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:31.283962 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53da728e-ea0a-4931-945d-5df732f5d2e0 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:31.284727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53da728e-ea0a-4931-945d-5df732f5d2e0 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:31.285227 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53da728e-ea0a-4931-945d-5df732f5d2e0 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:31.286026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53da728e-ea0a-4931-945d-5df732f5d2e0 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:31.369130 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-23833792-9c5c-4c98-abf0-11649e92f3cd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:31.380198 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-23833792-9c5c-4c98-abf0-11649e92f3cd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1290 microversion: 2.1 time: 0.150732 Aug 28 00:55:31.395356 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 59/128] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:31 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1290 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:31.403158 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ed334818-b837-4cf6-ab60-7dbf7338f14f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:31.404098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ed334818-b837-4cf6-ab60-7dbf7338f14f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:31.412927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed334818-b837-4cf6-ab60-7dbf7338f14f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:31.413901 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed334818-b837-4cf6-ab60-7dbf7338f14f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:31.414769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed334818-b837-4cf6-ab60-7dbf7338f14f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:31.415807 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed334818-b837-4cf6-ab60-7dbf7338f14f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:31.416459 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed334818-b837-4cf6-ab60-7dbf7338f14f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:31.417103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed334818-b837-4cf6-ab60-7dbf7338f14f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:31.689901 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-53da728e-ea0a-4931-945d-5df732f5d2e0 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:31.707235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-53da728e-ea0a-4931-945d-5df732f5d2e0 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.447682 Aug 28 00:55:31.709473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 70/129] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:31 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:31.745087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ed334818-b837-4cf6-ab60-7dbf7338f14f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:31.752788 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ed334818-b837-4cf6-ab60-7dbf7338f14f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.356257 Aug 28 00:55:31.753398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 60/130] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:31 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:32.249693 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9d5dfa39-231a-46c2-aa4a-ba1556bf585d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:32.250434 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9d5dfa39-231a-46c2-aa4a-ba1556bf585d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:32.265311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d5dfa39-231a-46c2-aa4a-ba1556bf585d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:32.265879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d5dfa39-231a-46c2-aa4a-ba1556bf585d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:32.266326 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d5dfa39-231a-46c2-aa4a-ba1556bf585d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:32.267064 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d5dfa39-231a-46c2-aa4a-ba1556bf585d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:32.267634 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d5dfa39-231a-46c2-aa4a-ba1556bf585d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:32.268044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d5dfa39-231a-46c2-aa4a-ba1556bf585d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:32.279821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7c10f252-f653-4251-9625-2bdfc356fa3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:32.280431 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7c10f252-f653-4251-9625-2bdfc356fa3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:32.294459 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c10f252-f653-4251-9625-2bdfc356fa3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:32.295148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c10f252-f653-4251-9625-2bdfc356fa3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:32.295531 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c10f252-f653-4251-9625-2bdfc356fa3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:32.296203 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c10f252-f653-4251-9625-2bdfc356fa3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:32.296809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c10f252-f653-4251-9625-2bdfc356fa3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:32.297422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c10f252-f653-4251-9625-2bdfc356fa3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:32.494431 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7c10f252-f653-4251-9625-2bdfc356fa3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:32.501351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9d5dfa39-231a-46c2-aa4a-ba1556bf585d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:32.504244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7c10f252-f653-4251-9625-2bdfc356fa3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:32.506168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7c10f252-f653-4251-9625-2bdfc356fa3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:32.507443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7c10f252-f653-4251-9625-2bdfc356fa3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.231623 Aug 28 00:55:32.508075 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 61/131] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:32 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:55:32.509057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9d5dfa39-231a-46c2-aa4a-ba1556bf585d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.264634 Aug 28 00:55:32.512992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4fef0afe-1829-4842-8a38-72ad5a389d87 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:32.513821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4fef0afe-1829-4842-8a38-72ad5a389d87 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:32.520166 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fef0afe-1829-4842-8a38-72ad5a389d87 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:32.521259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fef0afe-1829-4842-8a38-72ad5a389d87 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:32.521802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 71/132] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:32 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:32.523860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fef0afe-1829-4842-8a38-72ad5a389d87 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:32.523860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fef0afe-1829-4842-8a38-72ad5a389d87 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:32.524268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fef0afe-1829-4842-8a38-72ad5a389d87 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:32.524690 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fef0afe-1829-4842-8a38-72ad5a389d87 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:32.595199 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4fef0afe-1829-4842-8a38-72ad5a389d87 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:32.603267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4fef0afe-1829-4842-8a38-72ad5a389d87 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1290 microversion: 2.1 time: 0.093776 Aug 28 00:55:32.603607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 62/133] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:32 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1290 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:32.721666 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-94fc335c-7fc9-43c1-b5fa-a1f846cda7ea req-a00ab06f-d0ee-4c0d-bd89-f2167cc93177 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4d664b12-2edd-46e0-bcf3-4d94eb576bcd", "name": "network-vif-plugged", "status": "completed", "tag": "ffe6cab9-475d-45bb-9156-c9785ed386fd"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:55:32.730800 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3690bd57-d428-47dd-8a33-d09ad0191955 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:32.731404 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3690bd57-d428-47dd-8a33-d09ad0191955 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:32.732411 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-94fc335c-7fc9-43c1-b5fa-a1f846cda7ea req-a00ab06f-d0ee-4c0d-bd89-f2167cc93177 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:32.732886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-94fc335c-7fc9-43c1-b5fa-a1f846cda7ea req-a00ab06f-d0ee-4c0d-bd89-f2167cc93177 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:32.733369 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-94fc335c-7fc9-43c1-b5fa-a1f846cda7ea req-a00ab06f-d0ee-4c0d-bd89-f2167cc93177 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:32.739427 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3690bd57-d428-47dd-8a33-d09ad0191955 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:32.740226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3690bd57-d428-47dd-8a33-d09ad0191955 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:32.741026 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3690bd57-d428-47dd-8a33-d09ad0191955 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:32.741912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3690bd57-d428-47dd-8a33-d09ad0191955 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:32.742560 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3690bd57-d428-47dd-8a33-d09ad0191955 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:32.743239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3690bd57-d428-47dd-8a33-d09ad0191955 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:32.743900 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:32.745959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-94fc335c-7fc9-43c1-b5fa-a1f846cda7ea req-a00ab06f-d0ee-4c0d-bd89-f2167cc93177 service nova] Creating event network-vif-plugged:ffe6cab9-475d-45bb-9156-c9785ed386fd for instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd on hv-cinder-maste Aug 28 00:55:32.753417 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-94fc335c-7fc9-43c1-b5fa-a1f846cda7ea req-a00ab06f-d0ee-4c0d-bd89-f2167cc93177 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.229996 Aug 28 00:55:32.754385 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 72/134] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:55:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:55:32.776695 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e00224a5-0654-4623-b159-25733636b5df tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:32.776695 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e00224a5-0654-4623-b159-25733636b5df tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:32.783586 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e00224a5-0654-4623-b159-25733636b5df tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:32.784601 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e00224a5-0654-4623-b159-25733636b5df tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:32.786120 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e00224a5-0654-4623-b159-25733636b5df tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:32.787911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e00224a5-0654-4623-b159-25733636b5df tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:32.788997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e00224a5-0654-4623-b159-25733636b5df tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:32.790571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e00224a5-0654-4623-b159-25733636b5df tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:32.942573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3690bd57-d428-47dd-8a33-d09ad0191955 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:32.954181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3690bd57-d428-47dd-8a33-d09ad0191955 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.227777 Aug 28 00:55:32.954887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 63/135] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:32 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:32.975822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e00224a5-0654-4623-b159-25733636b5df tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:32.983518 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e00224a5-0654-4623-b159-25733636b5df tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.213213 Aug 28 00:55:32.984078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 73/136] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:32 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:55:33.531611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3caa4a96-2db6-419a-a1c8-050ede4d9985 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:33.532282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3caa4a96-2db6-419a-a1c8-050ede4d9985 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:33.536486 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-83ccd7db-4934-426f-be31-088c8902ab60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:33.537615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-83ccd7db-4934-426f-be31-088c8902ab60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:33.538748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3caa4a96-2db6-419a-a1c8-050ede4d9985 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:33.539228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3caa4a96-2db6-419a-a1c8-050ede4d9985 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:33.540007 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3caa4a96-2db6-419a-a1c8-050ede4d9985 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:33.540867 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3caa4a96-2db6-419a-a1c8-050ede4d9985 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:33.541539 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3caa4a96-2db6-419a-a1c8-050ede4d9985 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:33.542276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3caa4a96-2db6-419a-a1c8-050ede4d9985 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:33.545465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-83ccd7db-4934-426f-be31-088c8902ab60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:33.546599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-83ccd7db-4934-426f-be31-088c8902ab60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:33.547557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-83ccd7db-4934-426f-be31-088c8902ab60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:33.548388 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-83ccd7db-4934-426f-be31-088c8902ab60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:33.548989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-83ccd7db-4934-426f-be31-088c8902ab60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:33.549565 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-83ccd7db-4934-426f-be31-088c8902ab60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:34.146629 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3caa4a96-2db6-419a-a1c8-050ede4d9985 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:34.148898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-83ccd7db-4934-426f-be31-088c8902ab60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:34.158682 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3caa4a96-2db6-419a-a1c8-050ede4d9985 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.630238 Aug 28 00:55:34.159630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 64/137] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:33 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 633 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:34.164596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-83ccd7db-4934-426f-be31-088c8902ab60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:34.165264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-08678707-fb11-4ef5-8107-0f7481b290eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:34.167677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-08678707-fb11-4ef5-8107-0f7481b290eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:34.182658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08678707-fb11-4ef5-8107-0f7481b290eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:34.184022 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08678707-fb11-4ef5-8107-0f7481b290eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:34.184598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08678707-fb11-4ef5-8107-0f7481b290eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:34.188017 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-83ccd7db-4934-426f-be31-088c8902ab60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:34.188017 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-83ccd7db-4934-426f-be31-088c8902ab60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.657562 Aug 28 00:55:34.189245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08678707-fb11-4ef5-8107-0f7481b290eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:34.189679 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 74/138] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:33 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 660 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:55:34.190208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08678707-fb11-4ef5-8107-0f7481b290eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:34.190634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08678707-fb11-4ef5-8107-0f7481b290eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:34.210971 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-98431ef1-703a-44cd-8269-706c45aeff3e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:34.211734 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-98431ef1-703a-44cd-8269-706c45aeff3e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:34.218914 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98431ef1-703a-44cd-8269-706c45aeff3e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:34.219842 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98431ef1-703a-44cd-8269-706c45aeff3e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:34.220277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98431ef1-703a-44cd-8269-706c45aeff3e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:34.220990 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98431ef1-703a-44cd-8269-706c45aeff3e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:34.221456 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98431ef1-703a-44cd-8269-706c45aeff3e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:34.222001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98431ef1-703a-44cd-8269-706c45aeff3e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:34.264580 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-08678707-fb11-4ef5-8107-0f7481b290eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:34.273286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-08678707-fb11-4ef5-8107-0f7481b290eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1290 microversion: 2.1 time: 0.112285 Aug 28 00:55:34.281784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 65/139] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:34 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1290 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:34.289129 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-09e54d0a-52df-4e65-8849-4634cc9ab7b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:34.289947 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-09e54d0a-52df-4e65-8849-4634cc9ab7b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:34.297220 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09e54d0a-52df-4e65-8849-4634cc9ab7b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:34.297970 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09e54d0a-52df-4e65-8849-4634cc9ab7b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:34.298513 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09e54d0a-52df-4e65-8849-4634cc9ab7b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:34.299276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09e54d0a-52df-4e65-8849-4634cc9ab7b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:34.299811 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09e54d0a-52df-4e65-8849-4634cc9ab7b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:34.300297 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09e54d0a-52df-4e65-8849-4634cc9ab7b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:34.434875 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-98431ef1-703a-44cd-8269-706c45aeff3e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:34.444656 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-98431ef1-703a-44cd-8269-706c45aeff3e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.253523 Aug 28 00:55:34.446023 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 75/140] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:34 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:34.477036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-09e54d0a-52df-4e65-8849-4634cc9ab7b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:34.487390 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-09e54d0a-52df-4e65-8849-4634cc9ab7b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.212535 Aug 28 00:55:34.488242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 66/141] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:34 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:35.180322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-611c6199-c280-4f4a-9d3c-f0927141160c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:35.180938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-611c6199-c280-4f4a-9d3c-f0927141160c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:35.187299 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-611c6199-c280-4f4a-9d3c-f0927141160c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:35.187834 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-611c6199-c280-4f4a-9d3c-f0927141160c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:35.188261 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-611c6199-c280-4f4a-9d3c-f0927141160c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:35.189115 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-611c6199-c280-4f4a-9d3c-f0927141160c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:35.189563 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-611c6199-c280-4f4a-9d3c-f0927141160c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:35.189981 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-611c6199-c280-4f4a-9d3c-f0927141160c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:35.212094 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7063874d-1460-447f-b6b7-0ae25db14f83 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:35.212722 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7063874d-1460-447f-b6b7-0ae25db14f83 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:35.221801 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7063874d-1460-447f-b6b7-0ae25db14f83 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:35.222506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7063874d-1460-447f-b6b7-0ae25db14f83 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:35.223071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7063874d-1460-447f-b6b7-0ae25db14f83 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:35.224556 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7063874d-1460-447f-b6b7-0ae25db14f83 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:35.225064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7063874d-1460-447f-b6b7-0ae25db14f83 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:35.225676 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7063874d-1460-447f-b6b7-0ae25db14f83 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:35.405864 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-611c6199-c280-4f4a-9d3c-f0927141160c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:35.416869 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-611c6199-c280-4f4a-9d3c-f0927141160c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.239853 Aug 28 00:55:35.433039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 76/142] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:35 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:35.439772 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0c8cee0d-6b9f-45c2-ace4-44f69ef16e6a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:35.440799 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0c8cee0d-6b9f-45c2-ace4-44f69ef16e6a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:35.448268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7063874d-1460-447f-b6b7-0ae25db14f83 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:35.467451 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7063874d-1460-447f-b6b7-0ae25db14f83 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:35.467674 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0c8cee0d-6b9f-45c2-ace4-44f69ef16e6a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:35.468398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0c8cee0d-6b9f-45c2-ace4-44f69ef16e6a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:35.469033 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0c8cee0d-6b9f-45c2-ace4-44f69ef16e6a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:35.469830 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0c8cee0d-6b9f-45c2-ace4-44f69ef16e6a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:35.470252 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7063874d-1460-447f-b6b7-0ae25db14f83 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:35.470468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0c8cee0d-6b9f-45c2-ace4-44f69ef16e6a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:35.470978 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0c8cee0d-6b9f-45c2-ace4-44f69ef16e6a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:35.471838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7063874d-1460-447f-b6b7-0ae25db14f83 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.263723 Aug 28 00:55:35.472534 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 67/143] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:35 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:55:35.478091 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f66d3ea4-7ccf-4f46-80b7-d8ad8605400e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:35.478927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f66d3ea4-7ccf-4f46-80b7-d8ad8605400e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:35.485678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f66d3ea4-7ccf-4f46-80b7-d8ad8605400e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:35.486661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f66d3ea4-7ccf-4f46-80b7-d8ad8605400e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:35.487299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f66d3ea4-7ccf-4f46-80b7-d8ad8605400e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:35.488062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f66d3ea4-7ccf-4f46-80b7-d8ad8605400e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:35.488628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f66d3ea4-7ccf-4f46-80b7-d8ad8605400e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:35.489068 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f66d3ea4-7ccf-4f46-80b7-d8ad8605400e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:35.594654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0c8cee0d-6b9f-45c2-ace4-44f69ef16e6a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:35.603259 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0c8cee0d-6b9f-45c2-ace4-44f69ef16e6a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1290 microversion: 2.1 time: 0.168801 Aug 28 00:55:35.616885 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 77/144] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:35 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1290 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:35.622155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b62ba065-aabd-4e44-aca2-13f43644bf71 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:35.622995 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b62ba065-aabd-4e44-aca2-13f43644bf71 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:35.630548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b62ba065-aabd-4e44-aca2-13f43644bf71 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:35.631180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b62ba065-aabd-4e44-aca2-13f43644bf71 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:35.631830 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b62ba065-aabd-4e44-aca2-13f43644bf71 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:35.633566 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b62ba065-aabd-4e44-aca2-13f43644bf71 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:35.634163 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b62ba065-aabd-4e44-aca2-13f43644bf71 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:35.634784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b62ba065-aabd-4e44-aca2-13f43644bf71 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:35.761811 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f66d3ea4-7ccf-4f46-80b7-d8ad8605400e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:35.770078 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f66d3ea4-7ccf-4f46-80b7-d8ad8605400e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.296265 Aug 28 00:55:35.770664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 68/145] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:35 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:35.850585 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b62ba065-aabd-4e44-aca2-13f43644bf71 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:35.859235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b62ba065-aabd-4e44-aca2-13f43644bf71 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.240965 Aug 28 00:55:35.859918 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 78/146] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:35 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:36.442997 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5d0ade22-0d30-4cce-90d1-7f33384ab381 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:36.443648 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5d0ade22-0d30-4cce-90d1-7f33384ab381 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:36.451078 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d0ade22-0d30-4cce-90d1-7f33384ab381 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:36.451563 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d0ade22-0d30-4cce-90d1-7f33384ab381 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:36.452046 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d0ade22-0d30-4cce-90d1-7f33384ab381 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:36.453042 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d0ade22-0d30-4cce-90d1-7f33384ab381 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:36.453526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d0ade22-0d30-4cce-90d1-7f33384ab381 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:36.454479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d0ade22-0d30-4cce-90d1-7f33384ab381 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:36.493284 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-382c3c1d-8321-4c98-a1d0-9bbe957ce4a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:36.493975 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-382c3c1d-8321-4c98-a1d0-9bbe957ce4a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:36.500029 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-382c3c1d-8321-4c98-a1d0-9bbe957ce4a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:36.500718 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-382c3c1d-8321-4c98-a1d0-9bbe957ce4a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:36.501494 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-382c3c1d-8321-4c98-a1d0-9bbe957ce4a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:36.502434 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-382c3c1d-8321-4c98-a1d0-9bbe957ce4a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:36.503083 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-382c3c1d-8321-4c98-a1d0-9bbe957ce4a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:36.504182 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-382c3c1d-8321-4c98-a1d0-9bbe957ce4a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:36.676753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5d0ade22-0d30-4cce-90d1-7f33384ab381 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:36.688855 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5d0ade22-0d30-4cce-90d1-7f33384ab381 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.248615 Aug 28 00:55:36.690087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 69/147] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:36 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:36.698893 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c4dc0a0a-5dad-4861-990e-308bbeb1854a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:36.700135 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c4dc0a0a-5dad-4861-990e-308bbeb1854a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:36.707847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c4dc0a0a-5dad-4861-990e-308bbeb1854a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:36.708420 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c4dc0a0a-5dad-4861-990e-308bbeb1854a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:36.709355 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c4dc0a0a-5dad-4861-990e-308bbeb1854a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:36.710252 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c4dc0a0a-5dad-4861-990e-308bbeb1854a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:36.710776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c4dc0a0a-5dad-4861-990e-308bbeb1854a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:36.711266 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c4dc0a0a-5dad-4861-990e-308bbeb1854a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:36.716016 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:36.733495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-382c3c1d-8321-4c98-a1d0-9bbe957ce4a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:36.743376 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-382c3c1d-8321-4c98-a1d0-9bbe957ce4a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:36.744953 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-382c3c1d-8321-4c98-a1d0-9bbe957ce4a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:36.746495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-382c3c1d-8321-4c98-a1d0-9bbe957ce4a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.257025 Aug 28 00:55:36.762189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 79/148] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:36 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:55:36.779061 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c4dc0a0a-5dad-4861-990e-308bbeb1854a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:36.789495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c4dc0a0a-5dad-4861-990e-308bbeb1854a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1290 microversion: 2.1 time: 0.095948 Aug 28 00:55:36.790243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 70/149] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:36 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1290 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:36.792042 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-309d4bcd-6e06-4230-9837-17b8031d6ad8 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:36.792647 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-309d4bcd-6e06-4230-9837-17b8031d6ad8 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:36.799058 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-309d4bcd-6e06-4230-9837-17b8031d6ad8 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:36.799534 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-309d4bcd-6e06-4230-9837-17b8031d6ad8 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:36.799959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-309d4bcd-6e06-4230-9837-17b8031d6ad8 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:36.800650 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-309d4bcd-6e06-4230-9837-17b8031d6ad8 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:36.801112 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-309d4bcd-6e06-4230-9837-17b8031d6ad8 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:36.801679 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-309d4bcd-6e06-4230-9837-17b8031d6ad8 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:36.882772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c3c5661f-1b49-4ba1-8a14-19b09a5292d4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:36.883647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c3c5661f-1b49-4ba1-8a14-19b09a5292d4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:36.888381 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:36.894152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3c5661f-1b49-4ba1-8a14-19b09a5292d4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:36.895450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3c5661f-1b49-4ba1-8a14-19b09a5292d4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:36.896873 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3c5661f-1b49-4ba1-8a14-19b09a5292d4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:36.897766 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3c5661f-1b49-4ba1-8a14-19b09a5292d4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:36.898389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3c5661f-1b49-4ba1-8a14-19b09a5292d4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:36.898946 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3c5661f-1b49-4ba1-8a14-19b09a5292d4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:36.944625 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-309d4bcd-6e06-4230-9837-17b8031d6ad8 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:36.953589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-309d4bcd-6e06-4230-9837-17b8031d6ad8 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.165409 Aug 28 00:55:36.954571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 80/150] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:36 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:37.056447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c3c5661f-1b49-4ba1-8a14-19b09a5292d4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:37.065686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c3c5661f-1b49-4ba1-8a14-19b09a5292d4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.188010 Aug 28 00:55:37.067883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 71/151] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:36 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:37.710571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0e925cd9-8a77-46c3-98d2-f2c3c9c1365d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:37.711793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0e925cd9-8a77-46c3-98d2-f2c3c9c1365d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:37.718497 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e925cd9-8a77-46c3-98d2-f2c3c9c1365d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:37.719439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e925cd9-8a77-46c3-98d2-f2c3c9c1365d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:37.720142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e925cd9-8a77-46c3-98d2-f2c3c9c1365d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:37.720895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e925cd9-8a77-46c3-98d2-f2c3c9c1365d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:37.721601 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e925cd9-8a77-46c3-98d2-f2c3c9c1365d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:37.722240 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e925cd9-8a77-46c3-98d2-f2c3c9c1365d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:37.770288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b008e46c-d011-480a-b854-5397b3034b9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:37.771439 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b008e46c-d011-480a-b854-5397b3034b9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:37.778484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b008e46c-d011-480a-b854-5397b3034b9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:37.783810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b008e46c-d011-480a-b854-5397b3034b9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:37.784466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b008e46c-d011-480a-b854-5397b3034b9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:37.785486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b008e46c-d011-480a-b854-5397b3034b9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:37.786154 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b008e46c-d011-480a-b854-5397b3034b9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:37.786859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b008e46c-d011-480a-b854-5397b3034b9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:37.949971 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0e925cd9-8a77-46c3-98d2-f2c3c9c1365d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:37.958549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0e925cd9-8a77-46c3-98d2-f2c3c9c1365d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.252361 Aug 28 00:55:37.959678 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 81/152] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:37 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:37.966018 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-84324dd1-8011-478f-9541-879a9c2ee136 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:37.966830 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-84324dd1-8011-478f-9541-879a9c2ee136 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:37.973716 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-84324dd1-8011-478f-9541-879a9c2ee136 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:37.974471 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-84324dd1-8011-478f-9541-879a9c2ee136 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:37.975084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-84324dd1-8011-478f-9541-879a9c2ee136 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:37.976014 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-84324dd1-8011-478f-9541-879a9c2ee136 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:37.976555 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-84324dd1-8011-478f-9541-879a9c2ee136 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:37.977569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-84324dd1-8011-478f-9541-879a9c2ee136 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:37.993883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b008e46c-d011-480a-b854-5397b3034b9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:38.014104 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b008e46c-d011-480a-b854-5397b3034b9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:38.016323 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b008e46c-d011-480a-b854-5397b3034b9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:38.017656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b008e46c-d011-480a-b854-5397b3034b9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.251124 Aug 28 00:55:38.018716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 72/153] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:37 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:55:38.023750 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-74bfd4f4-efb9-4fde-b869-163be15c6b6c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:38.024507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-74bfd4f4-efb9-4fde-b869-163be15c6b6c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:38.032763 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-74bfd4f4-efb9-4fde-b869-163be15c6b6c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:38.033612 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-74bfd4f4-efb9-4fde-b869-163be15c6b6c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:38.034497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-74bfd4f4-efb9-4fde-b869-163be15c6b6c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:38.035703 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-74bfd4f4-efb9-4fde-b869-163be15c6b6c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:38.036287 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-74bfd4f4-efb9-4fde-b869-163be15c6b6c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:38.037145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-74bfd4f4-efb9-4fde-b869-163be15c6b6c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:38.050716 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-84324dd1-8011-478f-9541-879a9c2ee136 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:38.058577 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-84324dd1-8011-478f-9541-879a9c2ee136 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1290 microversion: 2.1 time: 0.097690 Aug 28 00:55:38.059149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 82/154] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:37 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1290 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:38.089626 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5582b19e-a749-4e42-9b5a-3967f44a90ef tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:38.090720 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5582b19e-a749-4e42-9b5a-3967f44a90ef tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:38.097271 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5582b19e-a749-4e42-9b5a-3967f44a90ef tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:38.097967 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5582b19e-a749-4e42-9b5a-3967f44a90ef tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:38.098546 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5582b19e-a749-4e42-9b5a-3967f44a90ef tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:38.099379 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5582b19e-a749-4e42-9b5a-3967f44a90ef tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:38.099970 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5582b19e-a749-4e42-9b5a-3967f44a90ef tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:38.100530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5582b19e-a749-4e42-9b5a-3967f44a90ef tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:38.286504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-74bfd4f4-efb9-4fde-b869-163be15c6b6c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:38.301375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-74bfd4f4-efb9-4fde-b869-163be15c6b6c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.276530 Aug 28 00:55:38.302734 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5582b19e-a749-4e42-9b5a-3967f44a90ef tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:38.311712 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5582b19e-a749-4e42-9b5a-3967f44a90ef tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.226746 Aug 28 00:55:38.312326 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 83/155] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:38 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:38.315612 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 73/156] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:38 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:38.979846 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2aed7359-418e-4d43-9ef7-76514dd2c7c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:38.980446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2aed7359-418e-4d43-9ef7-76514dd2c7c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:38.988112 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2aed7359-418e-4d43-9ef7-76514dd2c7c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:38.988596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2aed7359-418e-4d43-9ef7-76514dd2c7c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:38.989054 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2aed7359-418e-4d43-9ef7-76514dd2c7c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:38.989732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2aed7359-418e-4d43-9ef7-76514dd2c7c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:38.990172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2aed7359-418e-4d43-9ef7-76514dd2c7c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:38.992024 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2aed7359-418e-4d43-9ef7-76514dd2c7c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:39.040059 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-992e89b1-0cfe-4dd5-8e52-c0206d05a12b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:39.044146 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-992e89b1-0cfe-4dd5-8e52-c0206d05a12b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:39.059703 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-992e89b1-0cfe-4dd5-8e52-c0206d05a12b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:39.060209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-992e89b1-0cfe-4dd5-8e52-c0206d05a12b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:39.060932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-992e89b1-0cfe-4dd5-8e52-c0206d05a12b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:39.064838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-992e89b1-0cfe-4dd5-8e52-c0206d05a12b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:39.065348 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-992e89b1-0cfe-4dd5-8e52-c0206d05a12b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:39.065780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-992e89b1-0cfe-4dd5-8e52-c0206d05a12b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:39.276670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2aed7359-418e-4d43-9ef7-76514dd2c7c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:39.284816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2aed7359-418e-4d43-9ef7-76514dd2c7c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.308766 Aug 28 00:55:39.297004 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 84/157] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:38 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:39.297004 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-721915e2-4d2d-403f-8f57-d9774329e16a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:39.297004 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-721915e2-4d2d-403f-8f57-d9774329e16a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:39.308677 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-721915e2-4d2d-403f-8f57-d9774329e16a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:39.309271 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-721915e2-4d2d-403f-8f57-d9774329e16a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:39.309817 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-721915e2-4d2d-403f-8f57-d9774329e16a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:39.310017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-992e89b1-0cfe-4dd5-8e52-c0206d05a12b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:39.310752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-721915e2-4d2d-403f-8f57-d9774329e16a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:39.311293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-721915e2-4d2d-403f-8f57-d9774329e16a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:39.311713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-721915e2-4d2d-403f-8f57-d9774329e16a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:39.325565 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-992e89b1-0cfe-4dd5-8e52-c0206d05a12b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:39.327391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-992e89b1-0cfe-4dd5-8e52-c0206d05a12b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:39.328853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-992e89b1-0cfe-4dd5-8e52-c0206d05a12b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.292733 Aug 28 00:55:39.343941 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 74/158] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:39 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:55:39.349166 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4a308ef7-da01-460f-aa25-cabc52b8cca6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:39.350071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4a308ef7-da01-460f-aa25-cabc52b8cca6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:39.356550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a308ef7-da01-460f-aa25-cabc52b8cca6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:39.357122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a308ef7-da01-460f-aa25-cabc52b8cca6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:39.357806 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a308ef7-da01-460f-aa25-cabc52b8cca6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:39.358589 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a308ef7-da01-460f-aa25-cabc52b8cca6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:39.359319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a308ef7-da01-460f-aa25-cabc52b8cca6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:39.359794 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a308ef7-da01-460f-aa25-cabc52b8cca6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:39.396177 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-721915e2-4d2d-403f-8f57-d9774329e16a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:39.405304 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-721915e2-4d2d-403f-8f57-d9774329e16a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1290 microversion: 2.1 time: 0.118819 Aug 28 00:55:39.405947 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 85/159] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:39 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1290 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:39.410721 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-26f07cdc-d053-482a-a868-9628e3c992d6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:39.411567 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-26f07cdc-d053-482a-a868-9628e3c992d6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:39.418568 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:39.420317 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26f07cdc-d053-482a-a868-9628e3c992d6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:39.420854 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26f07cdc-d053-482a-a868-9628e3c992d6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:39.421311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26f07cdc-d053-482a-a868-9628e3c992d6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:39.422051 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26f07cdc-d053-482a-a868-9628e3c992d6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:39.422494 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26f07cdc-d053-482a-a868-9628e3c992d6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:39.422898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26f07cdc-d053-482a-a868-9628e3c992d6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:39.614947 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4a308ef7-da01-460f-aa25-cabc52b8cca6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:39.623802 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4a308ef7-da01-460f-aa25-cabc52b8cca6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.278343 Aug 28 00:55:39.638213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 75/160] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:39 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:39.665346 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-26f07cdc-d053-482a-a868-9628e3c992d6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:39.682905 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-26f07cdc-d053-482a-a868-9628e3c992d6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.275661 Aug 28 00:55:39.684792 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 86/161] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:39 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:40.313033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fba0aa10-e7d0-44df-b661-7cb357cf1e3c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:40.315209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fba0aa10-e7d0-44df-b661-7cb357cf1e3c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:40.330023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fba0aa10-e7d0-44df-b661-7cb357cf1e3c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:40.338269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fba0aa10-e7d0-44df-b661-7cb357cf1e3c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:40.339150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fba0aa10-e7d0-44df-b661-7cb357cf1e3c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:40.343314 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fba0aa10-e7d0-44df-b661-7cb357cf1e3c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:40.345508 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fba0aa10-e7d0-44df-b661-7cb357cf1e3c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:40.347531 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fba0aa10-e7d0-44df-b661-7cb357cf1e3c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:40.353481 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c3611712-43c1-4ba0-a7a4-ac1fd1d2ea79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:40.354301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c3611712-43c1-4ba0-a7a4-ac1fd1d2ea79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:40.372483 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3611712-43c1-4ba0-a7a4-ac1fd1d2ea79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:40.373215 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3611712-43c1-4ba0-a7a4-ac1fd1d2ea79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:40.373808 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3611712-43c1-4ba0-a7a4-ac1fd1d2ea79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:40.376826 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3611712-43c1-4ba0-a7a4-ac1fd1d2ea79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:40.377683 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3611712-43c1-4ba0-a7a4-ac1fd1d2ea79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:40.379209 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3611712-43c1-4ba0-a7a4-ac1fd1d2ea79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:40.791469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fba0aa10-e7d0-44df-b661-7cb357cf1e3c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:40.804404 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fba0aa10-e7d0-44df-b661-7cb357cf1e3c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.495291 Aug 28 00:55:40.806343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 76/162] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:40 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:40.833746 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-096531a6-0025-48f7-b95b-15778df22c22 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:40.834726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-096531a6-0025-48f7-b95b-15778df22c22 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:40.845466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-096531a6-0025-48f7-b95b-15778df22c22 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:40.847753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-096531a6-0025-48f7-b95b-15778df22c22 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:40.849968 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-096531a6-0025-48f7-b95b-15778df22c22 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:40.851718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-096531a6-0025-48f7-b95b-15778df22c22 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:40.853332 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-096531a6-0025-48f7-b95b-15778df22c22 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:40.854510 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-096531a6-0025-48f7-b95b-15778df22c22 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:41.408162 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-096531a6-0025-48f7-b95b-15778df22c22 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:41.426423 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-096531a6-0025-48f7-b95b-15778df22c22 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1296 microversion: 2.1 time: 0.597353 Aug 28 00:55:41.427031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 77/163] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:40 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1296 bytes in 620 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:41.433622 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-44049074-a3f6-4e1c-850d-f52a8f406d4f req-72cb9019-370e-4a85-9cbf-da9c508a006f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f5f7fe76-9095-4d8d-967f-be0009648de9", "name": "network-vif-plugged", "status": "completed", "tag": "ff741c2a-17ad-4c15-a6a0-687d9dfcad8b"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:55:41.455472 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-44049074-a3f6-4e1c-850d-f52a8f406d4f req-72cb9019-370e-4a85-9cbf-da9c508a006f service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:41.469663 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-44049074-a3f6-4e1c-850d-f52a8f406d4f req-72cb9019-370e-4a85-9cbf-da9c508a006f service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:41.470082 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-44049074-a3f6-4e1c-850d-f52a8f406d4f req-72cb9019-370e-4a85-9cbf-da9c508a006f service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:41.483955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-44049074-a3f6-4e1c-850d-f52a8f406d4f req-72cb9019-370e-4a85-9cbf-da9c508a006f service nova] Creating event network-vif-plugged:ff741c2a-17ad-4c15-a6a0-687d9dfcad8b for instance f5f7fe76-9095-4d8d-967f-be0009648de9 on hv-cinder-maste Aug 28 00:55:41.503363 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-44049074-a3f6-4e1c-850d-f52a8f406d4f req-72cb9019-370e-4a85-9cbf-da9c508a006f service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.073973 Aug 28 00:55:41.504740 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 78/164] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:55:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:55:41.518369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ba9d4a00-a470-4dfa-9a80-c36fc0bfc3d7 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:41.519448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ba9d4a00-a470-4dfa-9a80-c36fc0bfc3d7 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:41.531018 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ba9d4a00-a470-4dfa-9a80-c36fc0bfc3d7 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:41.531783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ba9d4a00-a470-4dfa-9a80-c36fc0bfc3d7 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:41.532558 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ba9d4a00-a470-4dfa-9a80-c36fc0bfc3d7 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:41.534113 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ba9d4a00-a470-4dfa-9a80-c36fc0bfc3d7 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:41.535046 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ba9d4a00-a470-4dfa-9a80-c36fc0bfc3d7 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:41.536540 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ba9d4a00-a470-4dfa-9a80-c36fc0bfc3d7 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:41.576563 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c3611712-43c1-4ba0-a7a4-ac1fd1d2ea79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:41.593595 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c3611712-43c1-4ba0-a7a4-ac1fd1d2ea79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:41.596839 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c3611712-43c1-4ba0-a7a4-ac1fd1d2ea79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:41.598820 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c3611712-43c1-4ba0-a7a4-ac1fd1d2ea79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 1.250721 Aug 28 00:55:41.600466 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 87/165] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:40 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 1253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:55:41.607460 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2d115d3b-85a8-48fa-8a5e-940ca20d89b9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:41.609155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2d115d3b-85a8-48fa-8a5e-940ca20d89b9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:41.616117 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d115d3b-85a8-48fa-8a5e-940ca20d89b9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:41.617379 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d115d3b-85a8-48fa-8a5e-940ca20d89b9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:41.618099 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d115d3b-85a8-48fa-8a5e-940ca20d89b9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:41.619016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d115d3b-85a8-48fa-8a5e-940ca20d89b9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:41.619813 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d115d3b-85a8-48fa-8a5e-940ca20d89b9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:41.620462 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d115d3b-85a8-48fa-8a5e-940ca20d89b9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:41.791631 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ba9d4a00-a470-4dfa-9a80-c36fc0bfc3d7 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:41.805762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ba9d4a00-a470-4dfa-9a80-c36fc0bfc3d7 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.293406 Aug 28 00:55:41.806344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 79/166] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:41 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:41.828753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e1eea715-fd6e-46b5-986c-5a11d320c10f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:41.829399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e1eea715-fd6e-46b5-986c-5a11d320c10f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:41.838175 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1eea715-fd6e-46b5-986c-5a11d320c10f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:41.838665 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1eea715-fd6e-46b5-986c-5a11d320c10f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:41.839086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1eea715-fd6e-46b5-986c-5a11d320c10f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:41.839778 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1eea715-fd6e-46b5-986c-5a11d320c10f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:41.840219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1eea715-fd6e-46b5-986c-5a11d320c10f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:41.840643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1eea715-fd6e-46b5-986c-5a11d320c10f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:42.034183 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2d115d3b-85a8-48fa-8a5e-940ca20d89b9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:42.044541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2d115d3b-85a8-48fa-8a5e-940ca20d89b9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.441262 Aug 28 00:55:42.052669 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 88/167] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:41 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:42.359917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e1eea715-fd6e-46b5-986c-5a11d320c10f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:42.374339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e1eea715-fd6e-46b5-986c-5a11d320c10f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.550254 Aug 28 00:55:42.379013 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 80/168] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:41 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 555 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:42.457147 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3ae79030-e3d0-4fa7-afa1-4e28d1d00024 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:42.457930 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3ae79030-e3d0-4fa7-afa1-4e28d1d00024 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:42.465762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ae79030-e3d0-4fa7-afa1-4e28d1d00024 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:42.466506 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ae79030-e3d0-4fa7-afa1-4e28d1d00024 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:42.467439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ae79030-e3d0-4fa7-afa1-4e28d1d00024 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:42.472091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ae79030-e3d0-4fa7-afa1-4e28d1d00024 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:42.472091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ae79030-e3d0-4fa7-afa1-4e28d1d00024 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:42.472091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ae79030-e3d0-4fa7-afa1-4e28d1d00024 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:42.496185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:42.623536 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0bcaa041-97aa-4dbf-b5a7-fc8e7004684d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:42.624418 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0bcaa041-97aa-4dbf-b5a7-fc8e7004684d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:42.639472 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0bcaa041-97aa-4dbf-b5a7-fc8e7004684d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:42.640709 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0bcaa041-97aa-4dbf-b5a7-fc8e7004684d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:42.641174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0bcaa041-97aa-4dbf-b5a7-fc8e7004684d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:42.642250 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0bcaa041-97aa-4dbf-b5a7-fc8e7004684d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:42.642835 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0bcaa041-97aa-4dbf-b5a7-fc8e7004684d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:42.643391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0bcaa041-97aa-4dbf-b5a7-fc8e7004684d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:42.666551 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3ae79030-e3d0-4fa7-afa1-4e28d1d00024 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:42.675237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3ae79030-e3d0-4fa7-afa1-4e28d1d00024 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1364 microversion: 2.1 time: 0.222450 Aug 28 00:55:42.676238 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 89/169] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:42 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1364 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:42.822790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0bcaa041-97aa-4dbf-b5a7-fc8e7004684d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:42.827988 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a529d0a5-cea1-4ca7-9bb5-cfceeb801c1a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:42.828769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a529d0a5-cea1-4ca7-9bb5-cfceeb801c1a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:42.837720 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0bcaa041-97aa-4dbf-b5a7-fc8e7004684d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:42.839364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0bcaa041-97aa-4dbf-b5a7-fc8e7004684d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:42.840459 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a529d0a5-cea1-4ca7-9bb5-cfceeb801c1a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:42.840938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a529d0a5-cea1-4ca7-9bb5-cfceeb801c1a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:42.841087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0bcaa041-97aa-4dbf-b5a7-fc8e7004684d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.222649 Aug 28 00:55:42.841394 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a529d0a5-cea1-4ca7-9bb5-cfceeb801c1a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:42.841739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 81/170] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:42 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:55:42.845236 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a529d0a5-cea1-4ca7-9bb5-cfceeb801c1a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:42.845724 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a529d0a5-cea1-4ca7-9bb5-cfceeb801c1a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:42.846178 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a529d0a5-cea1-4ca7-9bb5-cfceeb801c1a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:43.000239 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a529d0a5-cea1-4ca7-9bb5-cfceeb801c1a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:43.008145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a529d0a5-cea1-4ca7-9bb5-cfceeb801c1a tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.185888 Aug 28 00:55:43.009326 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 90/171] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:42 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:43.067359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d092f322-7bde-4fae-8e7d-14bfde5a1413 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:43.067860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d092f322-7bde-4fae-8e7d-14bfde5a1413 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:43.077717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d092f322-7bde-4fae-8e7d-14bfde5a1413 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:43.078775 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d092f322-7bde-4fae-8e7d-14bfde5a1413 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:43.079571 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d092f322-7bde-4fae-8e7d-14bfde5a1413 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:43.080351 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d092f322-7bde-4fae-8e7d-14bfde5a1413 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:43.080912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d092f322-7bde-4fae-8e7d-14bfde5a1413 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:43.082081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d092f322-7bde-4fae-8e7d-14bfde5a1413 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:43.260078 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d092f322-7bde-4fae-8e7d-14bfde5a1413 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:43.270593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d092f322-7bde-4fae-8e7d-14bfde5a1413 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.207763 Aug 28 00:55:43.272264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 82/172] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:43 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:43.398836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e122cff0-f1dd-45e7-8e9e-8da4a9ab1e70 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:43.399578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e122cff0-f1dd-45e7-8e9e-8da4a9ab1e70 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:43.406538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e122cff0-f1dd-45e7-8e9e-8da4a9ab1e70 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:43.407017 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e122cff0-f1dd-45e7-8e9e-8da4a9ab1e70 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:43.409900 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e122cff0-f1dd-45e7-8e9e-8da4a9ab1e70 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:43.411054 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e122cff0-f1dd-45e7-8e9e-8da4a9ab1e70 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:43.412993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e122cff0-f1dd-45e7-8e9e-8da4a9ab1e70 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:43.414704 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e122cff0-f1dd-45e7-8e9e-8da4a9ab1e70 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:43.572914 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e122cff0-f1dd-45e7-8e9e-8da4a9ab1e70 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:43.583895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e122cff0-f1dd-45e7-8e9e-8da4a9ab1e70 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.188948 Aug 28 00:55:43.584473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 91/173] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:43 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:43.704852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-830aa5ba-595f-42f1-9ef8-d7057bacc10a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:43.706842 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-830aa5ba-595f-42f1-9ef8-d7057bacc10a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:43.717400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-830aa5ba-595f-42f1-9ef8-d7057bacc10a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:43.718773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-830aa5ba-595f-42f1-9ef8-d7057bacc10a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:43.719573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-830aa5ba-595f-42f1-9ef8-d7057bacc10a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:43.720839 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-830aa5ba-595f-42f1-9ef8-d7057bacc10a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:43.721664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-830aa5ba-595f-42f1-9ef8-d7057bacc10a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:43.722416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-830aa5ba-595f-42f1-9ef8-d7057bacc10a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:43.867106 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d2a3bace-4b7b-4dc5-8d98-a9cb63d098df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:43.868311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d2a3bace-4b7b-4dc5-8d98-a9cb63d098df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:43.885979 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2a3bace-4b7b-4dc5-8d98-a9cb63d098df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:43.887142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2a3bace-4b7b-4dc5-8d98-a9cb63d098df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:43.888445 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2a3bace-4b7b-4dc5-8d98-a9cb63d098df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:43.890259 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2a3bace-4b7b-4dc5-8d98-a9cb63d098df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:43.891428 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2a3bace-4b7b-4dc5-8d98-a9cb63d098df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:43.892346 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2a3bace-4b7b-4dc5-8d98-a9cb63d098df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:43.909320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-830aa5ba-595f-42f1-9ef8-d7057bacc10a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:43.916953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-830aa5ba-595f-42f1-9ef8-d7057bacc10a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1364 microversion: 2.1 time: 0.220695 Aug 28 00:55:43.924848 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 83/174] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:43 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1364 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:44.029080 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-46c4c090-ad73-422d-a39a-d9cebc947d9d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:44.030038 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-46c4c090-ad73-422d-a39a-d9cebc947d9d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:44.036305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46c4c090-ad73-422d-a39a-d9cebc947d9d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:44.037049 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46c4c090-ad73-422d-a39a-d9cebc947d9d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:44.037647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46c4c090-ad73-422d-a39a-d9cebc947d9d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:44.038520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46c4c090-ad73-422d-a39a-d9cebc947d9d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:44.039148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46c4c090-ad73-422d-a39a-d9cebc947d9d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:44.039896 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46c4c090-ad73-422d-a39a-d9cebc947d9d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:44.090931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d2a3bace-4b7b-4dc5-8d98-a9cb63d098df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:44.100023 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d2a3bace-4b7b-4dc5-8d98-a9cb63d098df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:44.101488 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d2a3bace-4b7b-4dc5-8d98-a9cb63d098df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:44.102865 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d2a3bace-4b7b-4dc5-8d98-a9cb63d098df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.242559 Aug 28 00:55:44.103557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 92/175] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:43 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:55:44.186173 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-46c4c090-ad73-422d-a39a-d9cebc947d9d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:44.193744 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-46c4c090-ad73-422d-a39a-d9cebc947d9d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.168435 Aug 28 00:55:44.194306 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 84/176] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:44 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:55:44.303889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ea6222eb-8652-4abd-85aa-8307a07cdaba tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:44.307640 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ea6222eb-8652-4abd-85aa-8307a07cdaba tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:44.348495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea6222eb-8652-4abd-85aa-8307a07cdaba tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:44.349948 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea6222eb-8652-4abd-85aa-8307a07cdaba tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:44.351288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea6222eb-8652-4abd-85aa-8307a07cdaba tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:44.353413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea6222eb-8652-4abd-85aa-8307a07cdaba tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:44.354182 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea6222eb-8652-4abd-85aa-8307a07cdaba tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:44.354996 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea6222eb-8652-4abd-85aa-8307a07cdaba tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:44.594035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-3b2d4dd3-4fe4-4a00-8615-7b773885131f req-02711c3e-cdb6-4953-9e49-4c4148b9193b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dff36aeb-f0c7-41f0-8ecc-9924adf8289c", "name": "network-vif-plugged", "status": "completed", "tag": "072b8279-6084-43c1-8f0e-c1a705ec9b99"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:55:44.613281 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-3b2d4dd3-4fe4-4a00-8615-7b773885131f req-02711c3e-cdb6-4953-9e49-4c4148b9193b service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:44.614511 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-3b2d4dd3-4fe4-4a00-8615-7b773885131f req-02711c3e-cdb6-4953-9e49-4c4148b9193b service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:44.615243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-3b2d4dd3-4fe4-4a00-8615-7b773885131f req-02711c3e-cdb6-4953-9e49-4c4148b9193b service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:44.631575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ea6222eb-8652-4abd-85aa-8307a07cdaba tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:44.634756 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-3b2d4dd3-4fe4-4a00-8615-7b773885131f req-02711c3e-cdb6-4953-9e49-4c4148b9193b service nova] Creating event network-vif-plugged:072b8279-6084-43c1-8f0e-c1a705ec9b99 for instance dff36aeb-f0c7-41f0-8ecc-9924adf8289c on hv-cinder-maste Aug 28 00:55:44.642301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ea6222eb-8652-4abd-85aa-8307a07cdaba tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.354203 Aug 28 00:55:44.642599 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-3b2d4dd3-4fe4-4a00-8615-7b773885131f req-02711c3e-cdb6-4953-9e49-4c4148b9193b service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.053175 Aug 28 00:55:44.643569 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 85/177] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:55:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:55:44.649649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 93/178] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:44 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:44.654196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8f64d7c6-74bc-49c8-9b23-a39eb49278ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:44.655197 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8f64d7c6-74bc-49c8-9b23-a39eb49278ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:44.667144 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f64d7c6-74bc-49c8-9b23-a39eb49278ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:44.670219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f64d7c6-74bc-49c8-9b23-a39eb49278ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:44.670219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f64d7c6-74bc-49c8-9b23-a39eb49278ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:44.670219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f64d7c6-74bc-49c8-9b23-a39eb49278ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:44.670219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f64d7c6-74bc-49c8-9b23-a39eb49278ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:44.670755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f64d7c6-74bc-49c8-9b23-a39eb49278ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:44.863037 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8f64d7c6-74bc-49c8-9b23-a39eb49278ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:44.873498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8f64d7c6-74bc-49c8-9b23-a39eb49278ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.227832 Aug 28 00:55:44.874999 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 86/179] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:44 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:44.942971 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-10359ab9-c8c7-420e-8c5b-4d7cd78c52a4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:44.944930 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-10359ab9-c8c7-420e-8c5b-4d7cd78c52a4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:44.960858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10359ab9-c8c7-420e-8c5b-4d7cd78c52a4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:44.964100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10359ab9-c8c7-420e-8c5b-4d7cd78c52a4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:44.966828 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10359ab9-c8c7-420e-8c5b-4d7cd78c52a4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:44.969591 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10359ab9-c8c7-420e-8c5b-4d7cd78c52a4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:44.970307 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10359ab9-c8c7-420e-8c5b-4d7cd78c52a4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:44.971125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10359ab9-c8c7-420e-8c5b-4d7cd78c52a4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:45.127844 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-56974c30-5ae2-407c-90fc-acb301c9684e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:45.128453 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-56974c30-5ae2-407c-90fc-acb301c9684e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:45.138937 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56974c30-5ae2-407c-90fc-acb301c9684e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:45.139427 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56974c30-5ae2-407c-90fc-acb301c9684e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:45.139838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56974c30-5ae2-407c-90fc-acb301c9684e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:45.140580 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56974c30-5ae2-407c-90fc-acb301c9684e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:45.141043 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56974c30-5ae2-407c-90fc-acb301c9684e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:45.145318 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56974c30-5ae2-407c-90fc-acb301c9684e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:45.499996 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-10359ab9-c8c7-420e-8c5b-4d7cd78c52a4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:45.509336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-10359ab9-c8c7-420e-8c5b-4d7cd78c52a4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1364 microversion: 2.1 time: 0.571433 Aug 28 00:55:45.510394 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 94/180] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:44 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1364 bytes in 574 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:45.515036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-56974c30-5ae2-407c-90fc-acb301c9684e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:45.521694 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c1acdc31-0f17-4c56-a658-74e4e454a54e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:45.522311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c1acdc31-0f17-4c56-a658-74e4e454a54e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:45.528898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1acdc31-0f17-4c56-a658-74e4e454a54e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:45.529735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1acdc31-0f17-4c56-a658-74e4e454a54e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:45.530399 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1acdc31-0f17-4c56-a658-74e4e454a54e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:45.531376 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1acdc31-0f17-4c56-a658-74e4e454a54e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:45.532032 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1acdc31-0f17-4c56-a658-74e4e454a54e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:45.532638 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1acdc31-0f17-4c56-a658-74e4e454a54e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:45.557647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-56974c30-5ae2-407c-90fc-acb301c9684e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:45.559790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-56974c30-5ae2-407c-90fc-acb301c9684e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:45.561010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-56974c30-5ae2-407c-90fc-acb301c9684e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.438609 Aug 28 00:55:45.561971 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 87/181] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:45 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:55:45.568748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-69364a79-fc13-4076-8b14-3ea72853ba56 req-88bad086-e771-46a1-b6a8-174d3626206f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e434e4f9-9d0e-46f2-bea3-395bae33fc92", "tag": "cb80f35b-738d-4eef-8768-99836091e5a6"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:55:45.583504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-69364a79-fc13-4076-8b14-3ea72853ba56 req-88bad086-e771-46a1-b6a8-174d3626206f service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:45.584222 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-69364a79-fc13-4076-8b14-3ea72853ba56 req-88bad086-e771-46a1-b6a8-174d3626206f service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:45.584842 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-69364a79-fc13-4076-8b14-3ea72853ba56 req-88bad086-e771-46a1-b6a8-174d3626206f service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:45.590501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:45.611097 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-69364a79-fc13-4076-8b14-3ea72853ba56 req-88bad086-e771-46a1-b6a8-174d3626206f service nova] Creating event network-changed:cb80f35b-738d-4eef-8768-99836091e5a6 for instance e434e4f9-9d0e-46f2-bea3-395bae33fc92 on hv-cinder-maste Aug 28 00:55:45.617099 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-69364a79-fc13-4076-8b14-3ea72853ba56 req-88bad086-e771-46a1-b6a8-174d3626206f service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.053399 Aug 28 00:55:45.617947 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 88/182] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:55:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:55:45.671858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6ea3a8df-451a-4e7e-be54-e7b81e906ac0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:45.673094 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6ea3a8df-451a-4e7e-be54-e7b81e906ac0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:45.681917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ea3a8df-451a-4e7e-be54-e7b81e906ac0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:45.682853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ea3a8df-451a-4e7e-be54-e7b81e906ac0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:45.683503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ea3a8df-451a-4e7e-be54-e7b81e906ac0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:45.684432 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ea3a8df-451a-4e7e-be54-e7b81e906ac0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:45.685119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ea3a8df-451a-4e7e-be54-e7b81e906ac0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:45.685745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ea3a8df-451a-4e7e-be54-e7b81e906ac0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:45.779551 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c1acdc31-0f17-4c56-a658-74e4e454a54e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:45.794148 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c1acdc31-0f17-4c56-a658-74e4e454a54e tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.281815 Aug 28 00:55:45.795468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 95/183] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:45 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:45.906711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6ea3a8df-451a-4e7e-be54-e7b81e906ac0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:45.911648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-64dabe8d-610a-4fd2-838a-01c580e65bcc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:45.913593 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-64dabe8d-610a-4fd2-838a-01c580e65bcc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:45.924646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6ea3a8df-451a-4e7e-be54-e7b81e906ac0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.257259 Aug 28 00:55:45.933879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-64dabe8d-610a-4fd2-838a-01c580e65bcc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:45.935546 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 89/184] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:45 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:45.936353 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-64dabe8d-610a-4fd2-838a-01c580e65bcc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:45.937553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-64dabe8d-610a-4fd2-838a-01c580e65bcc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:45.939065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-64dabe8d-610a-4fd2-838a-01c580e65bcc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:45.939945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-64dabe8d-610a-4fd2-838a-01c580e65bcc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:45.940947 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-64dabe8d-610a-4fd2-838a-01c580e65bcc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:46.102439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-64dabe8d-610a-4fd2-838a-01c580e65bcc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:46.111400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-64dabe8d-610a-4fd2-838a-01c580e65bcc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.219590 Aug 28 00:55:46.112260 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 96/185] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:45 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:46.531606 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-10609e42-372a-41a4-9932-34ccd0cd3989 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:46.532213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-10609e42-372a-41a4-9932-34ccd0cd3989 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:46.538597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10609e42-372a-41a4-9932-34ccd0cd3989 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:46.539149 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10609e42-372a-41a4-9932-34ccd0cd3989 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:46.539596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10609e42-372a-41a4-9932-34ccd0cd3989 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:46.540298 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10609e42-372a-41a4-9932-34ccd0cd3989 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:46.540742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10609e42-372a-41a4-9932-34ccd0cd3989 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:46.541365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10609e42-372a-41a4-9932-34ccd0cd3989 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:46.584922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-54047782-5b52-468f-8785-4b1e95a9707c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:46.585877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-54047782-5b52-468f-8785-4b1e95a9707c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:46.593837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54047782-5b52-468f-8785-4b1e95a9707c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:46.594625 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54047782-5b52-468f-8785-4b1e95a9707c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:46.595358 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54047782-5b52-468f-8785-4b1e95a9707c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:46.596094 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54047782-5b52-468f-8785-4b1e95a9707c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:46.596552 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54047782-5b52-468f-8785-4b1e95a9707c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:46.597136 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54047782-5b52-468f-8785-4b1e95a9707c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:46.822041 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-10609e42-372a-41a4-9932-34ccd0cd3989 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:46.831024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-10609e42-372a-41a4-9932-34ccd0cd3989 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1364 microversion: 2.1 time: 0.303263 Aug 28 00:55:46.831597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 90/186] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:46 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1364 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:46.836352 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-d10eb149-4157-470e-b98b-cd2592a37e8e req-b5c81a85-c629-4d61-a751-f224940b5073 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "460a7822-a544-4b66-9361-79ac7d118804", "name": "network-vif-plugged", "status": "completed", "tag": "b01cc365-e4c3-4f15-b660-e7eb82f6fcf4"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:55:46.846350 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-d10eb149-4157-470e-b98b-cd2592a37e8e req-b5c81a85-c629-4d61-a751-f224940b5073 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:46.846831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-d10eb149-4157-470e-b98b-cd2592a37e8e req-b5c81a85-c629-4d61-a751-f224940b5073 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:46.847248 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-d10eb149-4157-470e-b98b-cd2592a37e8e req-b5c81a85-c629-4d61-a751-f224940b5073 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:46.858607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-54047782-5b52-468f-8785-4b1e95a9707c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:46.858783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-d10eb149-4157-470e-b98b-cd2592a37e8e req-b5c81a85-c629-4d61-a751-f224940b5073 service nova] Creating event network-vif-plugged:b01cc365-e4c3-4f15-b660-e7eb82f6fcf4 for instance 460a7822-a544-4b66-9361-79ac7d118804 on hv-cinder-maste Aug 28 00:55:46.863739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-d10eb149-4157-470e-b98b-cd2592a37e8e req-b5c81a85-c629-4d61-a751-f224940b5073 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030753 Aug 28 00:55:46.864665 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 91/187] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:55:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:55:46.868560 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-54047782-5b52-468f-8785-4b1e95a9707c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:46.869756 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-903ffcdf-55a8-441a-83a3-0beef437deff tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:46.870020 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-54047782-5b52-468f-8785-4b1e95a9707c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:46.870451 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-903ffcdf-55a8-441a-83a3-0beef437deff tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:46.871658 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-54047782-5b52-468f-8785-4b1e95a9707c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.291646 Aug 28 00:55:46.872461 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 97/188] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:46 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:55:46.884776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-903ffcdf-55a8-441a-83a3-0beef437deff tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:46.885538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-903ffcdf-55a8-441a-83a3-0beef437deff tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:46.886474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-903ffcdf-55a8-441a-83a3-0beef437deff tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:46.887182 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-903ffcdf-55a8-441a-83a3-0beef437deff tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:46.887630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-903ffcdf-55a8-441a-83a3-0beef437deff tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:46.888045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-903ffcdf-55a8-441a-83a3-0beef437deff tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:46.956815 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-22bb17ff-a09a-43ef-94cf-abb2deefcdce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:46.957447 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-22bb17ff-a09a-43ef-94cf-abb2deefcdce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:46.966322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-22bb17ff-a09a-43ef-94cf-abb2deefcdce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:46.966805 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-22bb17ff-a09a-43ef-94cf-abb2deefcdce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:46.967225 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-22bb17ff-a09a-43ef-94cf-abb2deefcdce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:46.967911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-22bb17ff-a09a-43ef-94cf-abb2deefcdce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:46.968392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-22bb17ff-a09a-43ef-94cf-abb2deefcdce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:46.968796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-22bb17ff-a09a-43ef-94cf-abb2deefcdce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:47.057400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-903ffcdf-55a8-441a-83a3-0beef437deff tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:47.065582 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-903ffcdf-55a8-441a-83a3-0beef437deff tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.199340 Aug 28 00:55:47.066392 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 92/189] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:46 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:47.127997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-22bb17ff-a09a-43ef-94cf-abb2deefcdce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:47.136576 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a97175d2-63f4-4889-b59f-19509efbda13 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:47.137351 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a97175d2-63f4-4889-b59f-19509efbda13 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:47.139894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-22bb17ff-a09a-43ef-94cf-abb2deefcdce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.187849 Aug 28 00:55:47.141321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 98/190] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:46 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:55:47.145498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a97175d2-63f4-4889-b59f-19509efbda13 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:47.145985 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a97175d2-63f4-4889-b59f-19509efbda13 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:47.146470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a97175d2-63f4-4889-b59f-19509efbda13 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:47.147112 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a97175d2-63f4-4889-b59f-19509efbda13 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:47.147554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a97175d2-63f4-4889-b59f-19509efbda13 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:47.148144 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a97175d2-63f4-4889-b59f-19509efbda13 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:47.281186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a97175d2-63f4-4889-b59f-19509efbda13 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:47.289257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a97175d2-63f4-4889-b59f-19509efbda13 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.160643 Aug 28 00:55:47.290072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 93/191] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:47 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:47.852025 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-47acb11c-f38b-4559-b5cd-b8f547a8e239 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:47.852632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-47acb11c-f38b-4559-b5cd-b8f547a8e239 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:47.858964 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-47acb11c-f38b-4559-b5cd-b8f547a8e239 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:47.859478 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-47acb11c-f38b-4559-b5cd-b8f547a8e239 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:47.860188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-47acb11c-f38b-4559-b5cd-b8f547a8e239 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:47.861126 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-47acb11c-f38b-4559-b5cd-b8f547a8e239 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:47.861572 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-47acb11c-f38b-4559-b5cd-b8f547a8e239 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:47.861981 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-47acb11c-f38b-4559-b5cd-b8f547a8e239 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:47.897498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-748d7311-ec64-43a4-8b07-0588a4d87de9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:47.898436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-748d7311-ec64-43a4-8b07-0588a4d87de9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:47.909040 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-748d7311-ec64-43a4-8b07-0588a4d87de9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:47.909781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-748d7311-ec64-43a4-8b07-0588a4d87de9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:47.910464 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-748d7311-ec64-43a4-8b07-0588a4d87de9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:47.913739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-748d7311-ec64-43a4-8b07-0588a4d87de9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:47.914274 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-748d7311-ec64-43a4-8b07-0588a4d87de9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:47.914899 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-748d7311-ec64-43a4-8b07-0588a4d87de9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:48.105935 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-47acb11c-f38b-4559-b5cd-b8f547a8e239 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:48.118328 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-47acb11c-f38b-4559-b5cd-b8f547a8e239 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1364 microversion: 2.1 time: 0.270003 Aug 28 00:55:48.127225 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 99/192] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:47 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1364 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:48.127225 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2404b7ea-1da2-429e-89e9-4ce0e06fcb76 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:48.127225 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2404b7ea-1da2-429e-89e9-4ce0e06fcb76 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:48.134537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2404b7ea-1da2-429e-89e9-4ce0e06fcb76 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:48.135104 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2404b7ea-1da2-429e-89e9-4ce0e06fcb76 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:48.135574 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2404b7ea-1da2-429e-89e9-4ce0e06fcb76 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:48.136341 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2404b7ea-1da2-429e-89e9-4ce0e06fcb76 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:48.136925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2404b7ea-1da2-429e-89e9-4ce0e06fcb76 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:48.137226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2404b7ea-1da2-429e-89e9-4ce0e06fcb76 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:48.150660 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-748d7311-ec64-43a4-8b07-0588a4d87de9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:48.164221 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-748d7311-ec64-43a4-8b07-0588a4d87de9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:48.165743 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-748d7311-ec64-43a4-8b07-0588a4d87de9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:48.166807 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-748d7311-ec64-43a4-8b07-0588a4d87de9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.274900 Aug 28 00:55:48.181372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 94/193] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:47 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:55:48.188464 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9e95e4b7-d93e-411a-af94-36a8dfeb2729 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:48.189109 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9e95e4b7-d93e-411a-af94-36a8dfeb2729 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:48.195339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9e95e4b7-d93e-411a-af94-36a8dfeb2729 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:48.195816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9e95e4b7-d93e-411a-af94-36a8dfeb2729 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:48.196239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9e95e4b7-d93e-411a-af94-36a8dfeb2729 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:48.196939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9e95e4b7-d93e-411a-af94-36a8dfeb2729 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:48.197404 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9e95e4b7-d93e-411a-af94-36a8dfeb2729 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:48.197819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9e95e4b7-d93e-411a-af94-36a8dfeb2729 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:48.416104 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2404b7ea-1da2-429e-89e9-4ce0e06fcb76 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:48.424202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2404b7ea-1da2-429e-89e9-4ce0e06fcb76 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.303186 Aug 28 00:55:48.424775 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 100/194] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:48 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:48.430054 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b275cdae-6fc0-42e0-8a7b-26db82852655 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:48.430693 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b275cdae-6fc0-42e0-8a7b-26db82852655 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:48.437275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b275cdae-6fc0-42e0-8a7b-26db82852655 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:48.437768 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b275cdae-6fc0-42e0-8a7b-26db82852655 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:48.438201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b275cdae-6fc0-42e0-8a7b-26db82852655 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:48.438889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b275cdae-6fc0-42e0-8a7b-26db82852655 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:48.439337 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b275cdae-6fc0-42e0-8a7b-26db82852655 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:48.439752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b275cdae-6fc0-42e0-8a7b-26db82852655 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:48.458663 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9e95e4b7-d93e-411a-af94-36a8dfeb2729 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:48.468263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9e95e4b7-d93e-411a-af94-36a8dfeb2729 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.285786 Aug 28 00:55:48.468991 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 95/195] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:48 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:48.578939 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b275cdae-6fc0-42e0-8a7b-26db82852655 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:48.586869 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b275cdae-6fc0-42e0-8a7b-26db82852655 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.161098 Aug 28 00:55:48.587443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 101/196] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:48 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:49.142458 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-dfcafb23-478f-410a-a477-fbf6153a3a67 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:49.143621 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-dfcafb23-478f-410a-a477-fbf6153a3a67 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:49.150594 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dfcafb23-478f-410a-a477-fbf6153a3a67 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:49.151289 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dfcafb23-478f-410a-a477-fbf6153a3a67 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:49.152633 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dfcafb23-478f-410a-a477-fbf6153a3a67 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:49.153445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dfcafb23-478f-410a-a477-fbf6153a3a67 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:49.154105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dfcafb23-478f-410a-a477-fbf6153a3a67 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:49.154677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dfcafb23-478f-410a-a477-fbf6153a3a67 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:49.195215 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-44b95086-7d56-4967-9c23-364857aac10b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:49.196029 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-44b95086-7d56-4967-9c23-364857aac10b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:49.205989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-44b95086-7d56-4967-9c23-364857aac10b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:49.206475 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-44b95086-7d56-4967-9c23-364857aac10b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:49.206898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-44b95086-7d56-4967-9c23-364857aac10b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:49.207590 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-44b95086-7d56-4967-9c23-364857aac10b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:49.209559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-44b95086-7d56-4967-9c23-364857aac10b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:49.210036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-44b95086-7d56-4967-9c23-364857aac10b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:49.346399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dfcafb23-478f-410a-a477-fbf6153a3a67 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:49.357695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-dfcafb23-478f-410a-a477-fbf6153a3a67 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1364 microversion: 2.1 time: 0.219430 Aug 28 00:55:49.359192 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 96/197] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:49 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1364 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:55:49.376540 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-44b95086-7d56-4967-9c23-364857aac10b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:49.385357 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-44b95086-7d56-4967-9c23-364857aac10b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:49.386873 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-44b95086-7d56-4967-9c23-364857aac10b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:49.388045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-44b95086-7d56-4967-9c23-364857aac10b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.197599 Aug 28 00:55:49.388645 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 102/198] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:49 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:55:49.443806 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-195e1c8f-9c0f-49f4-9f64-583a777b9f53 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:49.444851 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-195e1c8f-9c0f-49f4-9f64-583a777b9f53 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:49.452402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-195e1c8f-9c0f-49f4-9f64-583a777b9f53 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:49.454677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-195e1c8f-9c0f-49f4-9f64-583a777b9f53 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:49.455654 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-195e1c8f-9c0f-49f4-9f64-583a777b9f53 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:49.456553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-195e1c8f-9c0f-49f4-9f64-583a777b9f53 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:49.457343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-195e1c8f-9c0f-49f4-9f64-583a777b9f53 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:49.457951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-195e1c8f-9c0f-49f4-9f64-583a777b9f53 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:49.491012 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d111c4f4-922d-4bb8-a2ec-e1e57ad80dc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:49.492340 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d111c4f4-922d-4bb8-a2ec-e1e57ad80dc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:49.504512 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d111c4f4-922d-4bb8-a2ec-e1e57ad80dc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:49.505538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d111c4f4-922d-4bb8-a2ec-e1e57ad80dc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:49.506010 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d111c4f4-922d-4bb8-a2ec-e1e57ad80dc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:49.506953 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d111c4f4-922d-4bb8-a2ec-e1e57ad80dc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:49.507567 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d111c4f4-922d-4bb8-a2ec-e1e57ad80dc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:49.508235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d111c4f4-922d-4bb8-a2ec-e1e57ad80dc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:49.652046 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-195e1c8f-9c0f-49f4-9f64-583a777b9f53 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:49.661610 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-195e1c8f-9c0f-49f4-9f64-583a777b9f53 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1315 microversion: 2.1 time: 0.221331 Aug 28 00:55:49.662615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 97/199] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:49 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1315 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:55:49.669088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ae6ec236-c103-4226-98cd-2ef9a539f95a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:49.669770 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ae6ec236-c103-4226-98cd-2ef9a539f95a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:49.679488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ae6ec236-c103-4226-98cd-2ef9a539f95a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:49.680114 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ae6ec236-c103-4226-98cd-2ef9a539f95a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:49.680660 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ae6ec236-c103-4226-98cd-2ef9a539f95a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:49.681853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ae6ec236-c103-4226-98cd-2ef9a539f95a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:49.682035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ae6ec236-c103-4226-98cd-2ef9a539f95a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:49.682543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ae6ec236-c103-4226-98cd-2ef9a539f95a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:49.693196 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d111c4f4-922d-4bb8-a2ec-e1e57ad80dc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:49.701816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d111c4f4-922d-4bb8-a2ec-e1e57ad80dc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.216968 Aug 28 00:55:49.702912 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 103/200] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:49 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:50.057138 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ae6ec236-c103-4226-98cd-2ef9a539f95a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:50.069385 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ae6ec236-c103-4226-98cd-2ef9a539f95a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.405250 Aug 28 00:55:50.070504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 98/201] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:49 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:50.381145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0f1e285d-ff47-479f-8b4a-96c725c190e6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:50.381762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0f1e285d-ff47-479f-8b4a-96c725c190e6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:50.388768 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f1e285d-ff47-479f-8b4a-96c725c190e6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:50.389558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f1e285d-ff47-479f-8b4a-96c725c190e6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:50.390321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f1e285d-ff47-479f-8b4a-96c725c190e6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:50.390905 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f1e285d-ff47-479f-8b4a-96c725c190e6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:50.391660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f1e285d-ff47-479f-8b4a-96c725c190e6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:50.392641 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f1e285d-ff47-479f-8b4a-96c725c190e6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:50.413010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bb176ebb-5daf-4473-9c6d-f086fb92b09c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:50.414223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bb176ebb-5daf-4473-9c6d-f086fb92b09c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:50.421659 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bb176ebb-5daf-4473-9c6d-f086fb92b09c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:50.422181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bb176ebb-5daf-4473-9c6d-f086fb92b09c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:50.422607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bb176ebb-5daf-4473-9c6d-f086fb92b09c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:50.423365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bb176ebb-5daf-4473-9c6d-f086fb92b09c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:50.423815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bb176ebb-5daf-4473-9c6d-f086fb92b09c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:50.424209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bb176ebb-5daf-4473-9c6d-f086fb92b09c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:50.651071 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0f1e285d-ff47-479f-8b4a-96c725c190e6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:50.661432 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0f1e285d-ff47-479f-8b4a-96c725c190e6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1364 microversion: 2.1 time: 0.285584 Aug 28 00:55:50.663531 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 104/202] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:50 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1364 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:50.683960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bb176ebb-5daf-4473-9c6d-f086fb92b09c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:50.695957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ecc8ae67-2548-41ed-b62f-bfcc8eb0e7ae tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:50.699688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bb176ebb-5daf-4473-9c6d-f086fb92b09c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:50.700485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ecc8ae67-2548-41ed-b62f-bfcc8eb0e7ae tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:50.702296 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bb176ebb-5daf-4473-9c6d-f086fb92b09c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:50.706708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bb176ebb-5daf-4473-9c6d-f086fb92b09c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.298874 Aug 28 00:55:50.726581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 99/203] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:50 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:55:50.731727 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-171b4e08-0b7f-487d-983a-142d5f842b77 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:50.734179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ecc8ae67-2548-41ed-b62f-bfcc8eb0e7ae tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:50.734721 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ecc8ae67-2548-41ed-b62f-bfcc8eb0e7ae tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:50.735454 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ecc8ae67-2548-41ed-b62f-bfcc8eb0e7ae tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:50.736184 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ecc8ae67-2548-41ed-b62f-bfcc8eb0e7ae tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:50.736614 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ecc8ae67-2548-41ed-b62f-bfcc8eb0e7ae tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:50.737071 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ecc8ae67-2548-41ed-b62f-bfcc8eb0e7ae tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:50.737873 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-171b4e08-0b7f-487d-983a-142d5f842b77 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:50.745387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-171b4e08-0b7f-487d-983a-142d5f842b77 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:50.745992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-171b4e08-0b7f-487d-983a-142d5f842b77 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:50.746697 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-171b4e08-0b7f-487d-983a-142d5f842b77 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:50.747593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-171b4e08-0b7f-487d-983a-142d5f842b77 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:50.748178 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-171b4e08-0b7f-487d-983a-142d5f842b77 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:50.748492 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-171b4e08-0b7f-487d-983a-142d5f842b77 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:50.944049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ecc8ae67-2548-41ed-b62f-bfcc8eb0e7ae tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:50.948112 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-171b4e08-0b7f-487d-983a-142d5f842b77 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:50.953100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ecc8ae67-2548-41ed-b62f-bfcc8eb0e7ae tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1503 microversion: 2.1 time: 0.263471 Aug 28 00:55:50.953674 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 105/204] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:50 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1503 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:50.956559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-171b4e08-0b7f-487d-983a-142d5f842b77 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.228662 Aug 28 00:55:50.957160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 100/205] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:50 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:51.089813 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ba0d312f-40dd-4605-a530-d467a5013d93 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:51.090755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ba0d312f-40dd-4605-a530-d467a5013d93 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:51.097153 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba0d312f-40dd-4605-a530-d467a5013d93 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:51.097747 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba0d312f-40dd-4605-a530-d467a5013d93 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:51.098270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba0d312f-40dd-4605-a530-d467a5013d93 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:51.099048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba0d312f-40dd-4605-a530-d467a5013d93 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:51.099582 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba0d312f-40dd-4605-a530-d467a5013d93 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:51.100115 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba0d312f-40dd-4605-a530-d467a5013d93 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:51.104263 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:51.262740 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ba0d312f-40dd-4605-a530-d467a5013d93 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:51.271553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ba0d312f-40dd-4605-a530-d467a5013d93 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.185405 Aug 28 00:55:51.272459 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 106/206] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:51 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:51.424246 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-39912471-003b-4f3f-9c82-32f71e034692 req-459c9c8e-1d2d-4650-9d2f-8c75770aa877 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dd61123b-7a96-4c9e-936c-5794a4fdd7df", "name": "network-vif-plugged", "status": "completed", "tag": "522d4a1b-32bc-4106-afb0-5995aecc53c0"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:55:51.433976 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-39912471-003b-4f3f-9c82-32f71e034692 req-459c9c8e-1d2d-4650-9d2f-8c75770aa877 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:51.434453 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-39912471-003b-4f3f-9c82-32f71e034692 req-459c9c8e-1d2d-4650-9d2f-8c75770aa877 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:51.434922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-39912471-003b-4f3f-9c82-32f71e034692 req-459c9c8e-1d2d-4650-9d2f-8c75770aa877 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:51.446334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-39912471-003b-4f3f-9c82-32f71e034692 req-459c9c8e-1d2d-4650-9d2f-8c75770aa877 service nova] Creating event network-vif-plugged:522d4a1b-32bc-4106-afb0-5995aecc53c0 for instance dd61123b-7a96-4c9e-936c-5794a4fdd7df on hv-cinder-maste Aug 28 00:55:51.451727 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-39912471-003b-4f3f-9c82-32f71e034692 req-459c9c8e-1d2d-4650-9d2f-8c75770aa877 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031327 Aug 28 00:55:51.452320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 101/207] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:55:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:55:51.685163 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1a0490d8-511e-420f-9863-6d817a529b18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:51.685710 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1a0490d8-511e-420f-9863-6d817a529b18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:51.691895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1a0490d8-511e-420f-9863-6d817a529b18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:51.692378 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1a0490d8-511e-420f-9863-6d817a529b18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:51.692808 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1a0490d8-511e-420f-9863-6d817a529b18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:51.693515 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1a0490d8-511e-420f-9863-6d817a529b18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:51.693970 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1a0490d8-511e-420f-9863-6d817a529b18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:51.694550 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1a0490d8-511e-420f-9863-6d817a529b18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:51.733044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a57e7109-630b-4415-8da1-cc80c89cf622 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:51.734018 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a57e7109-630b-4415-8da1-cc80c89cf622 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:51.741374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a57e7109-630b-4415-8da1-cc80c89cf622 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:51.742098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a57e7109-630b-4415-8da1-cc80c89cf622 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:51.742726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a57e7109-630b-4415-8da1-cc80c89cf622 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:51.743540 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a57e7109-630b-4415-8da1-cc80c89cf622 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:51.744084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a57e7109-630b-4415-8da1-cc80c89cf622 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:51.744574 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a57e7109-630b-4415-8da1-cc80c89cf622 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:51.980291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1a0490d8-511e-420f-9863-6d817a529b18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:51.991883 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1a0490d8-511e-420f-9863-6d817a529b18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1364 microversion: 2.1 time: 0.311625 Aug 28 00:55:51.999484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 107/208] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:51 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1364 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:52.004825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-60606c24-f6a7-4664-bc94-dd96a8741802 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:52.005645 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-60606c24-f6a7-4664-bc94-dd96a8741802 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:52.017309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-60606c24-f6a7-4664-bc94-dd96a8741802 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:52.017796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-60606c24-f6a7-4664-bc94-dd96a8741802 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:52.018226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-60606c24-f6a7-4664-bc94-dd96a8741802 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:52.018924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-60606c24-f6a7-4664-bc94-dd96a8741802 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:52.019369 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-60606c24-f6a7-4664-bc94-dd96a8741802 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:52.019781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-60606c24-f6a7-4664-bc94-dd96a8741802 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:52.062477 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a57e7109-630b-4415-8da1-cc80c89cf622 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:52.075310 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a57e7109-630b-4415-8da1-cc80c89cf622 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:52.078285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a57e7109-630b-4415-8da1-cc80c89cf622 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:52.082742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a57e7109-630b-4415-8da1-cc80c89cf622 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.353041 Aug 28 00:55:52.097492 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 102/209] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:51 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:55:52.215786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-60606c24-f6a7-4664-bc94-dd96a8741802 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:52.225098 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-60606c24-f6a7-4664-bc94-dd96a8741802 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1368 microversion: 2.1 time: 0.224150 Aug 28 00:55:52.225890 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 108/210] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:51 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1368 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:52.296059 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3be69fdc-9a77-489b-bb17-a5aa302eb36f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:52.297907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3be69fdc-9a77-489b-bb17-a5aa302eb36f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:52.304614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3be69fdc-9a77-489b-bb17-a5aa302eb36f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:52.305875 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3be69fdc-9a77-489b-bb17-a5aa302eb36f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:52.306496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3be69fdc-9a77-489b-bb17-a5aa302eb36f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:52.307398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3be69fdc-9a77-489b-bb17-a5aa302eb36f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:52.308101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3be69fdc-9a77-489b-bb17-a5aa302eb36f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:52.308911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3be69fdc-9a77-489b-bb17-a5aa302eb36f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:52.395014 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5148912c-62f7-4eb5-aae7-e7ace06badee tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "be1d8c2a-787b-460f-81c3-d3e306de8c72"}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:55:52.397560 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5148912c-62f7-4eb5-aae7-e7ace06badee tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:52.405036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5148912c-62f7-4eb5-aae7-e7ace06badee tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:52.406024 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5148912c-62f7-4eb5-aae7-e7ace06badee tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:52.406668 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5148912c-62f7-4eb5-aae7-e7ace06badee tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:52.407656 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5148912c-62f7-4eb5-aae7-e7ace06badee tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:52.408548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5148912c-62f7-4eb5-aae7-e7ace06badee tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:52.409180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5148912c-62f7-4eb5-aae7-e7ace06badee tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:52.503228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3be69fdc-9a77-489b-bb17-a5aa302eb36f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:52.518098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3be69fdc-9a77-489b-bb17-a5aa302eb36f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.226226 Aug 28 00:55:52.518761 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 103/211] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:52 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:53.014124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ea077eb5-5398-4536-9c2f-34106f88d489 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:53.014829 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ea077eb5-5398-4536-9c2f-34106f88d489 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:53.021103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea077eb5-5398-4536-9c2f-34106f88d489 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:53.021580 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea077eb5-5398-4536-9c2f-34106f88d489 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:53.021987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea077eb5-5398-4536-9c2f-34106f88d489 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:53.022676 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea077eb5-5398-4536-9c2f-34106f88d489 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:53.023116 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea077eb5-5398-4536-9c2f-34106f88d489 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:53.023527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea077eb5-5398-4536-9c2f-34106f88d489 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:53.170372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ea077eb5-5398-4536-9c2f-34106f88d489 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:53.180889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ea077eb5-5398-4536-9c2f-34106f88d489 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1364 microversion: 2.1 time: 0.170438 Aug 28 00:55:53.182064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 104/212] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:53 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1364 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:53.187248 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e9e692b4-cbbd-47f3-bc6e-00e15e06b556 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:53.188086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e9e692b4-cbbd-47f3-bc6e-00e15e06b556 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:53.202035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e9e692b4-cbbd-47f3-bc6e-00e15e06b556 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:53.202971 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e9e692b4-cbbd-47f3-bc6e-00e15e06b556 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:53.203809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e9e692b4-cbbd-47f3-bc6e-00e15e06b556 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:53.226605 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e9e692b4-cbbd-47f3-bc6e-00e15e06b556 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:53.230885 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e9e692b4-cbbd-47f3-bc6e-00e15e06b556 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.025s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:53.232708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e9e692b4-cbbd-47f3-bc6e-00e15e06b556 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:53.436148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e9e692b4-cbbd-47f3-bc6e-00e15e06b556 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:53.449418 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e9e692b4-cbbd-47f3-bc6e-00e15e06b556 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:53.450870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e9e692b4-cbbd-47f3-bc6e-00e15e06b556 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:53.452043 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e9e692b4-cbbd-47f3-bc6e-00e15e06b556 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.268873 Aug 28 00:55:53.465417 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 105/213] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:53 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:55:53.472927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0c1b10ff-6b80-441c-b18d-b2ac07c005fc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:53.473852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0c1b10ff-6b80-441c-b18d-b2ac07c005fc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:53.480315 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c1b10ff-6b80-441c-b18d-b2ac07c005fc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:53.480978 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c1b10ff-6b80-441c-b18d-b2ac07c005fc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:53.481577 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c1b10ff-6b80-441c-b18d-b2ac07c005fc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:53.482438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c1b10ff-6b80-441c-b18d-b2ac07c005fc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:53.483023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c1b10ff-6b80-441c-b18d-b2ac07c005fc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:53.483582 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c1b10ff-6b80-441c-b18d-b2ac07c005fc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:53.666563 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0c1b10ff-6b80-441c-b18d-b2ac07c005fc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:53.672748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:53.679158 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0c1b10ff-6b80-441c-b18d-b2ac07c005fc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1559 microversion: 2.1 time: 0.212643 Aug 28 00:55:53.679745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 106/214] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:53 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1559 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:53.684176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6bc433b2-c7cd-41ab-808a-6cc5e461396c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:53.684932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6bc433b2-c7cd-41ab-808a-6cc5e461396c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:53.690931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bc433b2-c7cd-41ab-808a-6cc5e461396c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:53.691589 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bc433b2-c7cd-41ab-808a-6cc5e461396c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:53.692177 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bc433b2-c7cd-41ab-808a-6cc5e461396c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:53.692868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bc433b2-c7cd-41ab-808a-6cc5e461396c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:53.693510 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bc433b2-c7cd-41ab-808a-6cc5e461396c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:53.693955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bc433b2-c7cd-41ab-808a-6cc5e461396c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:53.841785 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6bc433b2-c7cd-41ab-808a-6cc5e461396c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:53.850840 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6bc433b2-c7cd-41ab-808a-6cc5e461396c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.170039 Aug 28 00:55:53.851359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 107/215] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:53 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:53.863917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d00a4893-36a7-43bf-a29f-4369edbdff2a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:53.864651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d00a4893-36a7-43bf-a29f-4369edbdff2a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:53.874440 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d00a4893-36a7-43bf-a29f-4369edbdff2a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:53.875954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d00a4893-36a7-43bf-a29f-4369edbdff2a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:53.876603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d00a4893-36a7-43bf-a29f-4369edbdff2a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:53.877460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d00a4893-36a7-43bf-a29f-4369edbdff2a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:53.878076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d00a4893-36a7-43bf-a29f-4369edbdff2a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:53.878512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d00a4893-36a7-43bf-a29f-4369edbdff2a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:54.001208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d00a4893-36a7-43bf-a29f-4369edbdff2a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.148568 Aug 28 00:55:54.001924 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 108/216] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Aug 28 00:55:53 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-interface => generated 285 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:55:54.207922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-87a061c4-bd50-448a-b32b-97e5ebb8e8ea tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:54.208868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-87a061c4-bd50-448a-b32b-97e5ebb8e8ea tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:54.219699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87a061c4-bd50-448a-b32b-97e5ebb8e8ea tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:54.220328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87a061c4-bd50-448a-b32b-97e5ebb8e8ea tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:54.221016 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87a061c4-bd50-448a-b32b-97e5ebb8e8ea tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:54.221964 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87a061c4-bd50-448a-b32b-97e5ebb8e8ea tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:54.222404 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87a061c4-bd50-448a-b32b-97e5ebb8e8ea tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:54.222995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87a061c4-bd50-448a-b32b-97e5ebb8e8ea tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:54.539532 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-87a061c4-bd50-448a-b32b-97e5ebb8e8ea tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:54.549757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-87a061c4-bd50-448a-b32b-97e5ebb8e8ea tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1364 microversion: 2.1 time: 0.345751 Aug 28 00:55:54.550707 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 109/217] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:54 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1364 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:54.569642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f6af5af6-f854-49ad-99b1-75a7b39fd082 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:54.570563 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f6af5af6-f854-49ad-99b1-75a7b39fd082 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:54.577245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6af5af6-f854-49ad-99b1-75a7b39fd082 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:54.577928 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6af5af6-f854-49ad-99b1-75a7b39fd082 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:54.578504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6af5af6-f854-49ad-99b1-75a7b39fd082 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:54.579065 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6af5af6-f854-49ad-99b1-75a7b39fd082 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:54.579668 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6af5af6-f854-49ad-99b1-75a7b39fd082 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:54.580163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6af5af6-f854-49ad-99b1-75a7b39fd082 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:55.421910 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f6af5af6-f854-49ad-99b1-75a7b39fd082 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:55.432346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f6af5af6-f854-49ad-99b1-75a7b39fd082 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:55.435087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f6af5af6-f854-49ad-99b1-75a7b39fd082 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:55.436665 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f6af5af6-f854-49ad-99b1-75a7b39fd082 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.873048 Aug 28 00:55:55.437795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 110/218] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:54 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 875 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:55:55.444639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-63a1732e-557a-4f72-95bb-3e19afb950e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:55.446429 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-63a1732e-557a-4f72-95bb-3e19afb950e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:55.453778 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63a1732e-557a-4f72-95bb-3e19afb950e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:55.454265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63a1732e-557a-4f72-95bb-3e19afb950e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:55.454917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63a1732e-557a-4f72-95bb-3e19afb950e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:55.455789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63a1732e-557a-4f72-95bb-3e19afb950e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:55.456266 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63a1732e-557a-4f72-95bb-3e19afb950e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:55.456851 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63a1732e-557a-4f72-95bb-3e19afb950e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:55.776176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-63a1732e-557a-4f72-95bb-3e19afb950e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:55.784420 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-63a1732e-557a-4f72-95bb-3e19afb950e7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.345197 Aug 28 00:55:55.799298 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 111/219] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:55 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:55.804321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a9400abe-b410-46ef-acce-38aafabde375 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:55.805502 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a9400abe-b410-46ef-acce-38aafabde375 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:55.812446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9400abe-b410-46ef-acce-38aafabde375 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:55.812923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9400abe-b410-46ef-acce-38aafabde375 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:55.813399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9400abe-b410-46ef-acce-38aafabde375 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:55.814062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9400abe-b410-46ef-acce-38aafabde375 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:55.814503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9400abe-b410-46ef-acce-38aafabde375 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:55.814912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9400abe-b410-46ef-acce-38aafabde375 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:56.064911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a9400abe-b410-46ef-acce-38aafabde375 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:56.085305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a9400abe-b410-46ef-acce-38aafabde375 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1364 microversion: 2.1 time: 0.281969 Aug 28 00:55:56.087082 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 112/220] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:55 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1364 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:56.264010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-ec664464-1dac-4743-a242-515cda4ccd64 req-fb863ca9-aa55-4253-b568-831cf892c122 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dff36aeb-f0c7-41f0-8ecc-9924adf8289c", "tag": "072b8279-6084-43c1-8f0e-c1a705ec9b99"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:55:56.280537 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-ec664464-1dac-4743-a242-515cda4ccd64 req-fb863ca9-aa55-4253-b568-831cf892c122 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:56.281052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-ec664464-1dac-4743-a242-515cda4ccd64 req-fb863ca9-aa55-4253-b568-831cf892c122 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:56.284547 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-ec664464-1dac-4743-a242-515cda4ccd64 req-fb863ca9-aa55-4253-b568-831cf892c122 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:56.300137 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-ec664464-1dac-4743-a242-515cda4ccd64 req-fb863ca9-aa55-4253-b568-831cf892c122 service nova] Creating event network-changed:072b8279-6084-43c1-8f0e-c1a705ec9b99 for instance dff36aeb-f0c7-41f0-8ecc-9924adf8289c on hv-cinder-maste Aug 28 00:55:56.307262 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-ec664464-1dac-4743-a242-515cda4ccd64 req-fb863ca9-aa55-4253-b568-831cf892c122 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047047 Aug 28 00:55:56.308791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 113/221] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:55:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:55:56.461441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6c7d43cd-69c7-4272-a0e4-564867451f99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:56.463263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6c7d43cd-69c7-4272-a0e4-564867451f99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:56.471967 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c7d43cd-69c7-4272-a0e4-564867451f99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:56.472812 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c7d43cd-69c7-4272-a0e4-564867451f99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:56.473599 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c7d43cd-69c7-4272-a0e4-564867451f99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:56.474470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c7d43cd-69c7-4272-a0e4-564867451f99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:56.475419 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c7d43cd-69c7-4272-a0e4-564867451f99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:56.476381 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c7d43cd-69c7-4272-a0e4-564867451f99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:56.700939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6c7d43cd-69c7-4272-a0e4-564867451f99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:56.728765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6c7d43cd-69c7-4272-a0e4-564867451f99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:56.738468 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6c7d43cd-69c7-4272-a0e4-564867451f99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:56.741164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6c7d43cd-69c7-4272-a0e4-564867451f99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.283156 Aug 28 00:55:56.742484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 114/222] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:56 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:55:56.807489 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0693fb54-3691-4065-97fa-998f77f407aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:56.809949 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0693fb54-3691-4065-97fa-998f77f407aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:56.818502 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0693fb54-3691-4065-97fa-998f77f407aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:56.819603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0693fb54-3691-4065-97fa-998f77f407aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:56.820435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0693fb54-3691-4065-97fa-998f77f407aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:56.821260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0693fb54-3691-4065-97fa-998f77f407aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:56.821783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0693fb54-3691-4065-97fa-998f77f407aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:56.822426 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0693fb54-3691-4065-97fa-998f77f407aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:56.988360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0693fb54-3691-4065-97fa-998f77f407aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:57.001608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0693fb54-3691-4065-97fa-998f77f407aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1322 microversion: 2.1 time: 0.199686 Aug 28 00:55:57.002620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 115/223] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:56 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1322 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:55:57.110760 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-21263cbd-dbfe-45c8-83c6-0cd551a91640 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:57.111373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-21263cbd-dbfe-45c8-83c6-0cd551a91640 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:57.119159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-21263cbd-dbfe-45c8-83c6-0cd551a91640 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:57.121301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-21263cbd-dbfe-45c8-83c6-0cd551a91640 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:57.122037 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-21263cbd-dbfe-45c8-83c6-0cd551a91640 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:57.123191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-21263cbd-dbfe-45c8-83c6-0cd551a91640 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:57.123918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-21263cbd-dbfe-45c8-83c6-0cd551a91640 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:57.124680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-21263cbd-dbfe-45c8-83c6-0cd551a91640 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:57.323237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-21263cbd-dbfe-45c8-83c6-0cd551a91640 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:57.332685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-21263cbd-dbfe-45c8-83c6-0cd551a91640 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1364 microversion: 2.1 time: 0.225847 Aug 28 00:55:57.335121 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 116/224] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:57 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1364 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:57.699405 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5148912c-62f7-4eb5-aae7-e7ace06badee tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "POST /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 5.307910 Aug 28 00:55:57.707228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 109/225] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Aug 28 00:55:52 2022] POST /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd/os-volume_attachments => generated 194 bytes in 5317 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:55:57.765300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-03006e1b-c643-4105-bb09-0a0dd5fe4d8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:57.766932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-03006e1b-c643-4105-bb09-0a0dd5fe4d8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:57.773873 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03006e1b-c643-4105-bb09-0a0dd5fe4d8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:57.774613 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03006e1b-c643-4105-bb09-0a0dd5fe4d8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:57.775209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03006e1b-c643-4105-bb09-0a0dd5fe4d8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:57.776005 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03006e1b-c643-4105-bb09-0a0dd5fe4d8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:57.776348 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03006e1b-c643-4105-bb09-0a0dd5fe4d8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:57.788771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03006e1b-c643-4105-bb09-0a0dd5fe4d8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:57.973099 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-03006e1b-c643-4105-bb09-0a0dd5fe4d8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:57.982347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-03006e1b-c643-4105-bb09-0a0dd5fe4d8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:57.984062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-03006e1b-c643-4105-bb09-0a0dd5fe4d8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:57.985335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-03006e1b-c643-4105-bb09-0a0dd5fe4d8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1405 microversion: 2.20 time: 0.224736 Aug 28 00:55:57.986435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 117/226] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:57 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1405 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:55:58.025768 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5098e561-db4a-4bd3-8a69-14939c7d8157 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:58.033535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5098e561-db4a-4bd3-8a69-14939c7d8157 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:58.034342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5098e561-db4a-4bd3-8a69-14939c7d8157 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:58.034945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5098e561-db4a-4bd3-8a69-14939c7d8157 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:58.035440 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5098e561-db4a-4bd3-8a69-14939c7d8157 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:58.036210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5098e561-db4a-4bd3-8a69-14939c7d8157 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:58.037595 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5098e561-db4a-4bd3-8a69-14939c7d8157 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:58.038422 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5098e561-db4a-4bd3-8a69-14939c7d8157 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:58.228029 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5098e561-db4a-4bd3-8a69-14939c7d8157 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:58.237521 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5098e561-db4a-4bd3-8a69-14939c7d8157 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1517 microversion: 2.1 time: 0.217738 Aug 28 00:55:58.238554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 110/227] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:58 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1517 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:58.260727 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-372f090a-a44b-45c1-bc14-71932ba7ec96 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:55:58.263167 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-372f090a-a44b-45c1-bc14-71932ba7ec96 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:58.271381 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-372f090a-a44b-45c1-bc14-71932ba7ec96 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:58.272112 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-372f090a-a44b-45c1-bc14-71932ba7ec96 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:58.272708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-372f090a-a44b-45c1-bc14-71932ba7ec96 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:58.273647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-372f090a-a44b-45c1-bc14-71932ba7ec96 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:58.274266 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-372f090a-a44b-45c1-bc14-71932ba7ec96 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:58.274846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-372f090a-a44b-45c1-bc14-71932ba7ec96 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:58.303755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.objects.instance [None req-372f090a-a44b-45c1-bc14-71932ba7ec96 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lazy-loading 'flavor' on Instance uuid 460a7822-a544-4b66-9361-79ac7d118804 {{(pid=120525) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 00:55:58.358139 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6467db2c-1e8c-4c89-bdf7-323adaa06ee5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:58.367223 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6467db2c-1e8c-4c89-bdf7-323adaa06ee5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:58.382300 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6467db2c-1e8c-4c89-bdf7-323adaa06ee5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:58.383447 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6467db2c-1e8c-4c89-bdf7-323adaa06ee5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:58.384524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6467db2c-1e8c-4c89-bdf7-323adaa06ee5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:58.387951 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6467db2c-1e8c-4c89-bdf7-323adaa06ee5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:58.389826 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6467db2c-1e8c-4c89-bdf7-323adaa06ee5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:58.390784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6467db2c-1e8c-4c89-bdf7-323adaa06ee5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:58.473865 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-372f090a-a44b-45c1-bc14-71932ba7ec96 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "POST /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804/action" status: 200 len: 32 microversion: 2.1 time: 0.218122 Aug 28 00:55:58.478966 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 118/228] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Aug 28 00:55:58 2022] POST /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804/action => generated 32 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 28 00:55:58.486384 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2d9c5cf1-f028-47d7-a787-8001511e5705 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:58.487427 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2d9c5cf1-f028-47d7-a787-8001511e5705 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:58.494105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d9c5cf1-f028-47d7-a787-8001511e5705 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:58.494762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d9c5cf1-f028-47d7-a787-8001511e5705 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:58.495379 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d9c5cf1-f028-47d7-a787-8001511e5705 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:58.496513 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d9c5cf1-f028-47d7-a787-8001511e5705 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:58.497145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d9c5cf1-f028-47d7-a787-8001511e5705 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:58.497817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d9c5cf1-f028-47d7-a787-8001511e5705 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:58.649998 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6467db2c-1e8c-4c89-bdf7-323adaa06ee5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:58.663547 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6467db2c-1e8c-4c89-bdf7-323adaa06ee5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1364 microversion: 2.1 time: 0.309258 Aug 28 00:55:58.665808 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 111/229] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:58 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1364 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:58.772893 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2d9c5cf1-f028-47d7-a787-8001511e5705 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:58.785360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2d9c5cf1-f028-47d7-a787-8001511e5705 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.303552 Aug 28 00:55:58.786237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 119/230] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:58 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:55:59.008860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e7b39afa-0743-410c-95db-073790b255bb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:59.010301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e7b39afa-0743-410c-95db-073790b255bb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:59.030161 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e7b39afa-0743-410c-95db-073790b255bb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:59.030745 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e7b39afa-0743-410c-95db-073790b255bb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:59.031409 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e7b39afa-0743-410c-95db-073790b255bb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:59.033918 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e7b39afa-0743-410c-95db-073790b255bb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:59.036198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e7b39afa-0743-410c-95db-073790b255bb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:59.036644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e7b39afa-0743-410c-95db-073790b255bb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:59.460224 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e7b39afa-0743-410c-95db-073790b255bb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:59.471307 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e7b39afa-0743-410c-95db-073790b255bb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:59.472892 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e7b39afa-0743-410c-95db-073790b255bb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:59.474167 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e7b39afa-0743-410c-95db-073790b255bb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1597 microversion: 2.20 time: 0.470425 Aug 28 00:55:59.474730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 112/231] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:55:59 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1597 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:55:59.498356 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fde5e420-a767-4877-b921-3cbf3ae3bc67 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:59.499353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fde5e420-a767-4877-b921-3cbf3ae3bc67 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:59.506232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fde5e420-a767-4877-b921-3cbf3ae3bc67 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:59.506941 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fde5e420-a767-4877-b921-3cbf3ae3bc67 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:59.507596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fde5e420-a767-4877-b921-3cbf3ae3bc67 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:59.508690 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fde5e420-a767-4877-b921-3cbf3ae3bc67 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:59.509442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fde5e420-a767-4877-b921-3cbf3ae3bc67 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:59.510009 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fde5e420-a767-4877-b921-3cbf3ae3bc67 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:59.643173 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fde5e420-a767-4877-b921-3cbf3ae3bc67 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.149235 Aug 28 00:55:59.645046 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 120/232] 10.222.0.42 () {60 vars in 1316 bytes} [Sun Aug 28 00:55:59 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-interface => generated 284 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 28 00:55:59.695292 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a572a13c-5858-4b33-b76e-b4d483d5fd66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:59.697178 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a572a13c-5858-4b33-b76e-b4d483d5fd66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:59.706306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a572a13c-5858-4b33-b76e-b4d483d5fd66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:59.714718 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a572a13c-5858-4b33-b76e-b4d483d5fd66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:59.715463 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a572a13c-5858-4b33-b76e-b4d483d5fd66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:59.716383 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a572a13c-5858-4b33-b76e-b4d483d5fd66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:59.716879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a572a13c-5858-4b33-b76e-b4d483d5fd66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:59.717347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a572a13c-5858-4b33-b76e-b4d483d5fd66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:59.813715 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b2e4f694-81d1-4ebb-92f6-38a10cd95eed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:55:59.814915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b2e4f694-81d1-4ebb-92f6-38a10cd95eed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:55:59.817548 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:55:59.825169 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2e4f694-81d1-4ebb-92f6-38a10cd95eed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:59.825940 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2e4f694-81d1-4ebb-92f6-38a10cd95eed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:59.826863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2e4f694-81d1-4ebb-92f6-38a10cd95eed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:59.842107 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2e4f694-81d1-4ebb-92f6-38a10cd95eed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:55:59.844423 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2e4f694-81d1-4ebb-92f6-38a10cd95eed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:55:59.845980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2e4f694-81d1-4ebb-92f6-38a10cd95eed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:55:59.904522 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a572a13c-5858-4b33-b76e-b4d483d5fd66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:55:59.928952 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a572a13c-5858-4b33-b76e-b4d483d5fd66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1364 microversion: 2.1 time: 0.234740 Aug 28 00:55:59.933424 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 113/233] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:59 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1364 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:00.144531 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b2e4f694-81d1-4ebb-92f6-38a10cd95eed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:00.157182 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b2e4f694-81d1-4ebb-92f6-38a10cd95eed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.347304 Aug 28 00:56:00.158778 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 121/234] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:55:59 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:00.952076 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-482cc0f1-1bbf-4f8e-9fa2-5bf3290a8bf7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:00.953582 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-482cc0f1-1bbf-4f8e-9fa2-5bf3290a8bf7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:00.959965 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-482cc0f1-1bbf-4f8e-9fa2-5bf3290a8bf7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:00.960449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-482cc0f1-1bbf-4f8e-9fa2-5bf3290a8bf7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:00.960876 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-482cc0f1-1bbf-4f8e-9fa2-5bf3290a8bf7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:00.961602 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-482cc0f1-1bbf-4f8e-9fa2-5bf3290a8bf7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:00.962049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-482cc0f1-1bbf-4f8e-9fa2-5bf3290a8bf7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:00.962463 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-482cc0f1-1bbf-4f8e-9fa2-5bf3290a8bf7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:01.181750 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8e78a8c7-0a1e-4229-8447-332d424a5fd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:01.182370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8e78a8c7-0a1e-4229-8447-332d424a5fd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:01.204323 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e78a8c7-0a1e-4229-8447-332d424a5fd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:01.205770 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e78a8c7-0a1e-4229-8447-332d424a5fd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:01.206929 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e78a8c7-0a1e-4229-8447-332d424a5fd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:01.208010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e78a8c7-0a1e-4229-8447-332d424a5fd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:01.210397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e78a8c7-0a1e-4229-8447-332d424a5fd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:01.211751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e78a8c7-0a1e-4229-8447-332d424a5fd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:01.651504 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-482cc0f1-1bbf-4f8e-9fa2-5bf3290a8bf7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:01.667672 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-482cc0f1-1bbf-4f8e-9fa2-5bf3290a8bf7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1364 microversion: 2.1 time: 0.720448 Aug 28 00:56:01.692211 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 114/235] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:00 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1364 bytes in 746 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:01.866139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8e78a8c7-0a1e-4229-8447-332d424a5fd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:01.870593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:56:01.884279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8e78a8c7-0a1e-4229-8447-332d424a5fd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.707482 Aug 28 00:56:01.891509 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 122/236] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:01 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 717 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:56:02.102778 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-a479f32f-97fb-425d-881f-fb77b5c25aca req-3313de96-3bd5-419d-b7cf-46a2e6d27c76 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e434e4f9-9d0e-46f2-bea3-395bae33fc92", "name": "network-vif-plugged", "status": "completed", "tag": "cb80f35b-738d-4eef-8768-99836091e5a6"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:56:02.164704 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-a479f32f-97fb-425d-881f-fb77b5c25aca req-3313de96-3bd5-419d-b7cf-46a2e6d27c76 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:02.166404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-a479f32f-97fb-425d-881f-fb77b5c25aca req-3313de96-3bd5-419d-b7cf-46a2e6d27c76 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:02.167490 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-a479f32f-97fb-425d-881f-fb77b5c25aca req-3313de96-3bd5-419d-b7cf-46a2e6d27c76 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:02.194305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-a479f32f-97fb-425d-881f-fb77b5c25aca req-3313de96-3bd5-419d-b7cf-46a2e6d27c76 service nova] Creating event network-vif-plugged:cb80f35b-738d-4eef-8768-99836091e5a6 for instance e434e4f9-9d0e-46f2-bea3-395bae33fc92 on hv-cinder-maste Aug 28 00:56:02.201431 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-a479f32f-97fb-425d-881f-fb77b5c25aca req-3313de96-3bd5-419d-b7cf-46a2e6d27c76 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.103779 Aug 28 00:56:02.202741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 115/237] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:56:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:56:02.440869 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-0e126ca3-1881-4ded-be24-3a9d5d220606 req-5b98d4a1-9b88-4fc1-8441-7a843f6f2ad7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dd61123b-7a96-4c9e-936c-5794a4fdd7df", "tag": "522d4a1b-32bc-4106-afb0-5995aecc53c0"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:56:02.466516 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-0e126ca3-1881-4ded-be24-3a9d5d220606 req-5b98d4a1-9b88-4fc1-8441-7a843f6f2ad7 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:02.467236 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-0e126ca3-1881-4ded-be24-3a9d5d220606 req-5b98d4a1-9b88-4fc1-8441-7a843f6f2ad7 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:02.467812 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-0e126ca3-1881-4ded-be24-3a9d5d220606 req-5b98d4a1-9b88-4fc1-8441-7a843f6f2ad7 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:02.482329 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-0e126ca3-1881-4ded-be24-3a9d5d220606 req-5b98d4a1-9b88-4fc1-8441-7a843f6f2ad7 service nova] Creating event network-changed:522d4a1b-32bc-4106-afb0-5995aecc53c0 for instance dd61123b-7a96-4c9e-936c-5794a4fdd7df on hv-cinder-maste Aug 28 00:56:02.488117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-0e126ca3-1881-4ded-be24-3a9d5d220606 req-5b98d4a1-9b88-4fc1-8441-7a843f6f2ad7 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.051241 Aug 28 00:56:02.492780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 123/238] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:56:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:56:02.698105 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0bb92cd2-70e2-488f-a018-f802265a41de tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:02.699387 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0bb92cd2-70e2-488f-a018-f802265a41de tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:02.706098 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0bb92cd2-70e2-488f-a018-f802265a41de tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:02.706861 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0bb92cd2-70e2-488f-a018-f802265a41de tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:02.707465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0bb92cd2-70e2-488f-a018-f802265a41de tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:02.708332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0bb92cd2-70e2-488f-a018-f802265a41de tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:02.708922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0bb92cd2-70e2-488f-a018-f802265a41de tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:02.709593 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0bb92cd2-70e2-488f-a018-f802265a41de tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:02.913778 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fe90c44c-5431-4641-aa6e-c56cb05a4543 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:02.914762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fe90c44c-5431-4641-aa6e-c56cb05a4543 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:02.922183 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe90c44c-5431-4641-aa6e-c56cb05a4543 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:02.923568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe90c44c-5431-4641-aa6e-c56cb05a4543 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:02.923981 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe90c44c-5431-4641-aa6e-c56cb05a4543 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:02.924686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe90c44c-5431-4641-aa6e-c56cb05a4543 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:02.925310 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe90c44c-5431-4641-aa6e-c56cb05a4543 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:02.925926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe90c44c-5431-4641-aa6e-c56cb05a4543 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:02.948077 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0bb92cd2-70e2-488f-a018-f802265a41de tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:02.961310 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0bb92cd2-70e2-488f-a018-f802265a41de tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1364 microversion: 2.1 time: 0.267187 Aug 28 00:56:02.963131 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 116/239] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:02 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1364 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:03.102407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fe90c44c-5431-4641-aa6e-c56cb05a4543 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:03.113558 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fe90c44c-5431-4641-aa6e-c56cb05a4543 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.204884 Aug 28 00:56:03.114465 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 124/240] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:02 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:56:03.984060 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-48f4a228-0df1-402a-9a79-b78751f35115 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:03.984692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-48f4a228-0df1-402a-9a79-b78751f35115 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:03.991786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48f4a228-0df1-402a-9a79-b78751f35115 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:03.992353 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48f4a228-0df1-402a-9a79-b78751f35115 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:03.992785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48f4a228-0df1-402a-9a79-b78751f35115 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:03.993540 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48f4a228-0df1-402a-9a79-b78751f35115 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:03.993990 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48f4a228-0df1-402a-9a79-b78751f35115 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:03.994395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48f4a228-0df1-402a-9a79-b78751f35115 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:04.135886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a7e2e2c1-90c8-4b80-a1f2-e3ee0896760d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:04.136493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a7e2e2c1-90c8-4b80-a1f2-e3ee0896760d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:04.146117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7e2e2c1-90c8-4b80-a1f2-e3ee0896760d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:04.146790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7e2e2c1-90c8-4b80-a1f2-e3ee0896760d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:04.147230 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7e2e2c1-90c8-4b80-a1f2-e3ee0896760d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:04.147975 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7e2e2c1-90c8-4b80-a1f2-e3ee0896760d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:04.148447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7e2e2c1-90c8-4b80-a1f2-e3ee0896760d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:04.148867 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7e2e2c1-90c8-4b80-a1f2-e3ee0896760d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:04.171570 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-48f4a228-0df1-402a-9a79-b78751f35115 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:04.180273 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-48f4a228-0df1-402a-9a79-b78751f35115 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1364 microversion: 2.1 time: 0.200150 Aug 28 00:56:04.190580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 117/241] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:03 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1364 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:04.569874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a7e2e2c1-90c8-4b80-a1f2-e3ee0896760d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:04.579754 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a7e2e2c1-90c8-4b80-a1f2-e3ee0896760d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.447732 Aug 28 00:56:04.580274 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 125/242] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:04 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:05.202225 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a3e7675f-2a44-41f3-a638-761337861e36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:05.202908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a3e7675f-2a44-41f3-a638-761337861e36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:05.210457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3e7675f-2a44-41f3-a638-761337861e36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:05.210990 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3e7675f-2a44-41f3-a638-761337861e36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:05.211506 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3e7675f-2a44-41f3-a638-761337861e36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:05.212218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3e7675f-2a44-41f3-a638-761337861e36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:05.212623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3e7675f-2a44-41f3-a638-761337861e36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:05.213074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3e7675f-2a44-41f3-a638-761337861e36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:05.385075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a3e7675f-2a44-41f3-a638-761337861e36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:05.393250 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a3e7675f-2a44-41f3-a638-761337861e36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1549 microversion: 2.1 time: 0.195122 Aug 28 00:56:05.393961 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 118/243] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:05 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1549 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:05.415260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-61764267-0819-42fb-aeae-a1c95c15e19b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:05.415948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-61764267-0819-42fb-aeae-a1c95c15e19b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:05.421945 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61764267-0819-42fb-aeae-a1c95c15e19b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:05.422425 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61764267-0819-42fb-aeae-a1c95c15e19b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:05.422847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61764267-0819-42fb-aeae-a1c95c15e19b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:05.423789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61764267-0819-42fb-aeae-a1c95c15e19b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:05.424429 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61764267-0819-42fb-aeae-a1c95c15e19b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:05.425294 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61764267-0819-42fb-aeae-a1c95c15e19b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:05.564915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-61764267-0819-42fb-aeae-a1c95c15e19b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.147726 Aug 28 00:56:05.569474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 126/244] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Aug 28 00:56:05 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-interface => generated 284 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:56:05.606557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e57d7892-daf2-401c-bdd9-b7020fbb1843 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:05.607577 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e57d7892-daf2-401c-bdd9-b7020fbb1843 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:05.615302 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e57d7892-daf2-401c-bdd9-b7020fbb1843 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:05.616029 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e57d7892-daf2-401c-bdd9-b7020fbb1843 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:05.616451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e57d7892-daf2-401c-bdd9-b7020fbb1843 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:05.617322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e57d7892-daf2-401c-bdd9-b7020fbb1843 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:05.617852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e57d7892-daf2-401c-bdd9-b7020fbb1843 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:05.618523 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e57d7892-daf2-401c-bdd9-b7020fbb1843 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:05.819423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e57d7892-daf2-401c-bdd9-b7020fbb1843 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:05.829333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e57d7892-daf2-401c-bdd9-b7020fbb1843 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.229037 Aug 28 00:56:05.830550 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 119/245] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:05 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:06.761243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-a5da666b-3ae0-4331-bec5-4183eab5b8eb req-37e2d4fa-a866-4e4a-8621-7dc5b7c88b7c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e434e4f9-9d0e-46f2-bea3-395bae33fc92", "tag": "cb80f35b-738d-4eef-8768-99836091e5a6"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:56:06.771432 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-a5da666b-3ae0-4331-bec5-4183eab5b8eb req-37e2d4fa-a866-4e4a-8621-7dc5b7c88b7c service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:06.771825 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-a5da666b-3ae0-4331-bec5-4183eab5b8eb req-37e2d4fa-a866-4e4a-8621-7dc5b7c88b7c service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:06.772303 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-a5da666b-3ae0-4331-bec5-4183eab5b8eb req-37e2d4fa-a866-4e4a-8621-7dc5b7c88b7c service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:06.786234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-a5da666b-3ae0-4331-bec5-4183eab5b8eb req-37e2d4fa-a866-4e4a-8621-7dc5b7c88b7c service nova] Creating event network-changed:cb80f35b-738d-4eef-8768-99836091e5a6 for instance e434e4f9-9d0e-46f2-bea3-395bae33fc92 on hv-cinder-maste Aug 28 00:56:06.791371 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-a5da666b-3ae0-4331-bec5-4183eab5b8eb req-37e2d4fa-a866-4e4a-8621-7dc5b7c88b7c service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034225 Aug 28 00:56:06.791974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 127/246] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:56:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:56:06.851153 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b786537d-80f4-483d-a106-84a240f3b140 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:06.851727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b786537d-80f4-483d-a106-84a240f3b140 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:06.857945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b786537d-80f4-483d-a106-84a240f3b140 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:06.858473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b786537d-80f4-483d-a106-84a240f3b140 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:06.858867 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b786537d-80f4-483d-a106-84a240f3b140 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:06.859563 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b786537d-80f4-483d-a106-84a240f3b140 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:06.860000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b786537d-80f4-483d-a106-84a240f3b140 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:06.860385 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b786537d-80f4-483d-a106-84a240f3b140 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:06.998422 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b786537d-80f4-483d-a106-84a240f3b140 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:07.006087 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b786537d-80f4-483d-a106-84a240f3b140 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.159085 Aug 28 00:56:07.006652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 120/247] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:06 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:08.028517 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-34980217-a7a5-4c42-ab27-b4d0a3bc7a07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:08.029231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-34980217-a7a5-4c42-ab27-b4d0a3bc7a07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:08.035772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-34980217-a7a5-4c42-ab27-b4d0a3bc7a07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:08.036254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-34980217-a7a5-4c42-ab27-b4d0a3bc7a07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:08.036777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-34980217-a7a5-4c42-ab27-b4d0a3bc7a07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:08.037502 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-34980217-a7a5-4c42-ab27-b4d0a3bc7a07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:08.037947 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-34980217-a7a5-4c42-ab27-b4d0a3bc7a07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:08.038363 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-34980217-a7a5-4c42-ab27-b4d0a3bc7a07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:08.185620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-34980217-a7a5-4c42-ab27-b4d0a3bc7a07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:08.194290 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-34980217-a7a5-4c42-ab27-b4d0a3bc7a07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.169972 Aug 28 00:56:08.194893 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 128/248] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:08 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:09.216114 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bc1b337c-a575-4c18-af45-dcd071f64ee5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:09.217448 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bc1b337c-a575-4c18-af45-dcd071f64ee5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:09.230468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc1b337c-a575-4c18-af45-dcd071f64ee5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:09.231177 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc1b337c-a575-4c18-af45-dcd071f64ee5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:09.231512 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc1b337c-a575-4c18-af45-dcd071f64ee5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:09.232260 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc1b337c-a575-4c18-af45-dcd071f64ee5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:09.232732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc1b337c-a575-4c18-af45-dcd071f64ee5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:09.233349 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc1b337c-a575-4c18-af45-dcd071f64ee5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:09.434964 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bc1b337c-a575-4c18-af45-dcd071f64ee5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:09.446878 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bc1b337c-a575-4c18-af45-dcd071f64ee5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.234612 Aug 28 00:56:09.449975 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 121/249] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:09 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:09.945827 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-088851c5-7369-4600-8b26-6e49c1d5c691 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:09.946320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-088851c5-7369-4600-8b26-6e49c1d5c691 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:09.953409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-088851c5-7369-4600-8b26-6e49c1d5c691 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:09.953877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-088851c5-7369-4600-8b26-6e49c1d5c691 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:09.954721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-088851c5-7369-4600-8b26-6e49c1d5c691 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:09.955428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-088851c5-7369-4600-8b26-6e49c1d5c691 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:09.955994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-088851c5-7369-4600-8b26-6e49c1d5c691 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:09.956410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-088851c5-7369-4600-8b26-6e49c1d5c691 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:09.984254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-088851c5-7369-4600-8b26-6e49c1d5c691 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Going to try to terminate instance {{(pid=120525) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 28 00:56:10.106878 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-088851c5-7369-4600-8b26-6e49c1d5c691 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "DELETE /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 204 len: 0 microversion: 2.1 time: 0.166466 Aug 28 00:56:10.107627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 129/250] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 00:56:09 2022] DELETE /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 0 bytes in 168 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 28 00:56:10.116353 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-08c59105-890b-4dfa-933a-1c8bf0c93dfa tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:10.117095 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-08c59105-890b-4dfa-933a-1c8bf0c93dfa tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:10.120844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=71 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:56:10.125774 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08c59105-890b-4dfa-933a-1c8bf0c93dfa tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:10.126450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08c59105-890b-4dfa-933a-1c8bf0c93dfa tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:10.126884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08c59105-890b-4dfa-933a-1c8bf0c93dfa tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:10.127979 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08c59105-890b-4dfa-933a-1c8bf0c93dfa tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:10.128406 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08c59105-890b-4dfa-933a-1c8bf0c93dfa tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:10.129004 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08c59105-890b-4dfa-933a-1c8bf0c93dfa tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:10.146835 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:56:10.279389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-08c59105-890b-4dfa-933a-1c8bf0c93dfa tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:10.288103 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-08c59105-890b-4dfa-933a-1c8bf0c93dfa tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1555 microversion: 2.1 time: 0.176102 Aug 28 00:56:10.288673 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 122/251] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:10 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1555 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:10.471391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9d943068-1952-406c-9fcb-ab07b1de1b89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:10.472000 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9d943068-1952-406c-9fcb-ab07b1de1b89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:10.478018 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d943068-1952-406c-9fcb-ab07b1de1b89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:10.478509 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d943068-1952-406c-9fcb-ab07b1de1b89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:10.478933 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d943068-1952-406c-9fcb-ab07b1de1b89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:10.479629 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d943068-1952-406c-9fcb-ab07b1de1b89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:10.480203 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d943068-1952-406c-9fcb-ab07b1de1b89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:10.480518 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d943068-1952-406c-9fcb-ab07b1de1b89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:10.617877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9d943068-1952-406c-9fcb-ab07b1de1b89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:10.632691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9d943068-1952-406c-9fcb-ab07b1de1b89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.165244 Aug 28 00:56:10.633770 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 130/252] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:10 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:11.309568 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-45a93d97-985e-41db-a8a3-dda140acf45f tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:11.310223 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-45a93d97-985e-41db-a8a3-dda140acf45f tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:11.316932 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-45a93d97-985e-41db-a8a3-dda140acf45f tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:11.317476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-45a93d97-985e-41db-a8a3-dda140acf45f tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:11.317896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-45a93d97-985e-41db-a8a3-dda140acf45f tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:11.318851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-45a93d97-985e-41db-a8a3-dda140acf45f tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:11.319400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-45a93d97-985e-41db-a8a3-dda140acf45f tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:11.319689 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-45a93d97-985e-41db-a8a3-dda140acf45f tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:11.472848 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-45a93d97-985e-41db-a8a3-dda140acf45f tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:11.481464 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-45a93d97-985e-41db-a8a3-dda140acf45f tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1555 microversion: 2.1 time: 0.175879 Aug 28 00:56:11.482038 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 123/253] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:11 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1555 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:11.654444 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6bb60c5b-f3e8-40d1-bf34-318a1bbc9afb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:11.655220 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6bb60c5b-f3e8-40d1-bf34-318a1bbc9afb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:11.661682 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bb60c5b-f3e8-40d1-bf34-318a1bbc9afb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:11.662276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bb60c5b-f3e8-40d1-bf34-318a1bbc9afb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:11.662734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bb60c5b-f3e8-40d1-bf34-318a1bbc9afb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:11.663408 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bb60c5b-f3e8-40d1-bf34-318a1bbc9afb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:11.663844 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bb60c5b-f3e8-40d1-bf34-318a1bbc9afb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:11.664254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bb60c5b-f3e8-40d1-bf34-318a1bbc9afb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:11.808751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6bb60c5b-f3e8-40d1-bf34-318a1bbc9afb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:11.816775 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6bb60c5b-f3e8-40d1-bf34-318a1bbc9afb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.166387 Aug 28 00:56:11.817602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 131/254] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:11 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:12.503242 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e6190432-ba39-4ab9-8980-971965982d4b tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:12.503864 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e6190432-ba39-4ab9-8980-971965982d4b tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:12.510650 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6190432-ba39-4ab9-8980-971965982d4b tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:12.511169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6190432-ba39-4ab9-8980-971965982d4b tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:12.511666 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6190432-ba39-4ab9-8980-971965982d4b tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:12.512373 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6190432-ba39-4ab9-8980-971965982d4b tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:12.512798 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6190432-ba39-4ab9-8980-971965982d4b tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:12.513241 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6190432-ba39-4ab9-8980-971965982d4b tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:12.660891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e6190432-ba39-4ab9-8980-971965982d4b tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:12.669360 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e6190432-ba39-4ab9-8980-971965982d4b tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1555 microversion: 2.1 time: 0.169977 Aug 28 00:56:12.669920 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 124/255] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:12 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1555 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:12.839380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3267e654-5aae-4b39-9862-084c72ad891f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:12.839958 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3267e654-5aae-4b39-9862-084c72ad891f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:12.847671 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3267e654-5aae-4b39-9862-084c72ad891f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:12.848149 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3267e654-5aae-4b39-9862-084c72ad891f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:12.848588 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3267e654-5aae-4b39-9862-084c72ad891f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:12.849307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3267e654-5aae-4b39-9862-084c72ad891f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:12.849753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3267e654-5aae-4b39-9862-084c72ad891f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:12.850196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3267e654-5aae-4b39-9862-084c72ad891f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:12.991825 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3267e654-5aae-4b39-9862-084c72ad891f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:13.000148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3267e654-5aae-4b39-9862-084c72ad891f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.164654 Aug 28 00:56:13.000732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 132/256] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:12 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:13.690670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f4232f5d-8dfb-4536-9e78-87934d7e7756 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:13.691246 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f4232f5d-8dfb-4536-9e78-87934d7e7756 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:13.697716 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4232f5d-8dfb-4536-9e78-87934d7e7756 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:13.698162 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4232f5d-8dfb-4536-9e78-87934d7e7756 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:13.699139 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4232f5d-8dfb-4536-9e78-87934d7e7756 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:13.699835 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4232f5d-8dfb-4536-9e78-87934d7e7756 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:13.700292 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4232f5d-8dfb-4536-9e78-87934d7e7756 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:13.700712 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4232f5d-8dfb-4536-9e78-87934d7e7756 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:13.840770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f4232f5d-8dfb-4536-9e78-87934d7e7756 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:13.850299 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f4232f5d-8dfb-4536-9e78-87934d7e7756 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1513 microversion: 2.1 time: 0.163615 Aug 28 00:56:13.851181 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 125/257] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:13 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1513 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:56:13.879215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-9e535507-06df-4a76-94d5-d172e5fec754 req-14e06851-029a-4b73-8cc8-51e6c0deebc3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4d664b12-2edd-46e0-bcf3-4d94eb576bcd", "name": "network-vif-deleted", "tag": "ffe6cab9-475d-45bb-9156-c9785ed386fd"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:56:13.888790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-9e535507-06df-4a76-94d5-d172e5fec754 req-14e06851-029a-4b73-8cc8-51e6c0deebc3 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:13.889311 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-9e535507-06df-4a76-94d5-d172e5fec754 req-14e06851-029a-4b73-8cc8-51e6c0deebc3 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:13.889729 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-9e535507-06df-4a76-94d5-d172e5fec754 req-14e06851-029a-4b73-8cc8-51e6c0deebc3 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:13.901477 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-9e535507-06df-4a76-94d5-d172e5fec754 req-14e06851-029a-4b73-8cc8-51e6c0deebc3 service nova] Creating event network-vif-deleted:ffe6cab9-475d-45bb-9156-c9785ed386fd for instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd on hv-cinder-maste Aug 28 00:56:13.906743 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-9e535507-06df-4a76-94d5-d172e5fec754 req-14e06851-029a-4b73-8cc8-51e6c0deebc3 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031188 Aug 28 00:56:13.907394 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 133/258] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:56:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:56:14.021417 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1934d0ab-8662-4e7d-813e-32de4df007f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:14.022092 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1934d0ab-8662-4e7d-813e-32de4df007f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:14.028321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1934d0ab-8662-4e7d-813e-32de4df007f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:14.028786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1934d0ab-8662-4e7d-813e-32de4df007f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:14.029239 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1934d0ab-8662-4e7d-813e-32de4df007f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:14.029925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1934d0ab-8662-4e7d-813e-32de4df007f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:14.030371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1934d0ab-8662-4e7d-813e-32de4df007f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:14.030797 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1934d0ab-8662-4e7d-813e-32de4df007f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:14.175058 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1934d0ab-8662-4e7d-813e-32de4df007f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:14.182931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1934d0ab-8662-4e7d-813e-32de4df007f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.165413 Aug 28 00:56:14.183496 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 126/259] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:14 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:14.872224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3be1c3e1-986d-4eb7-88a2-4e52884f626d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:14.872912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3be1c3e1-986d-4eb7-88a2-4e52884f626d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:14.879557 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3be1c3e1-986d-4eb7-88a2-4e52884f626d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:14.880160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3be1c3e1-986d-4eb7-88a2-4e52884f626d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:14.880445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3be1c3e1-986d-4eb7-88a2-4e52884f626d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:14.881157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3be1c3e1-986d-4eb7-88a2-4e52884f626d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:14.881614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3be1c3e1-986d-4eb7-88a2-4e52884f626d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:14.882029 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3be1c3e1-986d-4eb7-88a2-4e52884f626d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:14.914874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3be1c3e1-986d-4eb7-88a2-4e52884f626d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:14.915951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3be1c3e1-986d-4eb7-88a2-4e52884f626d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:14.915951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3be1c3e1-986d-4eb7-88a2-4e52884f626d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:14.916341 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.objects.instance [None req-3be1c3e1-986d-4eb7-88a2-4e52884f626d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lazy-loading 'fault' on Instance uuid 4d664b12-2edd-46e0-bcf3-4d94eb576bcd {{(pid=120525) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 00:56:14.968280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3be1c3e1-986d-4eb7-88a2-4e52884f626d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd49314743b14266a6a646ffd4d26698', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd36dd9ee223f496ebc7b6de74b92227a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:14.977396 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3be1c3e1-986d-4eb7-88a2-4e52884f626d tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 200 len: 1351 microversion: 2.1 time: 0.109376 Aug 28 00:56:14.977965 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 134/260] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:14 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 1351 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:15.206277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9bc09629-3d4e-4b9c-9d2a-8ad3b97634bb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:15.206915 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9bc09629-3d4e-4b9c-9d2a-8ad3b97634bb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:15.214106 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bc09629-3d4e-4b9c-9d2a-8ad3b97634bb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:15.214589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bc09629-3d4e-4b9c-9d2a-8ad3b97634bb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:15.215013 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bc09629-3d4e-4b9c-9d2a-8ad3b97634bb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:15.215790 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bc09629-3d4e-4b9c-9d2a-8ad3b97634bb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:15.216282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bc09629-3d4e-4b9c-9d2a-8ad3b97634bb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:15.216732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bc09629-3d4e-4b9c-9d2a-8ad3b97634bb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:15.371911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9bc09629-3d4e-4b9c-9d2a-8ad3b97634bb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:15.380019 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9bc09629-3d4e-4b9c-9d2a-8ad3b97634bb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.177694 Aug 28 00:56:15.380582 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 127/261] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:15 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:15.997908 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-91fc460a-d86b-489f-b670-30cf56028f7c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:15.998512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-91fc460a-d86b-489f-b670-30cf56028f7c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:16.005575 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91fc460a-d86b-489f-b670-30cf56028f7c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:16.006188 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91fc460a-d86b-489f-b670-30cf56028f7c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:16.006620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91fc460a-d86b-489f-b670-30cf56028f7c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:16.007374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91fc460a-d86b-489f-b670-30cf56028f7c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:16.007856 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91fc460a-d86b-489f-b670-30cf56028f7c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:16.008353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91fc460a-d86b-489f-b670-30cf56028f7c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:16.024749 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-91fc460a-d86b-489f-b670-30cf56028f7c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] HTTP exception thrown: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. Aug 28 00:56:16.030243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-91fc460a-d86b-489f-b670-30cf56028f7c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Request method failure captured: Aug 28 00:56:16.030243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd HTTP/1.1 Aug 28 00:56:16.030243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 00:56:16.030243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 00:56:16.030243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 00:56:16.030243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 00:56:16.030243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 00:56:16.030243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 00:56:16.030243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 00:56:16.030243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 00:56:16.030243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCrzdLYOJhWd3MeA7NiW3tfJ5mC92jhDopIhjsoyoncG18LLiiMDKChjjeXFl_IlV6DFztuqCoCPC_yg9W1eveIvTzTSPPmmtVUvfim7w9FALRP66exfVG5QsWJsA6Vn_C-TcxkItK7SpuKRcr2M-uph4uEF-MO3ep1d5wHQ1Zk4_GpE Aug 28 00:56:16.030243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 00:56:16.030243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 00:56:16.030243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 00:56:16.030243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 00:56:16.030243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 00:56:16.030243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 00:56:16.030243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: d36dd9ee223f496ebc7b6de74b92227a Aug 28 00:56:16.030243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-DeleteServersTestJSON-997478319 Aug 28 00:56:16.030243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 00:56:16.030243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 00:56:16.031857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_d36dd9ee223f496ebc7b6de74b92227a", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/d36dd9ee223f496ebc7b6de74b92227a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/d36dd9ee223f496ebc7b6de74b92227a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/d36dd9ee223f496ebc7b6de74b92227a"}]}] Aug 28 00:56:16.031857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-DeleteServersTestJSON-997478319 Aug 28 00:56:16.031857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: d36dd9ee223f496ebc7b6de74b92227a Aug 28 00:56:16.031857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-DeleteServersTestJSON-997478319 Aug 28 00:56:16.031857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-DeleteServersTestJSON-997478319-project Aug 28 00:56:16.031857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 00:56:16.031857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 00:56:16.031857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: dd49314743b14266a6a646ffd4d26698 Aug 28 00:56:16.031857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-DeleteServersTestJSON-997478319-project Aug 28 00:56:16.031857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 00:56:16.031857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. Aug 28 00:56:16.031857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'id': '4d664b12-2edd-46e0-bcf3-4d94eb576bcd'} Aug 28 00:56:16.032711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 00:56:16.032711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:56:16.032711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 00:56:16.032711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 00:56:16.032711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 00:56:16.032711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 00:56:16.032711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 00:56:16.032711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 00:56:16.032711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 00:56:16.032711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 00:56:16.032711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. Aug 28 00:56:16.032711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:56:16.032711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:56:16.032711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:56:16.032711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:56:16.032711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:56:16.032711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:56:16.032711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 00:56:16.032711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 00:56:16.032711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 00:56:16.033860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:56:16.033860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 00:56:16.033860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 00:56:16.033860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 00:56:16.033860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 00:56:16.033860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 00:56:16.033860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 00:56:16.033860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. Aug 28 00:56:16.033860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:56:16.033860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:56:16.033860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:56:16.033860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:56:16.033860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:56:16.033860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:56:16.033860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 00:56:16.033860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 00:56:16.033860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. Aug 28 00:56:16.033860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:56:16.034877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-91fc460a-d86b-489f-b670-30cf56028f7c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Returning 404 to user: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 00:56:16.034877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-91fc460a-d86b-489f-b670-30cf56028f7c tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 404 len: 111 microversion: 2.1 time: 0.037931 Aug 28 00:56:16.034877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 135/262] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:15 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 00:56:16.149343 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-267ae381-5d7e-4800-9632-8402310c6d47 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:16.151616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-267ae381-5d7e-4800-9632-8402310c6d47 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:16.157884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-267ae381-5d7e-4800-9632-8402310c6d47 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:16.158363 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-267ae381-5d7e-4800-9632-8402310c6d47 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:16.158791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-267ae381-5d7e-4800-9632-8402310c6d47 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:16.159470 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-267ae381-5d7e-4800-9632-8402310c6d47 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:16.159909 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-267ae381-5d7e-4800-9632-8402310c6d47 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:16.160339 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-267ae381-5d7e-4800-9632-8402310c6d47 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:16.176358 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-267ae381-5d7e-4800-9632-8402310c6d47 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] HTTP exception thrown: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. Aug 28 00:56:16.181347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-267ae381-5d7e-4800-9632-8402310c6d47 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Request method failure captured: Aug 28 00:56:16.181347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd/os-volume_attachments HTTP/1.1 Aug 28 00:56:16.181347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 00:56:16.181347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 00:56:16.181347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 00:56:16.181347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 00:56:16.181347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 00:56:16.181347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 00:56:16.181347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 00:56:16.181347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 00:56:16.181347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCrzdLYOJhWd3MeA7NiW3tfJ5mC92jhDopIhjsoyoncG18LLiiMDKChjjeXFl_IlV6DFztuqCoCPC_yg9W1eveIvTzTSPPmmtVUvfim7w9FALRP66exfVG5QsWJsA6Vn_C-TcxkItK7SpuKRcr2M-uph4uEF-MO3ep1d5wHQ1Zk4_GpE Aug 28 00:56:16.181347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 00:56:16.181347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 00:56:16.181347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 00:56:16.181347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 00:56:16.181347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 00:56:16.181347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 00:56:16.181347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: d36dd9ee223f496ebc7b6de74b92227a Aug 28 00:56:16.181347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-DeleteServersTestJSON-997478319 Aug 28 00:56:16.181347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 00:56:16.181347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 00:56:16.182755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_d36dd9ee223f496ebc7b6de74b92227a", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/d36dd9ee223f496ebc7b6de74b92227a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/d36dd9ee223f496ebc7b6de74b92227a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/d36dd9ee223f496ebc7b6de74b92227a"}]}] Aug 28 00:56:16.182755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-DeleteServersTestJSON-997478319 Aug 28 00:56:16.182755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: d36dd9ee223f496ebc7b6de74b92227a Aug 28 00:56:16.182755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-DeleteServersTestJSON-997478319 Aug 28 00:56:16.182755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-DeleteServersTestJSON-997478319-project Aug 28 00:56:16.182755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 00:56:16.182755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 00:56:16.182755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: dd49314743b14266a6a646ffd4d26698 Aug 28 00:56:16.182755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-DeleteServersTestJSON-997478319-project Aug 28 00:56:16.182755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 00:56:16.182755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. Aug 28 00:56:16.182755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'server_id': '4d664b12-2edd-46e0-bcf3-4d94eb576bcd'} Aug 28 00:56:16.183620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 00:56:16.183620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:56:16.183620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 00:56:16.183620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 00:56:16.183620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 00:56:16.183620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 00:56:16.183620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 00:56:16.183620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 00:56:16.183620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 00:56:16.183620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 00:56:16.183620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. Aug 28 00:56:16.183620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:56:16.183620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:56:16.183620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:56:16.183620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:56:16.183620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:56:16.183620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:56:16.183620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 00:56:16.183620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 00:56:16.183620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 00:56:16.185056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:56:16.185056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Aug 28 00:56:16.185056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 28 00:56:16.185056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Aug 28 00:56:16.185056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 28 00:56:16.185056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Aug 28 00:56:16.185056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Aug 28 00:56:16.185056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 00:56:16.185056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 00:56:16.185056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. Aug 28 00:56:16.185056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:56:16.185056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:56:16.185056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:56:16.185056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:56:16.185056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:56:16.185056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:56:16.185056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 00:56:16.185056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 00:56:16.185056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. Aug 28 00:56:16.185056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:56:16.186652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-267ae381-5d7e-4800-9632-8402310c6d47 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Returning 404 to user: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 00:56:16.186652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-267ae381-5d7e-4800-9632-8402310c6d47 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.037479 Aug 28 00:56:16.186652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 128/263] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 00:56:16 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd/os-volume_attachments => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 00:56:16.405799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6659e134-ebd9-480c-a771-5bcc080d04b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:16.406327 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6659e134-ebd9-480c-a771-5bcc080d04b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:16.413630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6659e134-ebd9-480c-a771-5bcc080d04b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:16.414132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6659e134-ebd9-480c-a771-5bcc080d04b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:16.414562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6659e134-ebd9-480c-a771-5bcc080d04b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:16.415321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6659e134-ebd9-480c-a771-5bcc080d04b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:16.415947 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6659e134-ebd9-480c-a771-5bcc080d04b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:16.416373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6659e134-ebd9-480c-a771-5bcc080d04b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:16.559018 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6659e134-ebd9-480c-a771-5bcc080d04b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:16.566929 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6659e134-ebd9-480c-a771-5bcc080d04b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.165310 Aug 28 00:56:16.567550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 136/264] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:16 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:17.404516 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0999639d-4a0a-4805-b626-534b9d478acd tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:17.405093 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0999639d-4a0a-4805-b626-534b9d478acd tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:17.416078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0999639d-4a0a-4805-b626-534b9d478acd tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:17.416829 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0999639d-4a0a-4805-b626-534b9d478acd tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:17.417467 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0999639d-4a0a-4805-b626-534b9d478acd tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:17.418370 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0999639d-4a0a-4805-b626-534b9d478acd tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:17.418820 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0999639d-4a0a-4805-b626-534b9d478acd tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:17.419238 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0999639d-4a0a-4805-b626-534b9d478acd tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:17.433959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-0999639d-4a0a-4805-b626-534b9d478acd tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] HTTP exception thrown: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. Aug 28 00:56:17.436169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0999639d-4a0a-4805-b626-534b9d478acd tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Request method failure captured: Aug 28 00:56:17.436169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: DELETE /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd HTTP/1.1 Aug 28 00:56:17.436169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 00:56:17.436169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 00:56:17.436169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 00:56:17.436169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 00:56:17.436169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 00:56:17.436169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 00:56:17.436169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 00:56:17.436169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 00:56:17.436169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCrzdLYOJhWd3MeA7NiW3tfJ5mC92jhDopIhjsoyoncG18LLiiMDKChjjeXFl_IlV6DFztuqCoCPC_yg9W1eveIvTzTSPPmmtVUvfim7w9FALRP66exfVG5QsWJsA6Vn_C-TcxkItK7SpuKRcr2M-uph4uEF-MO3ep1d5wHQ1Zk4_GpE Aug 28 00:56:17.436169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 00:56:17.436169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 00:56:17.436169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 00:56:17.436169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 00:56:17.436169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 00:56:17.436169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 00:56:17.436169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: d36dd9ee223f496ebc7b6de74b92227a Aug 28 00:56:17.436169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-DeleteServersTestJSON-997478319 Aug 28 00:56:17.436169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 00:56:17.436169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 00:56:17.437706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_d36dd9ee223f496ebc7b6de74b92227a", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/d36dd9ee223f496ebc7b6de74b92227a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/d36dd9ee223f496ebc7b6de74b92227a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/d36dd9ee223f496ebc7b6de74b92227a"}]}] Aug 28 00:56:17.437706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-DeleteServersTestJSON-997478319 Aug 28 00:56:17.437706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: d36dd9ee223f496ebc7b6de74b92227a Aug 28 00:56:17.437706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-DeleteServersTestJSON-997478319 Aug 28 00:56:17.437706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-DeleteServersTestJSON-997478319-project Aug 28 00:56:17.437706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 00:56:17.437706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 00:56:17.437706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: dd49314743b14266a6a646ffd4d26698 Aug 28 00:56:17.437706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-DeleteServersTestJSON-997478319-project Aug 28 00:56:17.437706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 00:56:17.437706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. Aug 28 00:56:17.437706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'id': '4d664b12-2edd-46e0-bcf3-4d94eb576bcd'} Aug 28 00:56:17.438541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 00:56:17.438541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:56:17.438541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 00:56:17.438541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 00:56:17.438541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 00:56:17.438541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 00:56:17.438541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 00:56:17.438541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 00:56:17.438541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 00:56:17.438541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 00:56:17.438541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. Aug 28 00:56:17.438541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:56:17.438541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:56:17.438541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:56:17.438541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:56:17.438541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:56:17.438541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:56:17.438541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 00:56:17.438541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 00:56:17.438541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 00:56:17.439608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:56:17.439608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 28 00:56:17.439608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 28 00:56:17.439608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 28 00:56:17.439608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 28 00:56:17.439608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 00:56:17.439608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 00:56:17.439608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 00:56:17.439608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 00:56:17.439608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. Aug 28 00:56:17.439608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:56:17.439608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:56:17.439608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:56:17.439608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:56:17.439608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:56:17.439608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:56:17.439608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 00:56:17.439608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 00:56:17.439608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. Aug 28 00:56:17.439608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 00:56:17.440700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0999639d-4a0a-4805-b626-534b9d478acd tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Returning 404 to user: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 00:56:17.440700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0999639d-4a0a-4805-b626-534b9d478acd tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "DELETE /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 404 len: 111 microversion: 2.1 time: 0.037671 Aug 28 00:56:17.440700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 129/265] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 00:56:17 2022] DELETE /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 00:56:17.446703 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-64552c98-c08e-4f3d-8ddb-d5b7c3b239f6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:17.447824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-64552c98-c08e-4f3d-8ddb-d5b7c3b239f6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] [instance: 4d664b12-2edd-46e0-bcf3-4d94eb576bcd] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:17.454555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64552c98-c08e-4f3d-8ddb-d5b7c3b239f6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:17.455272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64552c98-c08e-4f3d-8ddb-d5b7c3b239f6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:17.455705 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64552c98-c08e-4f3d-8ddb-d5b7c3b239f6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:17.456430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64552c98-c08e-4f3d-8ddb-d5b7c3b239f6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:17.456876 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64552c98-c08e-4f3d-8ddb-d5b7c3b239f6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:17.457429 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64552c98-c08e-4f3d-8ddb-d5b7c3b239f6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:17.471841 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-64552c98-c08e-4f3d-8ddb-d5b7c3b239f6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] HTTP exception thrown: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. Aug 28 00:56:17.473455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-64552c98-c08e-4f3d-8ddb-d5b7c3b239f6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Request method failure captured: Aug 28 00:56:17.473455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd HTTP/1.1 Aug 28 00:56:17.473455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 00:56:17.473455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 00:56:17.473455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 00:56:17.473455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 00:56:17.473455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 00:56:17.473455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 00:56:17.473455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 00:56:17.473455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 00:56:17.473455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCrzdLYOJhWd3MeA7NiW3tfJ5mC92jhDopIhjsoyoncG18LLiiMDKChjjeXFl_IlV6DFztuqCoCPC_yg9W1eveIvTzTSPPmmtVUvfim7w9FALRP66exfVG5QsWJsA6Vn_C-TcxkItK7SpuKRcr2M-uph4uEF-MO3ep1d5wHQ1Zk4_GpE Aug 28 00:56:17.473455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 00:56:17.473455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 00:56:17.473455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 00:56:17.473455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 00:56:17.473455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 00:56:17.473455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 00:56:17.473455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: d36dd9ee223f496ebc7b6de74b92227a Aug 28 00:56:17.473455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-DeleteServersTestJSON-997478319 Aug 28 00:56:17.473455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 00:56:17.473455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 00:56:17.474719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_d36dd9ee223f496ebc7b6de74b92227a", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/d36dd9ee223f496ebc7b6de74b92227a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/d36dd9ee223f496ebc7b6de74b92227a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/d36dd9ee223f496ebc7b6de74b92227a"}]}] Aug 28 00:56:17.474719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-DeleteServersTestJSON-997478319 Aug 28 00:56:17.474719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: d36dd9ee223f496ebc7b6de74b92227a Aug 28 00:56:17.474719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-DeleteServersTestJSON-997478319 Aug 28 00:56:17.474719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-DeleteServersTestJSON-997478319-project Aug 28 00:56:17.474719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 00:56:17.474719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 00:56:17.474719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: dd49314743b14266a6a646ffd4d26698 Aug 28 00:56:17.474719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-DeleteServersTestJSON-997478319-project Aug 28 00:56:17.474719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 00:56:17.474719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. Aug 28 00:56:17.474719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'id': '4d664b12-2edd-46e0-bcf3-4d94eb576bcd'} Aug 28 00:56:17.475374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 00:56:17.475374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:56:17.475374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 00:56:17.475374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 00:56:17.475374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 00:56:17.475374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 00:56:17.475374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 00:56:17.475374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 00:56:17.475374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 00:56:17.475374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 00:56:17.475374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. Aug 28 00:56:17.475374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:56:17.475374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:56:17.475374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:56:17.475374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:56:17.475374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:56:17.475374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:56:17.475374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 00:56:17.475374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 00:56:17.475374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 00:56:17.476493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:56:17.476493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 00:56:17.476493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 00:56:17.476493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 00:56:17.476493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 00:56:17.476493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 00:56:17.476493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 00:56:17.476493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. Aug 28 00:56:17.476493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:56:17.476493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:56:17.476493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:56:17.476493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:56:17.476493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:56:17.476493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:56:17.476493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 00:56:17.476493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 00:56:17.476493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. Aug 28 00:56:17.476493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:56:17.477567 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-64552c98-c08e-4f3d-8ddb-d5b7c3b239f6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] Returning 404 to user: Instance 4d664b12-2edd-46e0-bcf3-4d94eb576bcd could not be found. {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 00:56:17.477567 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-64552c98-c08e-4f3d-8ddb-d5b7c3b239f6 tempest-DeleteServersTestJSON-997478319 tempest-DeleteServersTestJSON-997478319-project] 10.222.0.42 "GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd" status: 404 len: 111 microversion: 2.1 time: 0.032416 Aug 28 00:56:17.477567 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 137/266] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:17 2022] GET /compute/v2.1/servers/4d664b12-2edd-46e0-bcf3-4d94eb576bcd => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 00:56:17.588781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ef749d31-1972-486f-9d38-efbc9a518942 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:17.589401 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ef749d31-1972-486f-9d38-efbc9a518942 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:17.597467 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ef749d31-1972-486f-9d38-efbc9a518942 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:17.598041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ef749d31-1972-486f-9d38-efbc9a518942 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:17.598476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ef749d31-1972-486f-9d38-efbc9a518942 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:17.599174 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ef749d31-1972-486f-9d38-efbc9a518942 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:17.599607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ef749d31-1972-486f-9d38-efbc9a518942 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:17.600020 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ef749d31-1972-486f-9d38-efbc9a518942 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:18.008554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ef749d31-1972-486f-9d38-efbc9a518942 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:18.027341 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ef749d31-1972-486f-9d38-efbc9a518942 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.443081 Aug 28 00:56:18.027886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 130/267] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:17 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:19.049357 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1a2506ea-b427-40d7-b98d-d158212546f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:19.050015 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1a2506ea-b427-40d7-b98d-d158212546f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:19.058697 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a2506ea-b427-40d7-b98d-d158212546f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:19.058697 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a2506ea-b427-40d7-b98d-d158212546f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:19.058697 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a2506ea-b427-40d7-b98d-d158212546f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:19.058697 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a2506ea-b427-40d7-b98d-d158212546f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:19.061729 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a2506ea-b427-40d7-b98d-d158212546f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:19.062210 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a2506ea-b427-40d7-b98d-d158212546f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:19.265165 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1a2506ea-b427-40d7-b98d-d158212546f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:19.273683 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1a2506ea-b427-40d7-b98d-d158212546f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.228263 Aug 28 00:56:19.274324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 138/268] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:19 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:20.296059 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9911b0c9-88dd-4c04-9cb9-c84019d40e06 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:20.296602 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9911b0c9-88dd-4c04-9cb9-c84019d40e06 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:20.305396 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9911b0c9-88dd-4c04-9cb9-c84019d40e06 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:20.306134 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9911b0c9-88dd-4c04-9cb9-c84019d40e06 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:20.306538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9911b0c9-88dd-4c04-9cb9-c84019d40e06 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:20.307400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9911b0c9-88dd-4c04-9cb9-c84019d40e06 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:20.307849 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9911b0c9-88dd-4c04-9cb9-c84019d40e06 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:20.308249 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9911b0c9-88dd-4c04-9cb9-c84019d40e06 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:20.557663 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9911b0c9-88dd-4c04-9cb9-c84019d40e06 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:20.572828 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9911b0c9-88dd-4c04-9cb9-c84019d40e06 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.280111 Aug 28 00:56:20.575812 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 131/269] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:20 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:21.596894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6dd8846e-5058-4753-abcd-ca0841ef8a46 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:21.597791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6dd8846e-5058-4753-abcd-ca0841ef8a46 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:21.606187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6dd8846e-5058-4753-abcd-ca0841ef8a46 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:21.606788 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6dd8846e-5058-4753-abcd-ca0841ef8a46 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:21.607342 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6dd8846e-5058-4753-abcd-ca0841ef8a46 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:21.608172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6dd8846e-5058-4753-abcd-ca0841ef8a46 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:21.608883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6dd8846e-5058-4753-abcd-ca0841ef8a46 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:21.609422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6dd8846e-5058-4753-abcd-ca0841ef8a46 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:21.766383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6dd8846e-5058-4753-abcd-ca0841ef8a46 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:21.776098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6dd8846e-5058-4753-abcd-ca0841ef8a46 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.183832 Aug 28 00:56:21.777176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 139/270] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:21 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:22.797943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c569b304-1387-4022-92c9-c7a4b352ba81 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:22.798492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c569b304-1387-4022-92c9-c7a4b352ba81 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:22.805005 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c569b304-1387-4022-92c9-c7a4b352ba81 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:22.805639 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c569b304-1387-4022-92c9-c7a4b352ba81 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:22.806105 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c569b304-1387-4022-92c9-c7a4b352ba81 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:22.806773 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c569b304-1387-4022-92c9-c7a4b352ba81 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:22.807302 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c569b304-1387-4022-92c9-c7a4b352ba81 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:22.807723 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c569b304-1387-4022-92c9-c7a4b352ba81 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:22.945569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c569b304-1387-4022-92c9-c7a4b352ba81 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:22.954532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c569b304-1387-4022-92c9-c7a4b352ba81 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.160779 Aug 28 00:56:22.955113 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 132/271] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:22 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:23.977463 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-558e8a27-6ec0-4220-a78b-dc801fdb830e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:23.978446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-558e8a27-6ec0-4220-a78b-dc801fdb830e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:23.985202 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-558e8a27-6ec0-4220-a78b-dc801fdb830e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:23.985722 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-558e8a27-6ec0-4220-a78b-dc801fdb830e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:23.986110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-558e8a27-6ec0-4220-a78b-dc801fdb830e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:23.986810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-558e8a27-6ec0-4220-a78b-dc801fdb830e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:23.987313 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-558e8a27-6ec0-4220-a78b-dc801fdb830e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:23.987734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-558e8a27-6ec0-4220-a78b-dc801fdb830e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:24.138238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-558e8a27-6ec0-4220-a78b-dc801fdb830e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:24.145979 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-558e8a27-6ec0-4220-a78b-dc801fdb830e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.173608 Aug 28 00:56:24.155685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 140/272] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:23 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:25.169737 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-36982bc2-60cc-4811-8019-c219b99306e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:25.170347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-36982bc2-60cc-4811-8019-c219b99306e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:25.177195 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36982bc2-60cc-4811-8019-c219b99306e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:25.177678 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36982bc2-60cc-4811-8019-c219b99306e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:25.178109 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36982bc2-60cc-4811-8019-c219b99306e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:25.178790 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36982bc2-60cc-4811-8019-c219b99306e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:25.179228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36982bc2-60cc-4811-8019-c219b99306e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:25.179643 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36982bc2-60cc-4811-8019-c219b99306e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:25.329555 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-36982bc2-60cc-4811-8019-c219b99306e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:25.338491 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-36982bc2-60cc-4811-8019-c219b99306e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.174614 Aug 28 00:56:25.339174 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 133/273] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:25 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:26.360354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-06bbd111-c1fe-484b-97bb-ae6b1673f5b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:26.360982 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-06bbd111-c1fe-484b-97bb-ae6b1673f5b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:26.367142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-06bbd111-c1fe-484b-97bb-ae6b1673f5b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:26.367663 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-06bbd111-c1fe-484b-97bb-ae6b1673f5b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:26.368056 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-06bbd111-c1fe-484b-97bb-ae6b1673f5b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:26.368935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-06bbd111-c1fe-484b-97bb-ae6b1673f5b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:26.369413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-06bbd111-c1fe-484b-97bb-ae6b1673f5b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:26.369824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-06bbd111-c1fe-484b-97bb-ae6b1673f5b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:26.374207 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:56:26.507335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-06bbd111-c1fe-484b-97bb-ae6b1673f5b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:26.515983 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-06bbd111-c1fe-484b-97bb-ae6b1673f5b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.159389 Aug 28 00:56:26.516710 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 141/274] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:26 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:27.536615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-311d7a08-2cb5-448c-81f4-8d959bde5071 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:27.537325 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-311d7a08-2cb5-448c-81f4-8d959bde5071 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:27.544308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-311d7a08-2cb5-448c-81f4-8d959bde5071 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:27.544791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-311d7a08-2cb5-448c-81f4-8d959bde5071 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:27.545272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-311d7a08-2cb5-448c-81f4-8d959bde5071 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:27.545967 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-311d7a08-2cb5-448c-81f4-8d959bde5071 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:27.546414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-311d7a08-2cb5-448c-81f4-8d959bde5071 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:27.546834 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-311d7a08-2cb5-448c-81f4-8d959bde5071 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:27.688299 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-311d7a08-2cb5-448c-81f4-8d959bde5071 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:27.696509 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-311d7a08-2cb5-448c-81f4-8d959bde5071 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.163771 Aug 28 00:56:27.697171 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 134/275] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:27 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:28.719051 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1f84b9ba-b335-49a2-8a50-926e40d5286c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:28.719668 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1f84b9ba-b335-49a2-8a50-926e40d5286c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:28.726323 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1f84b9ba-b335-49a2-8a50-926e40d5286c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:28.726719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1f84b9ba-b335-49a2-8a50-926e40d5286c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:28.727140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1f84b9ba-b335-49a2-8a50-926e40d5286c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:28.727898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1f84b9ba-b335-49a2-8a50-926e40d5286c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:28.728320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1f84b9ba-b335-49a2-8a50-926e40d5286c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:28.728864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1f84b9ba-b335-49a2-8a50-926e40d5286c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:28.863650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1f84b9ba-b335-49a2-8a50-926e40d5286c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:28.872400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1f84b9ba-b335-49a2-8a50-926e40d5286c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.157394 Aug 28 00:56:28.873683 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 142/276] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:28 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:29.893505 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-14fcb94e-0cb8-4419-ad38-2d2982144561 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:29.894209 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-14fcb94e-0cb8-4419-ad38-2d2982144561 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:29.900889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14fcb94e-0cb8-4419-ad38-2d2982144561 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:29.901401 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14fcb94e-0cb8-4419-ad38-2d2982144561 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:29.901823 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14fcb94e-0cb8-4419-ad38-2d2982144561 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:29.902510 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14fcb94e-0cb8-4419-ad38-2d2982144561 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:29.902950 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14fcb94e-0cb8-4419-ad38-2d2982144561 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:29.903362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14fcb94e-0cb8-4419-ad38-2d2982144561 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:30.040756 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-14fcb94e-0cb8-4419-ad38-2d2982144561 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:30.048619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-14fcb94e-0cb8-4419-ad38-2d2982144561 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.159019 Aug 28 00:56:30.049203 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 135/277] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:29 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:31.069123 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7848e189-3d9f-4a64-b6c6-c0bbd648448d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:31.070027 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7848e189-3d9f-4a64-b6c6-c0bbd648448d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:31.072083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=51,nova_api:UPDATE=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:56:31.077687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7848e189-3d9f-4a64-b6c6-c0bbd648448d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:31.078177 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7848e189-3d9f-4a64-b6c6-c0bbd648448d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:31.078596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7848e189-3d9f-4a64-b6c6-c0bbd648448d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:31.079339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7848e189-3d9f-4a64-b6c6-c0bbd648448d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:31.079935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7848e189-3d9f-4a64-b6c6-c0bbd648448d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:31.080358 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7848e189-3d9f-4a64-b6c6-c0bbd648448d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:31.232214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7848e189-3d9f-4a64-b6c6-c0bbd648448d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:31.240215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7848e189-3d9f-4a64-b6c6-c0bbd648448d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.174966 Aug 28 00:56:31.240784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 143/278] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:31 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:32.265628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0081f05a-be53-4c87-862f-86fedc28b6b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:32.266820 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0081f05a-be53-4c87-862f-86fedc28b6b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:32.278170 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0081f05a-be53-4c87-862f-86fedc28b6b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:32.279009 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0081f05a-be53-4c87-862f-86fedc28b6b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:32.279754 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0081f05a-be53-4c87-862f-86fedc28b6b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:32.280981 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0081f05a-be53-4c87-862f-86fedc28b6b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:32.281733 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0081f05a-be53-4c87-862f-86fedc28b6b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:32.282437 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0081f05a-be53-4c87-862f-86fedc28b6b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:32.445410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0081f05a-be53-4c87-862f-86fedc28b6b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:32.454461 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0081f05a-be53-4c87-862f-86fedc28b6b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.195553 Aug 28 00:56:32.454979 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 136/279] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:32 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:33.475702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c68675f0-7c15-4399-8241-31bcb8965be2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:33.476288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c68675f0-7c15-4399-8241-31bcb8965be2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:33.482779 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c68675f0-7c15-4399-8241-31bcb8965be2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:33.483251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c68675f0-7c15-4399-8241-31bcb8965be2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:33.483694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c68675f0-7c15-4399-8241-31bcb8965be2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:33.484361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c68675f0-7c15-4399-8241-31bcb8965be2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:33.484856 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c68675f0-7c15-4399-8241-31bcb8965be2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:33.485310 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c68675f0-7c15-4399-8241-31bcb8965be2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:33.663267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c68675f0-7c15-4399-8241-31bcb8965be2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:33.669413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c68675f0-7c15-4399-8241-31bcb8965be2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.197478 Aug 28 00:56:33.669976 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 144/280] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:33 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:34.690216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9757849b-b6b7-48b3-bb42-828c21310b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:34.690851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9757849b-b6b7-48b3-bb42-828c21310b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:34.697564 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9757849b-b6b7-48b3-bb42-828c21310b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:34.698056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9757849b-b6b7-48b3-bb42-828c21310b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:34.698791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9757849b-b6b7-48b3-bb42-828c21310b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:34.699491 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9757849b-b6b7-48b3-bb42-828c21310b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:34.699940 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9757849b-b6b7-48b3-bb42-828c21310b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:34.700361 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9757849b-b6b7-48b3-bb42-828c21310b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:34.872508 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9757849b-b6b7-48b3-bb42-828c21310b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:34.881278 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9757849b-b6b7-48b3-bb42-828c21310b85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.194782 Aug 28 00:56:34.881843 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 137/281] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:34 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:35.906442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3bd3733d-0d99-477f-9df4-fd54391a4a63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:35.907495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3bd3733d-0d99-477f-9df4-fd54391a4a63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:35.916526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bd3733d-0d99-477f-9df4-fd54391a4a63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:35.917211 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bd3733d-0d99-477f-9df4-fd54391a4a63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:35.918101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bd3733d-0d99-477f-9df4-fd54391a4a63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:35.919181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bd3733d-0d99-477f-9df4-fd54391a4a63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:35.919913 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bd3733d-0d99-477f-9df4-fd54391a4a63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:35.920272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bd3733d-0d99-477f-9df4-fd54391a4a63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:36.216201 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3bd3733d-0d99-477f-9df4-fd54391a4a63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:36.224132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3bd3733d-0d99-477f-9df4-fd54391a4a63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.324466 Aug 28 00:56:36.224696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 145/282] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:35 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:37.245535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5bf867e5-8a44-4760-bfc7-fa581ac6d82d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:37.246136 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5bf867e5-8a44-4760-bfc7-fa581ac6d82d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:37.253391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5bf867e5-8a44-4760-bfc7-fa581ac6d82d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:37.253877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5bf867e5-8a44-4760-bfc7-fa581ac6d82d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:37.254303 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5bf867e5-8a44-4760-bfc7-fa581ac6d82d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:37.254996 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5bf867e5-8a44-4760-bfc7-fa581ac6d82d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:37.255451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5bf867e5-8a44-4760-bfc7-fa581ac6d82d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:37.255873 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5bf867e5-8a44-4760-bfc7-fa581ac6d82d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:37.412156 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5bf867e5-8a44-4760-bfc7-fa581ac6d82d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:37.420044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5bf867e5-8a44-4760-bfc7-fa581ac6d82d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.178348 Aug 28 00:56:37.420615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 138/283] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:37 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:38.440711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f4482dd2-ee99-4f42-9f7c-1a970a5eeb5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:38.441380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f4482dd2-ee99-4f42-9f7c-1a970a5eeb5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:38.447638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f4482dd2-ee99-4f42-9f7c-1a970a5eeb5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:38.448103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f4482dd2-ee99-4f42-9f7c-1a970a5eeb5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:38.448895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f4482dd2-ee99-4f42-9f7c-1a970a5eeb5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:38.449635 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f4482dd2-ee99-4f42-9f7c-1a970a5eeb5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:38.450130 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f4482dd2-ee99-4f42-9f7c-1a970a5eeb5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:38.450534 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f4482dd2-ee99-4f42-9f7c-1a970a5eeb5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:38.587310 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f4482dd2-ee99-4f42-9f7c-1a970a5eeb5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:38.597962 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f4482dd2-ee99-4f42-9f7c-1a970a5eeb5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.160899 Aug 28 00:56:38.598563 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 146/284] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:38 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:39.623485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f3307a6b-75a7-4727-befa-a71a058ccfdf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:39.624088 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f3307a6b-75a7-4727-befa-a71a058ccfdf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:39.631331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f3307a6b-75a7-4727-befa-a71a058ccfdf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:39.631819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f3307a6b-75a7-4727-befa-a71a058ccfdf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:39.632294 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f3307a6b-75a7-4727-befa-a71a058ccfdf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:39.632992 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f3307a6b-75a7-4727-befa-a71a058ccfdf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:39.633462 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f3307a6b-75a7-4727-befa-a71a058ccfdf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:39.633895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f3307a6b-75a7-4727-befa-a71a058ccfdf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:39.790198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f3307a6b-75a7-4727-befa-a71a058ccfdf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:39.799433 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f3307a6b-75a7-4727-befa-a71a058ccfdf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.179957 Aug 28 00:56:39.800043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 139/285] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:39 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:40.821067 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8b6b7b93-b4ed-4836-b3e6-51c11dedd7b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:40.821831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8b6b7b93-b4ed-4836-b3e6-51c11dedd7b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:40.828759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b6b7b93-b4ed-4836-b3e6-51c11dedd7b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:40.829226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b6b7b93-b4ed-4836-b3e6-51c11dedd7b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:40.829656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b6b7b93-b4ed-4836-b3e6-51c11dedd7b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:40.830362 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b6b7b93-b4ed-4836-b3e6-51c11dedd7b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:40.832646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b6b7b93-b4ed-4836-b3e6-51c11dedd7b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:40.833186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b6b7b93-b4ed-4836-b3e6-51c11dedd7b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:40.979328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8b6b7b93-b4ed-4836-b3e6-51c11dedd7b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:40.988077 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8b6b7b93-b4ed-4836-b3e6-51c11dedd7b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.171247 Aug 28 00:56:40.988657 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 147/286] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:40 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:42.010400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1e233186-3a58-4436-a792-b6c9f3177169 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:42.010765 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1e233186-3a58-4436-a792-b6c9f3177169 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:42.012791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=36 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:56:42.018464 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1e233186-3a58-4436-a792-b6c9f3177169 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:42.018948 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1e233186-3a58-4436-a792-b6c9f3177169 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:42.019378 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1e233186-3a58-4436-a792-b6c9f3177169 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:42.020072 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1e233186-3a58-4436-a792-b6c9f3177169 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:42.020514 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1e233186-3a58-4436-a792-b6c9f3177169 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:42.020930 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1e233186-3a58-4436-a792-b6c9f3177169 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:42.023700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:56:42.164731 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1e233186-3a58-4436-a792-b6c9f3177169 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:42.172986 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1e233186-3a58-4436-a792-b6c9f3177169 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.167115 Aug 28 00:56:42.173558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 140/287] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:42 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:43.193935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c3fb97d8-2707-4d5d-b3d7-75bb0400a3ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:43.194380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c3fb97d8-2707-4d5d-b3d7-75bb0400a3ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:43.201307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3fb97d8-2707-4d5d-b3d7-75bb0400a3ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:43.201794 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3fb97d8-2707-4d5d-b3d7-75bb0400a3ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:43.202216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3fb97d8-2707-4d5d-b3d7-75bb0400a3ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:43.202935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3fb97d8-2707-4d5d-b3d7-75bb0400a3ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:43.203465 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3fb97d8-2707-4d5d-b3d7-75bb0400a3ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:43.203895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3fb97d8-2707-4d5d-b3d7-75bb0400a3ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:43.340160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c3fb97d8-2707-4d5d-b3d7-75bb0400a3ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:43.348395 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c3fb97d8-2707-4d5d-b3d7-75bb0400a3ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.158347 Aug 28 00:56:43.349146 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 148/288] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:43 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:44.369701 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-03507d8d-ee3e-4c58-8739-c95c67383f0f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:44.370265 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-03507d8d-ee3e-4c58-8739-c95c67383f0f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:44.378100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-03507d8d-ee3e-4c58-8739-c95c67383f0f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:44.378569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-03507d8d-ee3e-4c58-8739-c95c67383f0f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:44.379036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-03507d8d-ee3e-4c58-8739-c95c67383f0f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:44.379816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-03507d8d-ee3e-4c58-8739-c95c67383f0f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:44.380254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-03507d8d-ee3e-4c58-8739-c95c67383f0f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:44.380728 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-03507d8d-ee3e-4c58-8739-c95c67383f0f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:44.531222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-03507d8d-ee3e-4c58-8739-c95c67383f0f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:44.540293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-03507d8d-ee3e-4c58-8739-c95c67383f0f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.174630 Aug 28 00:56:44.540900 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 141/289] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:44 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:45.562085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5711ad87-4e81-4313-b03e-022270644dbc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:45.563058 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5711ad87-4e81-4313-b03e-022270644dbc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:45.570804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5711ad87-4e81-4313-b03e-022270644dbc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:45.571383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5711ad87-4e81-4313-b03e-022270644dbc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:45.571870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5711ad87-4e81-4313-b03e-022270644dbc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:45.572821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5711ad87-4e81-4313-b03e-022270644dbc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:45.573826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5711ad87-4e81-4313-b03e-022270644dbc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:45.574434 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5711ad87-4e81-4313-b03e-022270644dbc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:45.751300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5711ad87-4e81-4313-b03e-022270644dbc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:45.772234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5711ad87-4e81-4313-b03e-022270644dbc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.213866 Aug 28 00:56:45.773205 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 149/290] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:45 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:46.807967 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-966ba617-125a-4835-bbf0-75099de02667 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:46.808926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-966ba617-125a-4835-bbf0-75099de02667 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:46.817028 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-966ba617-125a-4835-bbf0-75099de02667 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:46.817518 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-966ba617-125a-4835-bbf0-75099de02667 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:46.817945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-966ba617-125a-4835-bbf0-75099de02667 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:46.818642 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-966ba617-125a-4835-bbf0-75099de02667 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:46.819088 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-966ba617-125a-4835-bbf0-75099de02667 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:46.819503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-966ba617-125a-4835-bbf0-75099de02667 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:46.967496 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-966ba617-125a-4835-bbf0-75099de02667 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:46.976198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-966ba617-125a-4835-bbf0-75099de02667 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.172417 Aug 28 00:56:46.976774 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 142/291] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:46 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:48.006667 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fa8f6bbf-6b2b-4be4-b57b-f1293c9b47cc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:48.007220 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fa8f6bbf-6b2b-4be4-b57b-f1293c9b47cc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:48.016810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fa8f6bbf-6b2b-4be4-b57b-f1293c9b47cc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:48.017682 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fa8f6bbf-6b2b-4be4-b57b-f1293c9b47cc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:48.018454 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fa8f6bbf-6b2b-4be4-b57b-f1293c9b47cc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:48.019430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fa8f6bbf-6b2b-4be4-b57b-f1293c9b47cc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:48.019889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fa8f6bbf-6b2b-4be4-b57b-f1293c9b47cc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:48.020316 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fa8f6bbf-6b2b-4be4-b57b-f1293c9b47cc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:48.162515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fa8f6bbf-6b2b-4be4-b57b-f1293c9b47cc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:48.171894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fa8f6bbf-6b2b-4be4-b57b-f1293c9b47cc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1523 microversion: 2.1 time: 0.169314 Aug 28 00:56:48.172582 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 150/292] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:48 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1523 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:49.194589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-54e7656c-a560-4511-8f6c-75e6212a6dc5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:49.195167 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-54e7656c-a560-4511-8f6c-75e6212a6dc5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:49.202232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54e7656c-a560-4511-8f6c-75e6212a6dc5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:49.202617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54e7656c-a560-4511-8f6c-75e6212a6dc5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:49.203036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54e7656c-a560-4511-8f6c-75e6212a6dc5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:49.203771 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54e7656c-a560-4511-8f6c-75e6212a6dc5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:49.204209 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54e7656c-a560-4511-8f6c-75e6212a6dc5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:49.204619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54e7656c-a560-4511-8f6c-75e6212a6dc5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:49.342145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-54e7656c-a560-4511-8f6c-75e6212a6dc5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:49.349978 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-54e7656c-a560-4511-8f6c-75e6212a6dc5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1503 microversion: 2.1 time: 0.159345 Aug 28 00:56:49.350547 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 143/293] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:49 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1503 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:49.370969 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f719a853-707f-4262-9677-8c167d8264eb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:49.371515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f719a853-707f-4262-9677-8c167d8264eb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:49.377981 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f719a853-707f-4262-9677-8c167d8264eb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:49.378460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f719a853-707f-4262-9677-8c167d8264eb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:49.378889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f719a853-707f-4262-9677-8c167d8264eb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:49.379579 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f719a853-707f-4262-9677-8c167d8264eb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:49.380061 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f719a853-707f-4262-9677-8c167d8264eb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:49.380475 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f719a853-707f-4262-9677-8c167d8264eb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:49.518557 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f719a853-707f-4262-9677-8c167d8264eb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:49.526863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f719a853-707f-4262-9677-8c167d8264eb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1515 microversion: 2.1 time: 0.160333 Aug 28 00:56:49.527437 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 151/294] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:49 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1515 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:55.213293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b747cb7c-4caf-43c0-9590-f3e741ad67f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:56:55.216136 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b747cb7c-4caf-43c0-9590-f3e741ad67f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:55.224017 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b747cb7c-4caf-43c0-9590-f3e741ad67f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:55.224745 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b747cb7c-4caf-43c0-9590-f3e741ad67f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:55.225764 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b747cb7c-4caf-43c0-9590-f3e741ad67f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:55.226738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b747cb7c-4caf-43c0-9590-f3e741ad67f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:55.227416 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b747cb7c-4caf-43c0-9590-f3e741ad67f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:55.227803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b747cb7c-4caf-43c0-9590-f3e741ad67f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:55.254063 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.objects.instance [None req-b747cb7c-4caf-43c0-9590-f3e741ad67f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lazy-loading 'flavor' on Instance uuid f5f7fe76-9095-4d8d-967f-be0009648de9 {{(pid=120524) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 00:56:55.401688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b747cb7c-4caf-43c0-9590-f3e741ad67f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "POST /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9/action" status: 200 len: 32 microversion: 2.1 time: 0.192512 Aug 28 00:56:55.403038 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 144/295] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Aug 28 00:56:55 2022] POST /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9/action => generated 32 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 28 00:56:55.415613 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-93ed41ec-0fde-4359-b739-1c75c591c693 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:55.416694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-93ed41ec-0fde-4359-b739-1c75c591c693 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:55.428480 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93ed41ec-0fde-4359-b739-1c75c591c693 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:55.429287 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93ed41ec-0fde-4359-b739-1c75c591c693 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:55.430050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93ed41ec-0fde-4359-b739-1c75c591c693 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:55.431164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93ed41ec-0fde-4359-b739-1c75c591c693 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:55.431887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93ed41ec-0fde-4359-b739-1c75c591c693 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:55.432608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93ed41ec-0fde-4359-b739-1c75c591c693 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:55.871656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-93ed41ec-0fde-4359-b739-1c75c591c693 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:55.879607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-93ed41ec-0fde-4359-b739-1c75c591c693 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1521 microversion: 2.1 time: 0.470615 Aug 28 00:56:55.880172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 152/296] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:55 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1521 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:56.900400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5448024e-15e8-461d-9486-26152aefe0a2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:56.901203 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5448024e-15e8-461d-9486-26152aefe0a2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:56.911400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5448024e-15e8-461d-9486-26152aefe0a2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:56.911895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5448024e-15e8-461d-9486-26152aefe0a2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:56.912350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5448024e-15e8-461d-9486-26152aefe0a2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:56.913142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5448024e-15e8-461d-9486-26152aefe0a2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:56.913875 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5448024e-15e8-461d-9486-26152aefe0a2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:56.914474 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5448024e-15e8-461d-9486-26152aefe0a2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:57.221916 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5448024e-15e8-461d-9486-26152aefe0a2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:57.231191 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5448024e-15e8-461d-9486-26152aefe0a2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1521 microversion: 2.1 time: 0.334939 Aug 28 00:56:57.231905 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 145/297] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:56 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1521 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:58.254863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-01600803-c1a4-45ed-b818-9d37d784cb68 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:58.255604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-01600803-c1a4-45ed-b818-9d37d784cb68 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:58.262558 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-01600803-c1a4-45ed-b818-9d37d784cb68 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:58.263037 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-01600803-c1a4-45ed-b818-9d37d784cb68 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:58.263459 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-01600803-c1a4-45ed-b818-9d37d784cb68 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:58.264172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-01600803-c1a4-45ed-b818-9d37d784cb68 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:58.264617 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-01600803-c1a4-45ed-b818-9d37d784cb68 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:58.265106 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-01600803-c1a4-45ed-b818-9d37d784cb68 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:58.267928 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=72 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:56:58.414406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-01600803-c1a4-45ed-b818-9d37d784cb68 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:58.422327 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-01600803-c1a4-45ed-b818-9d37d784cb68 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1521 microversion: 2.1 time: 0.171616 Aug 28 00:56:58.422903 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 153/298] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:58 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1521 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:56:59.445632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ea607be8-3ccd-41f0-9bbe-f50f28bfab7a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:56:59.446271 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ea607be8-3ccd-41f0-9bbe-f50f28bfab7a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:56:59.454009 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea607be8-3ccd-41f0-9bbe-f50f28bfab7a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:59.454516 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea607be8-3ccd-41f0-9bbe-f50f28bfab7a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:59.454939 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea607be8-3ccd-41f0-9bbe-f50f28bfab7a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:59.455643 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea607be8-3ccd-41f0-9bbe-f50f28bfab7a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:56:59.456098 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea607be8-3ccd-41f0-9bbe-f50f28bfab7a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:56:59.456517 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea607be8-3ccd-41f0-9bbe-f50f28bfab7a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:56:59.713626 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ea607be8-3ccd-41f0-9bbe-f50f28bfab7a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:56:59.722800 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ea607be8-3ccd-41f0-9bbe-f50f28bfab7a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1521 microversion: 2.1 time: 0.280946 Aug 28 00:56:59.723389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 146/299] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:56:59 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1521 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:00.746080 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4bff2af8-09ce-484e-bc02-98898e972d46 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:00.746697 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4bff2af8-09ce-484e-bc02-98898e972d46 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:00.754239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bff2af8-09ce-484e-bc02-98898e972d46 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:00.754944 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bff2af8-09ce-484e-bc02-98898e972d46 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:00.755698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bff2af8-09ce-484e-bc02-98898e972d46 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:00.756615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bff2af8-09ce-484e-bc02-98898e972d46 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:00.757073 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bff2af8-09ce-484e-bc02-98898e972d46 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:00.757921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bff2af8-09ce-484e-bc02-98898e972d46 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:00.906903 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4bff2af8-09ce-484e-bc02-98898e972d46 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:00.915118 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4bff2af8-09ce-484e-bc02-98898e972d46 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1521 microversion: 2.1 time: 0.174777 Aug 28 00:57:00.915680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 154/300] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:00 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1521 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:01.936382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f08007e1-86ea-42d3-bdf5-5ce09c258fc1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:01.937043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f08007e1-86ea-42d3-bdf5-5ce09c258fc1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:01.943687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f08007e1-86ea-42d3-bdf5-5ce09c258fc1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:01.944104 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f08007e1-86ea-42d3-bdf5-5ce09c258fc1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:01.944536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f08007e1-86ea-42d3-bdf5-5ce09c258fc1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:01.945265 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f08007e1-86ea-42d3-bdf5-5ce09c258fc1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:01.945716 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f08007e1-86ea-42d3-bdf5-5ce09c258fc1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:01.946137 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f08007e1-86ea-42d3-bdf5-5ce09c258fc1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:02.083520 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f08007e1-86ea-42d3-bdf5-5ce09c258fc1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:02.092602 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f08007e1-86ea-42d3-bdf5-5ce09c258fc1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1521 microversion: 2.1 time: 0.160026 Aug 28 00:57:02.093429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 147/301] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:01 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1521 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:02.929119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e6043c8f-f7dd-4bf7-a03b-1fee92235afa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1511977602", "imageRef": "2c2585b2-bde5-4652-aed7-9f53b1bd5d94", "flavorRef": "452", "networks": [{"uuid": "7884bb82-c947-4f13-a88b-bdc2fd404591"}]}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:57:02.937387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=24 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:57:02.984519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.network.neutron [None req-e6043c8f-f7dd-4bf7-a03b-1fee92235afa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] validate_networks() for [('7884bb82-c947-4f13-a88b-bdc2fd404591', None, None, None, None, None)] {{(pid=120525) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 28 00:57:03.113608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-96dfd48a-8e66-4af7-9668-afeed3dc7e6d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:03.114883 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-96dfd48a-8e66-4af7-9668-afeed3dc7e6d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:03.121916 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96dfd48a-8e66-4af7-9668-afeed3dc7e6d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:03.122463 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96dfd48a-8e66-4af7-9668-afeed3dc7e6d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:03.122954 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96dfd48a-8e66-4af7-9668-afeed3dc7e6d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:03.123710 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96dfd48a-8e66-4af7-9668-afeed3dc7e6d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:03.124144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96dfd48a-8e66-4af7-9668-afeed3dc7e6d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:03.124667 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96dfd48a-8e66-4af7-9668-afeed3dc7e6d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:03.367054 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-96dfd48a-8e66-4af7-9668-afeed3dc7e6d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:03.376147 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-96dfd48a-8e66-4af7-9668-afeed3dc7e6d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1521 microversion: 2.1 time: 0.266268 Aug 28 00:57:03.376837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 148/302] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:03 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1521 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:03.410670 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-e6043c8f-f7dd-4bf7-a03b-1fee92235afa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Flavor limits 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 28 00:57:03.411082 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-e6043c8f-f7dd-4bf7-a03b-1fee92235afa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Image limits 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 28 00:57:03.411863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-e6043c8f-f7dd-4bf7-a03b-1fee92235afa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Flavor pref 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 28 00:57:03.412270 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-e6043c8f-f7dd-4bf7-a03b-1fee92235afa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Image pref 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 28 00:57:03.412669 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-e6043c8f-f7dd-4bf7-a03b-1fee92235afa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 28 00:57:03.427494 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6043c8f-f7dd-4bf7-a03b-1fee92235afa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:03.427976 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6043c8f-f7dd-4bf7-a03b-1fee92235afa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:03.428573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6043c8f-f7dd-4bf7-a03b-1fee92235afa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:03.429311 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6043c8f-f7dd-4bf7-a03b-1fee92235afa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:03.429755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6043c8f-f7dd-4bf7-a03b-1fee92235afa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:03.430181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6043c8f-f7dd-4bf7-a03b-1fee92235afa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:03.447894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-e6043c8f-f7dd-4bf7-a03b-1fee92235afa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Getting quotas for project 315d6d72ee5844edb203b740ab2d08ce. Resources: {'ram', 'instances', 'cores'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 00:57:03.452870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-e6043c8f-f7dd-4bf7-a03b-1fee92235afa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Getting quotas for user 59b04859082547149bf2776aefa63b43 and project 315d6d72ee5844edb203b740ab2d08ce. Resources: {'ram', 'instances', 'cores'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 00:57:03.466727 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e6043c8f-f7dd-4bf7-a03b-1fee92235afa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Going to run 1 instances... {{(pid=120525) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 28 00:57:03.466832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e6043c8f-f7dd-4bf7-a03b-1fee92235afa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 1964f323-9efc-456e-9a2a-981a0878e3e2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2c2585b2-bde5-4652-aed7-9f53b1bd5d94',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120525) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 28 00:57:03.517363 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e6043c8f-f7dd-4bf7-a03b-1fee92235afa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.591892 Aug 28 00:57:03.518690 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 155/303] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Aug 28 00:57:02 2022] POST /compute/v2.1/servers => generated 380 bytes in 594 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 28 00:57:03.529259 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3486a067-775b-4062-b608-24cec685023c tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:03.529903 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3486a067-775b-4062-b608-24cec685023c tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 1964f323-9efc-456e-9a2a-981a0878e3e2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:03.589781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3486a067-775b-4062-b608-24cec685023c tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:03.598636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3486a067-775b-4062-b608-24cec685023c tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2" status: 200 len: 1214 microversion: 2.1 time: 0.073114 Aug 28 00:57:03.600954 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 149/304] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:03 2022] GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2 => generated 1214 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:04.399079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-476ef2e6-a975-46c8-94ce-56abd884b822 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:04.399688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-476ef2e6-a975-46c8-94ce-56abd884b822 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:04.407831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-476ef2e6-a975-46c8-94ce-56abd884b822 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:04.408595 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-476ef2e6-a975-46c8-94ce-56abd884b822 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:04.409404 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-476ef2e6-a975-46c8-94ce-56abd884b822 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:04.410195 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-476ef2e6-a975-46c8-94ce-56abd884b822 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:04.410855 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-476ef2e6-a975-46c8-94ce-56abd884b822 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:04.411493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-476ef2e6-a975-46c8-94ce-56abd884b822 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:04.621944 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-aa18f92c-76ce-4b83-9ea3-b0f2ce5ab1cb tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:04.622561 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-aa18f92c-76ce-4b83-9ea3-b0f2ce5ab1cb tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 1964f323-9efc-456e-9a2a-981a0878e3e2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:04.630500 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aa18f92c-76ce-4b83-9ea3-b0f2ce5ab1cb tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:04.630996 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aa18f92c-76ce-4b83-9ea3-b0f2ce5ab1cb tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:04.631423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aa18f92c-76ce-4b83-9ea3-b0f2ce5ab1cb tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:04.632274 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aa18f92c-76ce-4b83-9ea3-b0f2ce5ab1cb tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:04.632721 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aa18f92c-76ce-4b83-9ea3-b0f2ce5ab1cb tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:04.633205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aa18f92c-76ce-4b83-9ea3-b0f2ce5ab1cb tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:04.663934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-476ef2e6-a975-46c8-94ce-56abd884b822 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:04.672385 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-476ef2e6-a975-46c8-94ce-56abd884b822 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1521 microversion: 2.1 time: 0.277734 Aug 28 00:57:04.672748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 156/305] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:04 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1521 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:04.716400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-aa18f92c-76ce-4b83-9ea3-b0f2ce5ab1cb tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:04.725313 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-aa18f92c-76ce-4b83-9ea3-b0f2ce5ab1cb tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2" status: 200 len: 1266 microversion: 2.1 time: 0.107068 Aug 28 00:57:04.725751 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 150/306] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:04 2022] GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2 => generated 1266 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:05.693443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ea8f66eb-6958-4c07-a249-2916d1dda98d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:05.694221 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ea8f66eb-6958-4c07-a249-2916d1dda98d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:05.701493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea8f66eb-6958-4c07-a249-2916d1dda98d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:05.702247 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea8f66eb-6958-4c07-a249-2916d1dda98d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:05.702773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea8f66eb-6958-4c07-a249-2916d1dda98d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:05.703466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea8f66eb-6958-4c07-a249-2916d1dda98d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:05.703965 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea8f66eb-6958-4c07-a249-2916d1dda98d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:05.704384 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea8f66eb-6958-4c07-a249-2916d1dda98d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:05.748170 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b49c4cb8-ce03-4929-809f-f4eec22fead0 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:05.748873 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b49c4cb8-ce03-4929-809f-f4eec22fead0 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 1964f323-9efc-456e-9a2a-981a0878e3e2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:05.756221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b49c4cb8-ce03-4929-809f-f4eec22fead0 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:05.756812 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b49c4cb8-ce03-4929-809f-f4eec22fead0 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:05.757345 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b49c4cb8-ce03-4929-809f-f4eec22fead0 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:05.758047 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b49c4cb8-ce03-4929-809f-f4eec22fead0 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:05.758496 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b49c4cb8-ce03-4929-809f-f4eec22fead0 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:05.758918 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b49c4cb8-ce03-4929-809f-f4eec22fead0 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:06.037479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ea8f66eb-6958-4c07-a249-2916d1dda98d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:06.063393 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ea8f66eb-6958-4c07-a249-2916d1dda98d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1521 microversion: 2.1 time: 0.372522 Aug 28 00:57:06.064938 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 157/307] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:05 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1521 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:57:06.074705 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b49c4cb8-ce03-4929-809f-f4eec22fead0 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:06.084436 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b49c4cb8-ce03-4929-809f-f4eec22fead0 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2" status: 200 len: 1314 microversion: 2.1 time: 0.339883 Aug 28 00:57:06.093984 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 151/308] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:05 2022] GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2 => generated 1314 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:06.426211 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-ce5d640d-984c-40e3-be09-384bfc01655c req-7032d253-8c65-45fd-9787-98e117c87659 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1964f323-9efc-456e-9a2a-981a0878e3e2", "tag": "ea28f43d-6b9d-440f-856a-e133794342e6"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:57:06.440995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-ce5d640d-984c-40e3-be09-384bfc01655c req-7032d253-8c65-45fd-9787-98e117c87659 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:06.441376 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-ce5d640d-984c-40e3-be09-384bfc01655c req-7032d253-8c65-45fd-9787-98e117c87659 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:06.441808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-ce5d640d-984c-40e3-be09-384bfc01655c req-7032d253-8c65-45fd-9787-98e117c87659 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:06.455015 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-ce5d640d-984c-40e3-be09-384bfc01655c req-7032d253-8c65-45fd-9787-98e117c87659 service nova] Creating event network-changed:ea28f43d-6b9d-440f-856a-e133794342e6 for instance 1964f323-9efc-456e-9a2a-981a0878e3e2 on hv-cinder-maste Aug 28 00:57:06.460816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-ce5d640d-984c-40e3-be09-384bfc01655c req-7032d253-8c65-45fd-9787-98e117c87659 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044793 Aug 28 00:57:06.461851 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 158/309] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:57:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:57:07.086390 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c59700a1-63a6-4653-9526-ca4480bae0f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:07.086949 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c59700a1-63a6-4653-9526-ca4480bae0f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:07.093955 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c59700a1-63a6-4653-9526-ca4480bae0f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:07.094441 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c59700a1-63a6-4653-9526-ca4480bae0f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:07.094866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c59700a1-63a6-4653-9526-ca4480bae0f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:07.095814 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c59700a1-63a6-4653-9526-ca4480bae0f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:07.096351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c59700a1-63a6-4653-9526-ca4480bae0f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:07.096860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c59700a1-63a6-4653-9526-ca4480bae0f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:07.113806 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-617a403e-a741-46b9-8bdb-32a7a19e8682 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:07.115095 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-617a403e-a741-46b9-8bdb-32a7a19e8682 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 1964f323-9efc-456e-9a2a-981a0878e3e2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:07.124625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-617a403e-a741-46b9-8bdb-32a7a19e8682 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:07.125128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-617a403e-a741-46b9-8bdb-32a7a19e8682 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:07.125519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-617a403e-a741-46b9-8bdb-32a7a19e8682 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:07.126915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-617a403e-a741-46b9-8bdb-32a7a19e8682 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:07.127270 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-617a403e-a741-46b9-8bdb-32a7a19e8682 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:07.127968 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-617a403e-a741-46b9-8bdb-32a7a19e8682 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:07.303999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c59700a1-63a6-4653-9526-ca4480bae0f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:07.312335 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c59700a1-63a6-4653-9526-ca4480bae0f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1521 microversion: 2.1 time: 0.229594 Aug 28 00:57:07.312920 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 152/310] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:07 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1521 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:07.329105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-617a403e-a741-46b9-8bdb-32a7a19e8682 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:07.336944 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-617a403e-a741-46b9-8bdb-32a7a19e8682 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2" status: 200 len: 1314 microversion: 2.1 time: 0.232031 Aug 28 00:57:07.337839 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 159/311] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:07 2022] GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2 => generated 1314 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:08.334475 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9d6e5874-ee82-4bbb-9ea8-0d798c3da805 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:08.335080 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9d6e5874-ee82-4bbb-9ea8-0d798c3da805 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:08.342472 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d6e5874-ee82-4bbb-9ea8-0d798c3da805 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:08.342973 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d6e5874-ee82-4bbb-9ea8-0d798c3da805 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:08.343518 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d6e5874-ee82-4bbb-9ea8-0d798c3da805 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:08.344243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d6e5874-ee82-4bbb-9ea8-0d798c3da805 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:08.344689 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d6e5874-ee82-4bbb-9ea8-0d798c3da805 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:08.345134 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d6e5874-ee82-4bbb-9ea8-0d798c3da805 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:08.359785 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-93429b24-ef27-413d-a67d-c97fd2b1bd09 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:08.360573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-93429b24-ef27-413d-a67d-c97fd2b1bd09 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 1964f323-9efc-456e-9a2a-981a0878e3e2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:08.367521 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93429b24-ef27-413d-a67d-c97fd2b1bd09 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:08.368428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93429b24-ef27-413d-a67d-c97fd2b1bd09 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:08.369202 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93429b24-ef27-413d-a67d-c97fd2b1bd09 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:08.370197 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93429b24-ef27-413d-a67d-c97fd2b1bd09 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:08.370831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93429b24-ef27-413d-a67d-c97fd2b1bd09 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:08.371411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93429b24-ef27-413d-a67d-c97fd2b1bd09 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:08.572408 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9d6e5874-ee82-4bbb-9ea8-0d798c3da805 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:08.583579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9d6e5874-ee82-4bbb-9ea8-0d798c3da805 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1521 microversion: 2.1 time: 0.252903 Aug 28 00:57:08.584639 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 153/312] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:08 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1521 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:08.590346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-93429b24-ef27-413d-a67d-c97fd2b1bd09 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:08.600183 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-93429b24-ef27-413d-a67d-c97fd2b1bd09 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2" status: 200 len: 1314 microversion: 2.1 time: 0.244615 Aug 28 00:57:08.600758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 160/313] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:08 2022] GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2 => generated 1314 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:09.606903 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-23919d83-0a02-45da-b3cb-870b52936b41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:09.607668 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-23919d83-0a02-45da-b3cb-870b52936b41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:09.616410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23919d83-0a02-45da-b3cb-870b52936b41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:09.622027 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1851de2e-8477-42cb-b2c9-01c81b57f041 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:09.622694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1851de2e-8477-42cb-b2c9-01c81b57f041 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 1964f323-9efc-456e-9a2a-981a0878e3e2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:09.625697 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23919d83-0a02-45da-b3cb-870b52936b41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:09.629039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23919d83-0a02-45da-b3cb-870b52936b41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:09.629881 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23919d83-0a02-45da-b3cb-870b52936b41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:09.631954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1851de2e-8477-42cb-b2c9-01c81b57f041 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:09.632576 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1851de2e-8477-42cb-b2c9-01c81b57f041 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:09.633140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1851de2e-8477-42cb-b2c9-01c81b57f041 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:09.633914 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1851de2e-8477-42cb-b2c9-01c81b57f041 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:09.634459 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1851de2e-8477-42cb-b2c9-01c81b57f041 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:09.634948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1851de2e-8477-42cb-b2c9-01c81b57f041 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:09.639911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23919d83-0a02-45da-b3cb-870b52936b41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:09.640595 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23919d83-0a02-45da-b3cb-870b52936b41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:09.888494 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-23919d83-0a02-45da-b3cb-870b52936b41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:09.898750 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1851de2e-8477-42cb-b2c9-01c81b57f041 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:09.899070 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-23919d83-0a02-45da-b3cb-870b52936b41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1521 microversion: 2.1 time: 0.296564 Aug 28 00:57:09.900113 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 154/314] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:09 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1521 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:09.919896 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1851de2e-8477-42cb-b2c9-01c81b57f041 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2" status: 200 len: 1314 microversion: 2.1 time: 0.301763 Aug 28 00:57:09.922094 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 161/315] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:09 2022] GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2 => generated 1314 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:57:10.929816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4e20e960-b63f-4424-a9b9-abf1bbbac929 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:10.931079 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4e20e960-b63f-4424-a9b9-abf1bbbac929 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:10.942610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e20e960-b63f-4424-a9b9-abf1bbbac929 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:10.943744 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e20e960-b63f-4424-a9b9-abf1bbbac929 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:10.944291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e20e960-b63f-4424-a9b9-abf1bbbac929 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:10.946943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e20e960-b63f-4424-a9b9-abf1bbbac929 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:10.947557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e20e960-b63f-4424-a9b9-abf1bbbac929 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:10.948430 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e20e960-b63f-4424-a9b9-abf1bbbac929 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:10.952409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-eb44350d-d9c1-42bf-ab98-107094f93693 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:10.953863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-eb44350d-d9c1-42bf-ab98-107094f93693 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 1964f323-9efc-456e-9a2a-981a0878e3e2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:10.973229 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb44350d-d9c1-42bf-ab98-107094f93693 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:10.977426 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb44350d-d9c1-42bf-ab98-107094f93693 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:10.983770 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb44350d-d9c1-42bf-ab98-107094f93693 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:10.985960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb44350d-d9c1-42bf-ab98-107094f93693 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:10.986457 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb44350d-d9c1-42bf-ab98-107094f93693 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:10.986902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb44350d-d9c1-42bf-ab98-107094f93693 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:11.807620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-eb44350d-d9c1-42bf-ab98-107094f93693 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:11.820701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-eb44350d-d9c1-42bf-ab98-107094f93693 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2" status: 200 len: 1314 microversion: 2.1 time: 0.873911 Aug 28 00:57:11.832990 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 162/316] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:10 2022] GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2 => generated 1314 bytes in 875 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:11.844421 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4e20e960-b63f-4424-a9b9-abf1bbbac929 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:11.848829 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:57:11.858072 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4e20e960-b63f-4424-a9b9-abf1bbbac929 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1521 microversion: 2.1 time: 0.932558 Aug 28 00:57:11.859443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 155/317] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:10 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1521 bytes in 934 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:12.402269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-6e9fd328-b50d-41c9-8961-ada196e2d993 req-bc0ec97a-ec9b-4ec9-ac79-c1dc744631fd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1964f323-9efc-456e-9a2a-981a0878e3e2", "name": "network-vif-plugged", "status": "completed", "tag": "ea28f43d-6b9d-440f-856a-e133794342e6"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:57:12.417561 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-6e9fd328-b50d-41c9-8961-ada196e2d993 req-bc0ec97a-ec9b-4ec9-ac79-c1dc744631fd service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:12.418382 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-6e9fd328-b50d-41c9-8961-ada196e2d993 req-bc0ec97a-ec9b-4ec9-ac79-c1dc744631fd service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:12.419063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-6e9fd328-b50d-41c9-8961-ada196e2d993 req-bc0ec97a-ec9b-4ec9-ac79-c1dc744631fd service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:12.432468 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-6e9fd328-b50d-41c9-8961-ada196e2d993 req-bc0ec97a-ec9b-4ec9-ac79-c1dc744631fd service nova] Creating event network-vif-plugged:ea28f43d-6b9d-440f-856a-e133794342e6 for instance 1964f323-9efc-456e-9a2a-981a0878e3e2 on hv-cinder-maste Aug 28 00:57:12.438764 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-6e9fd328-b50d-41c9-8961-ada196e2d993 req-bc0ec97a-ec9b-4ec9-ac79-c1dc744631fd service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041443 Aug 28 00:57:12.439313 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 163/318] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:57:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:57:12.845751 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f4c14727-4396-4ec2-b62a-cc6f9bd9ad96 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:12.846524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f4c14727-4396-4ec2-b62a-cc6f9bd9ad96 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 1964f323-9efc-456e-9a2a-981a0878e3e2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:12.854369 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=34 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:57:12.880435 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4c14727-4396-4ec2-b62a-cc6f9bd9ad96 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:12.882174 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4c14727-4396-4ec2-b62a-cc6f9bd9ad96 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:12.882745 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4c14727-4396-4ec2-b62a-cc6f9bd9ad96 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:12.883437 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1db35e64-86ce-40c8-887b-6a82743002e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:12.883696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4c14727-4396-4ec2-b62a-cc6f9bd9ad96 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:12.884169 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1db35e64-86ce-40c8-887b-6a82743002e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:12.884336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4c14727-4396-4ec2-b62a-cc6f9bd9ad96 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:12.884857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4c14727-4396-4ec2-b62a-cc6f9bd9ad96 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:12.897512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1db35e64-86ce-40c8-887b-6a82743002e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:12.898278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1db35e64-86ce-40c8-887b-6a82743002e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:12.898773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1db35e64-86ce-40c8-887b-6a82743002e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:12.899429 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1db35e64-86ce-40c8-887b-6a82743002e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:12.900209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1db35e64-86ce-40c8-887b-6a82743002e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:12.900949 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1db35e64-86ce-40c8-887b-6a82743002e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:13.428457 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1db35e64-86ce-40c8-887b-6a82743002e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:13.436520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=2 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:57:13.440793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1db35e64-86ce-40c8-887b-6a82743002e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1521 microversion: 2.1 time: 0.564514 Aug 28 00:57:13.441410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 164/319] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:12 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1521 bytes in 566 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:13.443830 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f4c14727-4396-4ec2-b62a-cc6f9bd9ad96 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:13.462399 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f4c14727-4396-4ec2-b62a-cc6f9bd9ad96 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2" status: 200 len: 1314 microversion: 2.1 time: 0.620472 Aug 28 00:57:13.474155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 156/320] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:12 2022] GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2 => generated 1314 bytes in 633 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:14.485338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0c504819-be05-4876-81b6-2cb3bff2c059 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:14.486054 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0c504819-be05-4876-81b6-2cb3bff2c059 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:14.489566 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=2 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:57:14.491910 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=6 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:57:14.495038 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1566b561-293b-40da-a43e-14cbdd9ac4c1 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:14.495662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1566b561-293b-40da-a43e-14cbdd9ac4c1 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 1964f323-9efc-456e-9a2a-981a0878e3e2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:14.498613 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c504819-be05-4876-81b6-2cb3bff2c059 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:14.501787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c504819-be05-4876-81b6-2cb3bff2c059 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:14.502721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c504819-be05-4876-81b6-2cb3bff2c059 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:14.504136 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c504819-be05-4876-81b6-2cb3bff2c059 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:14.505127 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c504819-be05-4876-81b6-2cb3bff2c059 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:14.505956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c504819-be05-4876-81b6-2cb3bff2c059 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:14.506740 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1566b561-293b-40da-a43e-14cbdd9ac4c1 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:14.508167 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1566b561-293b-40da-a43e-14cbdd9ac4c1 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:14.508903 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1566b561-293b-40da-a43e-14cbdd9ac4c1 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:14.509963 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1566b561-293b-40da-a43e-14cbdd9ac4c1 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:14.510573 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1566b561-293b-40da-a43e-14cbdd9ac4c1 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:14.511158 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1566b561-293b-40da-a43e-14cbdd9ac4c1 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:14.765074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0c504819-be05-4876-81b6-2cb3bff2c059 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:14.768919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1566b561-293b-40da-a43e-14cbdd9ac4c1 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:14.776337 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0c504819-be05-4876-81b6-2cb3bff2c059 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1501 microversion: 2.1 time: 0.296410 Aug 28 00:57:14.777580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1566b561-293b-40da-a43e-14cbdd9ac4c1 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2" status: 200 len: 1314 microversion: 2.1 time: 0.288316 Aug 28 00:57:14.777748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 165/321] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:14 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1501 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:14.788999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 157/322] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:14 2022] GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2 => generated 1314 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:57:14.800459 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e3b8c56a-52aa-4fcd-adeb-53f19980f0dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2c6e542d-3058-4af0-98de-41143582a77e"}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:57:14.803090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e3b8c56a-52aa-4fcd-adeb-53f19980f0dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:14.809428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e3b8c56a-52aa-4fcd-adeb-53f19980f0dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:14.810335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e3b8c56a-52aa-4fcd-adeb-53f19980f0dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:14.810884 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e3b8c56a-52aa-4fcd-adeb-53f19980f0dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:14.812001 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e3b8c56a-52aa-4fcd-adeb-53f19980f0dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:14.812599 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e3b8c56a-52aa-4fcd-adeb-53f19980f0dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:14.813308 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e3b8c56a-52aa-4fcd-adeb-53f19980f0dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:14.850916 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-e3b8c56a-52aa-4fcd-adeb-53f19980f0dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] HTTP exception thrown: Cannot 'attach_volume' instance f5f7fe76-9095-4d8d-967f-be0009648de9 while it is in vm_state rescued Aug 28 00:57:14.854090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e3b8c56a-52aa-4fcd-adeb-53f19980f0dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Request method failure captured: Aug 28 00:57:14.854090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: POST /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9/os-volume_attachments HTTP/1.1 Aug 28 00:57:14.854090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 00:57:14.854090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 00:57:14.854090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 00:57:14.854090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 74 Aug 28 00:57:14.854090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 00:57:14.854090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 00:57:14.854090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 00:57:14.854090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 00:57:14.854090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCrzWcuiPK_0ynCrragXxXd_fRw7lHCmbNVMAqr4NhyopxGAZIiQrt0m3y3NawnXLWqyOVM4YbBc-yLRrJ4cXICU0MsT01bsu6VzxaCxGd6JRCChv8LlCV7kFwwyy7Ffwd9JpYuFLguuT-xhEcVlfMIZxgSYB22XTBbtvAeHM_AtK1Xg Aug 28 00:57:14.854090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 00:57:14.854090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 00:57:14.854090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 00:57:14.854090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 00:57:14.854090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 00:57:14.854090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 00:57:14.854090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 00:57:14.854090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 00:57:14.854090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 00:57:14.854090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 00:57:14.855529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_6f3acee4af62483ab1f2cbf3a47b09e1", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/6f3acee4af62483ab1f2cbf3a47b09e1"}]}] Aug 28 00:57:14.855529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 00:57:14.855529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 00:57:14.855529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 00:57:14.855529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 00:57:14.855529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 00:57:14.855529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 00:57:14.855529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: 1ab2fcb3759045c7a076e5c8222a421d Aug 28 00:57:14.855529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 00:57:14.855529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Aug 28 00:57:14.855529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {"volumeAttachment": {"volumeId": "2c6e542d-3058-4af0-98de-41143582a77e"}} Aug 28 00:57:14.855529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 00:57:14.856307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Cannot 'attach_volume' instance f5f7fe76-9095-4d8d-967f-be0009648de9 while it is in vm_state rescued Aug 28 00:57:14.856307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'server_id': 'f5f7fe76-9095-4d8d-967f-be0009648de9', 'body': {'volumeAttachment': {'volumeId': '2c6e542d-3058-4af0-98de-41143582a77e'}}} Aug 28 00:57:14.856307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 00:57:14.856307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:57:14.856307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Aug 28 00:57:14.856307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Aug 28 00:57:14.856307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Aug 28 00:57:14.856307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 00:57:14.856307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Aug 28 00:57:14.856307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Aug 28 00:57:14.856307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance f5f7fe76-9095-4d8d-967f-be0009648de9 in vm_state rescued. Cannot attach_volume while the instance is in this state. Aug 28 00:57:14.856307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:57:14.856307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:57:14.856307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:57:14.856307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:57:14.856307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:57:14.856307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:57:14.856307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 00:57:14.856307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 00:57:14.858638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 00:57:14.858638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:57:14.858638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 28 00:57:14.858638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 28 00:57:14.858638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 28 00:57:14.858638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 28 00:57:14.858638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 28 00:57:14.858638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 28 00:57:14.858638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Aug 28 00:57:14.858638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Aug 28 00:57:14.858638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Aug 28 00:57:14.858638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Aug 28 00:57:14.858638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance f5f7fe76-9095-4d8d-967f-be0009648de9 while it is in vm_state rescued Aug 28 00:57:14.858638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:57:14.858638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:57:14.858638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:57:14.858638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:57:14.858638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:57:14.858638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:57:14.859984 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 00:57:14.859984 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 00:57:14.859984 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance f5f7fe76-9095-4d8d-967f-be0009648de9 while it is in vm_state rescued Aug 28 00:57:14.859984 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:57:14.862875 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e3b8c56a-52aa-4fcd-adeb-53f19980f0dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Returning 409 to user: Cannot 'attach_volume' instance f5f7fe76-9095-4d8d-967f-be0009648de9 while it is in vm_state rescued {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 00:57:14.863936 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e3b8c56a-52aa-4fcd-adeb-53f19980f0dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "POST /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.068273 Aug 28 00:57:14.864677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 166/323] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Aug 28 00:57:14 2022] POST /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9/os-volume_attachments => generated 152 bytes in 70 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Aug 28 00:57:14.874324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-097bb36b-87f1-4d41-85f3-b2a26e06d1dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:57:14.874866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-097bb36b-87f1-4d41-85f3-b2a26e06d1dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:14.881077 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-097bb36b-87f1-4d41-85f3-b2a26e06d1dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:14.881491 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-097bb36b-87f1-4d41-85f3-b2a26e06d1dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:14.882135 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-097bb36b-87f1-4d41-85f3-b2a26e06d1dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:14.882852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-097bb36b-87f1-4d41-85f3-b2a26e06d1dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:14.883942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-097bb36b-87f1-4d41-85f3-b2a26e06d1dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:14.883942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-097bb36b-87f1-4d41-85f3-b2a26e06d1dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:14.959259 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-097bb36b-87f1-4d41-85f3-b2a26e06d1dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "POST /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9/action" status: 202 len: 0 microversion: 2.1 time: 0.088934 Aug 28 00:57:14.963139 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 158/324] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Aug 28 00:57:14 2022] POST /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9/action => generated 0 bytes in 91 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 00:57:14.968706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b2d62bf3-7e8c-48e5-a09c-e93aa03c2b4c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:14.969711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b2d62bf3-7e8c-48e5-a09c-e93aa03c2b4c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:14.976690 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2d62bf3-7e8c-48e5-a09c-e93aa03c2b4c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:14.977705 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2d62bf3-7e8c-48e5-a09c-e93aa03c2b4c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:14.978267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2d62bf3-7e8c-48e5-a09c-e93aa03c2b4c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:14.979268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2d62bf3-7e8c-48e5-a09c-e93aa03c2b4c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:14.979988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2d62bf3-7e8c-48e5-a09c-e93aa03c2b4c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:14.980670 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2d62bf3-7e8c-48e5-a09c-e93aa03c2b4c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:15.425530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b2d62bf3-7e8c-48e5-a09c-e93aa03c2b4c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:15.435183 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b2d62bf3-7e8c-48e5-a09c-e93aa03c2b4c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1509 microversion: 2.1 time: 0.470534 Aug 28 00:57:15.435766 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 167/325] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:14 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1509 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:15.800285 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c108bb6f-cab1-4bd0-bd3b-afe018537f5b tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:15.801186 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c108bb6f-cab1-4bd0-bd3b-afe018537f5b tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 1964f323-9efc-456e-9a2a-981a0878e3e2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:15.818267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c108bb6f-cab1-4bd0-bd3b-afe018537f5b tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:15.819245 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c108bb6f-cab1-4bd0-bd3b-afe018537f5b tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:15.819772 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c108bb6f-cab1-4bd0-bd3b-afe018537f5b tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:15.820575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c108bb6f-cab1-4bd0-bd3b-afe018537f5b tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:15.821351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c108bb6f-cab1-4bd0-bd3b-afe018537f5b tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:15.821773 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c108bb6f-cab1-4bd0-bd3b-afe018537f5b tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:16.025936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c108bb6f-cab1-4bd0-bd3b-afe018537f5b tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:16.039273 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c108bb6f-cab1-4bd0-bd3b-afe018537f5b tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2" status: 200 len: 1497 microversion: 2.1 time: 0.243754 Aug 28 00:57:16.041317 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 159/326] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:15 2022] GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2 => generated 1497 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:16.465185 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bfd52300-b41b-40ff-968f-53b5bf8833be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:16.466918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bfd52300-b41b-40ff-968f-53b5bf8833be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:16.481580 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bfd52300-b41b-40ff-968f-53b5bf8833be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:16.482083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bfd52300-b41b-40ff-968f-53b5bf8833be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:16.482522 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bfd52300-b41b-40ff-968f-53b5bf8833be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:16.483247 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bfd52300-b41b-40ff-968f-53b5bf8833be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:16.483741 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bfd52300-b41b-40ff-968f-53b5bf8833be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:16.484169 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bfd52300-b41b-40ff-968f-53b5bf8833be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:16.814409 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3eabb17b-0ee6-46e1-8504-80b52595be32 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:16.816123 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3eabb17b-0ee6-46e1-8504-80b52595be32 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 1964f323-9efc-456e-9a2a-981a0878e3e2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:16.832320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3eabb17b-0ee6-46e1-8504-80b52595be32 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:16.833279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3eabb17b-0ee6-46e1-8504-80b52595be32 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:16.845788 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3eabb17b-0ee6-46e1-8504-80b52595be32 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:16.849717 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3eabb17b-0ee6-46e1-8504-80b52595be32 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:16.852611 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3eabb17b-0ee6-46e1-8504-80b52595be32 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:16.857726 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3eabb17b-0ee6-46e1-8504-80b52595be32 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:16.878246 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bfd52300-b41b-40ff-968f-53b5bf8833be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:16.893428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bfd52300-b41b-40ff-968f-53b5bf8833be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1509 microversion: 2.1 time: 0.436754 Aug 28 00:57:16.898029 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3eabb17b-0ee6-46e1-8504-80b52595be32 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 1964f323-9efc-456e-9a2a-981a0878e3e2] Going to try to terminate instance {{(pid=120524) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 28 00:57:16.918565 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 168/327] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:16 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1509 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:17.054302 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3eabb17b-0ee6-46e1-8504-80b52595be32 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "DELETE /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2" status: 204 len: 0 microversion: 2.1 time: 0.245114 Aug 28 00:57:17.055051 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 160/328] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 00:57:16 2022] DELETE /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Aug 28 00:57:17.066443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b79ea3b5-6757-479e-84ce-987ea55a5fee tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:17.068476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b79ea3b5-6757-479e-84ce-987ea55a5fee tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 1964f323-9efc-456e-9a2a-981a0878e3e2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:17.076931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b79ea3b5-6757-479e-84ce-987ea55a5fee tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:17.077716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b79ea3b5-6757-479e-84ce-987ea55a5fee tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:17.078343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b79ea3b5-6757-479e-84ce-987ea55a5fee tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:17.079206 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b79ea3b5-6757-479e-84ce-987ea55a5fee tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:17.079807 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b79ea3b5-6757-479e-84ce-987ea55a5fee tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:17.080374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b79ea3b5-6757-479e-84ce-987ea55a5fee tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:17.388015 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b79ea3b5-6757-479e-84ce-987ea55a5fee tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:17.403572 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b79ea3b5-6757-479e-84ce-987ea55a5fee tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2" status: 200 len: 1503 microversion: 2.1 time: 0.342972 Aug 28 00:57:17.406411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 169/329] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:17 2022] GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2 => generated 1503 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:17.922211 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-95242ae0-f7d2-4e2b-bb0b-c75531d72ce3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:17.922985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-95242ae0-f7d2-4e2b-bb0b-c75531d72ce3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:17.930764 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95242ae0-f7d2-4e2b-bb0b-c75531d72ce3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:17.931426 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95242ae0-f7d2-4e2b-bb0b-c75531d72ce3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:17.931981 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95242ae0-f7d2-4e2b-bb0b-c75531d72ce3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:17.932791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95242ae0-f7d2-4e2b-bb0b-c75531d72ce3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:17.933367 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95242ae0-f7d2-4e2b-bb0b-c75531d72ce3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:17.933882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95242ae0-f7d2-4e2b-bb0b-c75531d72ce3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:18.160525 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-95242ae0-f7d2-4e2b-bb0b-c75531d72ce3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:18.175800 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-95242ae0-f7d2-4e2b-bb0b-c75531d72ce3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1509 microversion: 2.1 time: 0.257771 Aug 28 00:57:18.176326 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 161/330] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:17 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1509 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:18.427301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a75a5e4f-5a21-4ffe-a532-3284001171fc tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:18.427918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a75a5e4f-5a21-4ffe-a532-3284001171fc tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 1964f323-9efc-456e-9a2a-981a0878e3e2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:18.436889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a75a5e4f-5a21-4ffe-a532-3284001171fc tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:18.437428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a75a5e4f-5a21-4ffe-a532-3284001171fc tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:18.437867 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a75a5e4f-5a21-4ffe-a532-3284001171fc tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:18.438584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a75a5e4f-5a21-4ffe-a532-3284001171fc tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:18.442110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a75a5e4f-5a21-4ffe-a532-3284001171fc tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:18.462346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a75a5e4f-5a21-4ffe-a532-3284001171fc tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:18.706332 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a75a5e4f-5a21-4ffe-a532-3284001171fc tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:18.714909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a75a5e4f-5a21-4ffe-a532-3284001171fc tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2" status: 200 len: 1503 microversion: 2.1 time: 0.291576 Aug 28 00:57:18.719378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 170/331] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:18 2022] GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2 => generated 1503 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:19.199366 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8d726fc3-cbc4-4731-a618-4b98b9cb88b1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:19.200159 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8d726fc3-cbc4-4731-a618-4b98b9cb88b1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:19.210931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8d726fc3-cbc4-4731-a618-4b98b9cb88b1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:19.214331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8d726fc3-cbc4-4731-a618-4b98b9cb88b1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:19.215060 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8d726fc3-cbc4-4731-a618-4b98b9cb88b1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:19.218402 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8d726fc3-cbc4-4731-a618-4b98b9cb88b1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:19.219034 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8d726fc3-cbc4-4731-a618-4b98b9cb88b1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:19.219766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8d726fc3-cbc4-4731-a618-4b98b9cb88b1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:19.428079 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8d726fc3-cbc4-4731-a618-4b98b9cb88b1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:19.438286 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8d726fc3-cbc4-4731-a618-4b98b9cb88b1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1509 microversion: 2.1 time: 0.243374 Aug 28 00:57:19.439477 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 162/332] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:19 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1509 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:19.740332 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7cac2475-41c9-4254-8829-c400381f1768 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:19.741878 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7cac2475-41c9-4254-8829-c400381f1768 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 1964f323-9efc-456e-9a2a-981a0878e3e2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:19.751375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7cac2475-41c9-4254-8829-c400381f1768 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:19.751904 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7cac2475-41c9-4254-8829-c400381f1768 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:19.752345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7cac2475-41c9-4254-8829-c400381f1768 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:19.755104 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7cac2475-41c9-4254-8829-c400381f1768 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:19.758872 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7cac2475-41c9-4254-8829-c400381f1768 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:19.759653 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7cac2475-41c9-4254-8829-c400381f1768 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:19.977283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7cac2475-41c9-4254-8829-c400381f1768 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:19.986437 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7cac2475-41c9-4254-8829-c400381f1768 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2" status: 200 len: 1503 microversion: 2.1 time: 0.250398 Aug 28 00:57:19.987049 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 171/333] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:19 2022] GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2 => generated 1503 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:20.464064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2cd8dcaa-688d-4e85-acb0-77ff3b000716 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:20.464863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2cd8dcaa-688d-4e85-acb0-77ff3b000716 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:20.476067 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2cd8dcaa-688d-4e85-acb0-77ff3b000716 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:20.476886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2cd8dcaa-688d-4e85-acb0-77ff3b000716 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:20.491087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2cd8dcaa-688d-4e85-acb0-77ff3b000716 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:20.493017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2cd8dcaa-688d-4e85-acb0-77ff3b000716 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:20.493427 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2cd8dcaa-688d-4e85-acb0-77ff3b000716 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:20.499368 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2cd8dcaa-688d-4e85-acb0-77ff3b000716 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:20.514308 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:57:20.688241 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5098ca02-9b97-4f3a-99a4-73e13cef105d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Action: 'create', calling method: .version_select at 0x7f0e6e9bbe50>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1578003485", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBGZ2aiMVIYtw2L+GTrjYhUzO0Up55PRbEPsanQla7Yvq9jusQuyKCkRc9vBt9Z9HNY1LGIT+RTjuvjas5Eji/QAhpN0/pyLL4lDoeYIAPKSUUWjPpfmPCGynocKyl55OQ=="}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:57:20.725382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-5098ca02-9b97-4f3a-99a4-73e13cef105d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Getting quotas for project 287c855ef8e241978a4d1e7cb3c7a16f. Resources: {'key_pairs'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 00:57:20.735198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-5098ca02-9b97-4f3a-99a4-73e13cef105d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Getting quotas for user cac1ab72dcee40b19bcff4dd5a75d82d and project 287c855ef8e241978a4d1e7cb3c7a16f. Resources: {'key_pairs'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 00:57:20.777017 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5098ca02-9b97-4f3a-99a4-73e13cef105d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.435457 Aug 28 00:57:20.804638 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 163/334] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Aug 28 00:57:20 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:57:20.894403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2cd8dcaa-688d-4e85-acb0-77ff3b000716 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:20.904425 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2cd8dcaa-688d-4e85-acb0-77ff3b000716 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1509 microversion: 2.1 time: 0.446828 Aug 28 00:57:20.905838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 172/335] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:20 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1509 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:21.023899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-307f39e8-71af-4fdf-98b6-091dd6025d08 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:21.025043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-307f39e8-71af-4fdf-98b6-091dd6025d08 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 1964f323-9efc-456e-9a2a-981a0878e3e2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:21.039559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-307f39e8-71af-4fdf-98b6-091dd6025d08 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:21.040840 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-307f39e8-71af-4fdf-98b6-091dd6025d08 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:21.041877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-307f39e8-71af-4fdf-98b6-091dd6025d08 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:21.043353 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-307f39e8-71af-4fdf-98b6-091dd6025d08 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:21.044465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-307f39e8-71af-4fdf-98b6-091dd6025d08 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:21.045251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-307f39e8-71af-4fdf-98b6-091dd6025d08 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:21.548734 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-307f39e8-71af-4fdf-98b6-091dd6025d08 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:21.561678 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-307f39e8-71af-4fdf-98b6-091dd6025d08 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2" status: 200 len: 1503 microversion: 2.1 time: 0.544776 Aug 28 00:57:21.564785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 164/336] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:21 2022] GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2 => generated 1503 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:21.935085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-31af7acf-ea11-4627-a798-35fe57f87851 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:21.936834 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-31af7acf-ea11-4627-a798-35fe57f87851 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:21.944305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-31af7acf-ea11-4627-a798-35fe57f87851 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:21.945411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-31af7acf-ea11-4627-a798-35fe57f87851 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:21.946467 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-31af7acf-ea11-4627-a798-35fe57f87851 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:21.948491 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-31af7acf-ea11-4627-a798-35fe57f87851 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:21.949288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-31af7acf-ea11-4627-a798-35fe57f87851 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:21.950888 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-31af7acf-ea11-4627-a798-35fe57f87851 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:22.190842 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-8d1ea875-da2a-4a81-ab17-47b54a4fbf1f req-3313f5c8-9773-463f-bb83-2d2e8ddceb17 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1964f323-9efc-456e-9a2a-981a0878e3e2", "name": "network-vif-deleted", "tag": "ea28f43d-6b9d-440f-856a-e133794342e6"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:57:22.213915 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-8d1ea875-da2a-4a81-ab17-47b54a4fbf1f req-3313f5c8-9773-463f-bb83-2d2e8ddceb17 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:22.216185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-8d1ea875-da2a-4a81-ab17-47b54a4fbf1f req-3313f5c8-9773-463f-bb83-2d2e8ddceb17 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:22.217785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-8d1ea875-da2a-4a81-ab17-47b54a4fbf1f req-3313f5c8-9773-463f-bb83-2d2e8ddceb17 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:22.251776 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-8d1ea875-da2a-4a81-ab17-47b54a4fbf1f req-3313f5c8-9773-463f-bb83-2d2e8ddceb17 service nova] Creating event network-vif-deleted:ea28f43d-6b9d-440f-856a-e133794342e6 for instance 1964f323-9efc-456e-9a2a-981a0878e3e2 on hv-cinder-maste Aug 28 00:57:22.265565 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-8d1ea875-da2a-4a81-ab17-47b54a4fbf1f req-3313f5c8-9773-463f-bb83-2d2e8ddceb17 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.083931 Aug 28 00:57:22.267237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 165/337] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:57:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:57:22.358810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-31af7acf-ea11-4627-a798-35fe57f87851 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:22.369964 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-31af7acf-ea11-4627-a798-35fe57f87851 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1515 microversion: 2.1 time: 0.439224 Aug 28 00:57:22.371472 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 173/338] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:21 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1515 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:57:22.613756 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c491bd96-967b-48b6-8a3c-a406cffd9ea3 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:22.615645 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c491bd96-967b-48b6-8a3c-a406cffd9ea3 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 1964f323-9efc-456e-9a2a-981a0878e3e2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:22.645127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c491bd96-967b-48b6-8a3c-a406cffd9ea3 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:22.646025 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c491bd96-967b-48b6-8a3c-a406cffd9ea3 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:22.649228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c491bd96-967b-48b6-8a3c-a406cffd9ea3 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:22.653144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c491bd96-967b-48b6-8a3c-a406cffd9ea3 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:22.653899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c491bd96-967b-48b6-8a3c-a406cffd9ea3 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:22.661614 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c491bd96-967b-48b6-8a3c-a406cffd9ea3 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:22.741878 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c491bd96-967b-48b6-8a3c-a406cffd9ea3 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:22.756514 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c491bd96-967b-48b6-8a3c-a406cffd9ea3 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:22.757858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c491bd96-967b-48b6-8a3c-a406cffd9ea3 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:22.761519 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.objects.instance [None req-c491bd96-967b-48b6-8a3c-a406cffd9ea3 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lazy-loading 'fault' on Instance uuid 1964f323-9efc-456e-9a2a-981a0878e3e2 {{(pid=120524) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 00:57:22.820587 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c491bd96-967b-48b6-8a3c-a406cffd9ea3 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:22.831889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c491bd96-967b-48b6-8a3c-a406cffd9ea3 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2" status: 200 len: 1304 microversion: 2.1 time: 0.230633 Aug 28 00:57:22.832921 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 166/339] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:22 2022] GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2 => generated 1304 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:23.869364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0f44f5e1-5b33-4cd3-bf24-8114e20cbbf2 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:23.870060 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0f44f5e1-5b33-4cd3-bf24-8114e20cbbf2 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 1964f323-9efc-456e-9a2a-981a0878e3e2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:23.877118 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f44f5e1-5b33-4cd3-bf24-8114e20cbbf2 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:23.877614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f44f5e1-5b33-4cd3-bf24-8114e20cbbf2 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:23.878044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f44f5e1-5b33-4cd3-bf24-8114e20cbbf2 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:23.878749 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f44f5e1-5b33-4cd3-bf24-8114e20cbbf2 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:23.879190 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f44f5e1-5b33-4cd3-bf24-8114e20cbbf2 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:23.879708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f44f5e1-5b33-4cd3-bf24-8114e20cbbf2 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:23.894760 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-0f44f5e1-5b33-4cd3-bf24-8114e20cbbf2 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] HTTP exception thrown: Instance 1964f323-9efc-456e-9a2a-981a0878e3e2 could not be found. Aug 28 00:57:23.896335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0f44f5e1-5b33-4cd3-bf24-8114e20cbbf2 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Request method failure captured: Aug 28 00:57:23.896335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2 HTTP/1.1 Aug 28 00:57:23.896335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 00:57:23.896335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 00:57:23.896335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 00:57:23.896335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 00:57:23.896335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 00:57:23.896335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 00:57:23.896335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 00:57:23.896335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 00:57:23.896335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCr1cMoPUe0bkvwrgCXK1x-PXWG4dAWe3fu6cufayhImfmbLqvwWIogu97x1WAtOd9qulHdsmmTUCY1T5fZyOq4f_W03DfmAUWrbP9Hg7mLNiCZWh8JrBi7GlyYKfOqvl609NQaqGzXIftLkBDRoFi2q9yigsmw0U52NRuE3tVYaIv-k Aug 28 00:57:23.896335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 00:57:23.896335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 00:57:23.896335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 00:57:23.896335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 00:57:23.896335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 00:57:23.896335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 00:57:23.896335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 315d6d72ee5844edb203b740ab2d08ce Aug 28 00:57:23.896335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-VolumesActionsTest-1664214821 Aug 28 00:57:23.896335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 00:57:23.896335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 00:57:23.899031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_315d6d72ee5844edb203b740ab2d08ce", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/315d6d72ee5844edb203b740ab2d08ce"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/315d6d72ee5844edb203b740ab2d08ce"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/315d6d72ee5844edb203b740ab2d08ce"}]}] Aug 28 00:57:23.899031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-VolumesActionsTest-1664214821 Aug 28 00:57:23.899031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 315d6d72ee5844edb203b740ab2d08ce Aug 28 00:57:23.899031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-VolumesActionsTest-1664214821 Aug 28 00:57:23.899031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-VolumesActionsTest-1664214821-project Aug 28 00:57:23.899031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 00:57:23.899031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 00:57:23.899031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: 59b04859082547149bf2776aefa63b43 Aug 28 00:57:23.899031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-VolumesActionsTest-1664214821-project Aug 28 00:57:23.899031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 00:57:23.899031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Instance 1964f323-9efc-456e-9a2a-981a0878e3e2 could not be found. Aug 28 00:57:23.899031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'id': '1964f323-9efc-456e-9a2a-981a0878e3e2'} Aug 28 00:57:23.900503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 00:57:23.900503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:57:23.900503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 00:57:23.900503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 00:57:23.900503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 00:57:23.900503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 00:57:23.900503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 00:57:23.900503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 00:57:23.900503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 00:57:23.900503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 00:57:23.900503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1964f323-9efc-456e-9a2a-981a0878e3e2 could not be found. Aug 28 00:57:23.900503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:57:23.900503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:57:23.900503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:57:23.900503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:57:23.900503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:57:23.900503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:57:23.900503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 00:57:23.900503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 00:57:23.900503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 00:57:23.902953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:57:23.902953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 00:57:23.902953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 00:57:23.902953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 00:57:23.902953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 00:57:23.902953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 00:57:23.902953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 00:57:23.902953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1964f323-9efc-456e-9a2a-981a0878e3e2 could not be found. Aug 28 00:57:23.902953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:57:23.902953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:57:23.902953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:57:23.902953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:57:23.902953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:57:23.902953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:57:23.902953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 00:57:23.902953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 00:57:23.902953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1964f323-9efc-456e-9a2a-981a0878e3e2 could not be found. Aug 28 00:57:23.902953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:57:23.905121 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0f44f5e1-5b33-4cd3-bf24-8114e20cbbf2 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Returning 404 to user: Instance 1964f323-9efc-456e-9a2a-981a0878e3e2 could not be found. {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 00:57:23.905121 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0f44f5e1-5b33-4cd3-bf24-8114e20cbbf2 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2" status: 404 len: 111 microversion: 2.1 time: 0.033114 Aug 28 00:57:23.906893 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 174/340] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:23 2022] GET /compute/v2.1/servers/1964f323-9efc-456e-9a2a-981a0878e3e2 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 00:57:24.043741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4c8caf33-514d-403b-9373-9320b1cc5f41 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-92814670", "imageRef": "2c2585b2-bde5-4652-aed7-9f53b1bd5d94", "flavorRef": "452", "networks": [{"uuid": "7884bb82-c947-4f13-a88b-bdc2fd404591"}]}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:57:24.135579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.network.neutron [None req-4c8caf33-514d-403b-9373-9320b1cc5f41 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] validate_networks() for [('7884bb82-c947-4f13-a88b-bdc2fd404591', None, None, None, None, None)] {{(pid=120524) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 28 00:57:24.292375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2b16cd9e-3679-40e6-9e84-36dc8a5fc500 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Action: 'create', calling method: .version_select at 0x7f0e6ea604c0>, body: {"keypair": {"name": "tempest-keypair-225780864", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGUvFH3E3+/GhKCYFK3TBK452ou5naBCPxp6qnatSdVw+/88YrBsv+og/6bSiemgYVRhDC5ObhvrdvQbArNor3Q/84ZrJ2w6P24dZCN3l0sYD8mj015/8Unut5GxaGP6fw=="}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:57:24.333124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-2b16cd9e-3679-40e6-9e84-36dc8a5fc500 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Getting quotas for project 6f3acee4af62483ab1f2cbf3a47b09e1. Resources: {'key_pairs'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 00:57:24.338228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-2b16cd9e-3679-40e6-9e84-36dc8a5fc500 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Getting quotas for user 1ab2fcb3759045c7a076e5c8222a421d and project 6f3acee4af62483ab1f2cbf3a47b09e1. Resources: {'key_pairs'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 00:57:24.373924 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2b16cd9e-3679-40e6-9e84-36dc8a5fc500 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.090554 Aug 28 00:57:24.381805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 175/341] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Aug 28 00:57:24 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:57:24.674102 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-4c8caf33-514d-403b-9373-9320b1cc5f41 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Flavor limits 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 28 00:57:24.674714 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-4c8caf33-514d-403b-9373-9320b1cc5f41 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Image limits 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 28 00:57:24.675429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-4c8caf33-514d-403b-9373-9320b1cc5f41 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Flavor pref 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 28 00:57:24.676100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-4c8caf33-514d-403b-9373-9320b1cc5f41 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Image pref 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 28 00:57:24.676686 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-4c8caf33-514d-403b-9373-9320b1cc5f41 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 28 00:57:24.698404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c8caf33-514d-403b-9373-9320b1cc5f41 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:24.698917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c8caf33-514d-403b-9373-9320b1cc5f41 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:24.701758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c8caf33-514d-403b-9373-9320b1cc5f41 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:24.702765 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c8caf33-514d-403b-9373-9320b1cc5f41 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:24.703419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c8caf33-514d-403b-9373-9320b1cc5f41 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:24.704679 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c8caf33-514d-403b-9373-9320b1cc5f41 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:24.724888 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-4c8caf33-514d-403b-9373-9320b1cc5f41 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Getting quotas for project 315d6d72ee5844edb203b740ab2d08ce. Resources: {'ram', 'instances', 'cores'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 00:57:24.730041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-4c8caf33-514d-403b-9373-9320b1cc5f41 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Getting quotas for user 59b04859082547149bf2776aefa63b43 and project 315d6d72ee5844edb203b740ab2d08ce. Resources: {'ram', 'instances', 'cores'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 00:57:24.745517 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4c8caf33-514d-403b-9373-9320b1cc5f41 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Going to run 1 instances... {{(pid=120524) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 28 00:57:24.748818 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4c8caf33-514d-403b-9373-9320b1cc5f41 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2c2585b2-bde5-4652-aed7-9f53b1bd5d94',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120524) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 28 00:57:24.826538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4c8caf33-514d-403b-9373-9320b1cc5f41 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.788639 Aug 28 00:57:24.826538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 167/342] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Aug 28 00:57:24 2022] POST /compute/v2.1/servers => generated 380 bytes in 791 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 28 00:57:24.838713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7656f26a-2dda-4e8c-b5cc-749250f60f5a tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:24.839713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7656f26a-2dda-4e8c-b5cc-749250f60f5a tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:24.933572 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7656f26a-2dda-4e8c-b5cc-749250f60f5a tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:24.944943 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7656f26a-2dda-4e8c-b5cc-749250f60f5a tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 200 len: 1212 microversion: 2.1 time: 0.111462 Aug 28 00:57:24.945872 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 176/343] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:24 2022] GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 1212 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:25.976509 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9804f281-91bc-4554-888a-607594525295 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:25.978468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9804f281-91bc-4554-888a-607594525295 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:25.989876 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9804f281-91bc-4554-888a-607594525295 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:25.990932 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9804f281-91bc-4554-888a-607594525295 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:25.991561 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9804f281-91bc-4554-888a-607594525295 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:25.992534 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9804f281-91bc-4554-888a-607594525295 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:25.993419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9804f281-91bc-4554-888a-607594525295 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:25.994556 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9804f281-91bc-4554-888a-607594525295 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:26.094384 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9804f281-91bc-4554-888a-607594525295 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:26.102993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9804f281-91bc-4554-888a-607594525295 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 200 len: 1264 microversion: 2.1 time: 0.137042 Aug 28 00:57:26.103580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 168/344] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:25 2022] GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 1264 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:26.742632 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8042f6f4-8484-4dea-a08b-c7157edd2323 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-2140404227", "imageRef": "2c2585b2-bde5-4652-aed7-9f53b1bd5d94", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "92aee389-59bb-4ec3-9e65-fe3bcb195f1a"}], "security_groups": [{"name": "tempest-securitygroup--520573514"}], "key_name": "tempest-keypair-225780864", "user_data": "}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:57:26.927200 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.network.neutron [None req-8042f6f4-8484-4dea-a08b-c7157edd2323 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] validate_networks() for [('92aee389-59bb-4ec3-9e65-fe3bcb195f1a', None, None, None, None, None)] {{(pid=120525) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 28 00:57:27.133805 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ab1b35fb-6655-4329-bffc-663f82bf5e0f tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:27.134445 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ab1b35fb-6655-4329-bffc-663f82bf5e0f tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:27.145850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab1b35fb-6655-4329-bffc-663f82bf5e0f tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:27.146696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab1b35fb-6655-4329-bffc-663f82bf5e0f tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:27.147605 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab1b35fb-6655-4329-bffc-663f82bf5e0f tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:27.149229 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab1b35fb-6655-4329-bffc-663f82bf5e0f tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:27.149924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab1b35fb-6655-4329-bffc-663f82bf5e0f tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:27.150602 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab1b35fb-6655-4329-bffc-663f82bf5e0f tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:27.262784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ab1b35fb-6655-4329-bffc-663f82bf5e0f tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:27.277425 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ab1b35fb-6655-4329-bffc-663f82bf5e0f tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 200 len: 1270 microversion: 2.1 time: 0.148944 Aug 28 00:57:27.279370 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 169/345] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:27 2022] GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 1270 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:27.687768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-8042f6f4-8484-4dea-a08b-c7157edd2323 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Flavor limits 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 28 00:57:27.688492 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-8042f6f4-8484-4dea-a08b-c7157edd2323 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Image limits 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 28 00:57:27.689361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-8042f6f4-8484-4dea-a08b-c7157edd2323 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Flavor pref 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 28 00:57:27.689897 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-8042f6f4-8484-4dea-a08b-c7157edd2323 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Image pref 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 28 00:57:27.690312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-8042f6f4-8484-4dea-a08b-c7157edd2323 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 28 00:57:27.705675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8042f6f4-8484-4dea-a08b-c7157edd2323 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:27.706521 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8042f6f4-8484-4dea-a08b-c7157edd2323 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:27.707196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8042f6f4-8484-4dea-a08b-c7157edd2323 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:27.708490 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8042f6f4-8484-4dea-a08b-c7157edd2323 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:27.709257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8042f6f4-8484-4dea-a08b-c7157edd2323 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:27.709779 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8042f6f4-8484-4dea-a08b-c7157edd2323 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:27.741445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-8042f6f4-8484-4dea-a08b-c7157edd2323 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Getting quotas for project 6f3acee4af62483ab1f2cbf3a47b09e1. Resources: {'ram', 'instances', 'cores'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 00:57:27.748189 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-8042f6f4-8484-4dea-a08b-c7157edd2323 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Getting quotas for user 1ab2fcb3759045c7a076e5c8222a421d and project 6f3acee4af62483ab1f2cbf3a47b09e1. Resources: {'ram', 'instances', 'cores'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 00:57:27.772905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8042f6f4-8484-4dea-a08b-c7157edd2323 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Going to run 1 instances... {{(pid=120525) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 28 00:57:27.776566 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8042f6f4-8484-4dea-a08b-c7157edd2323 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2c2585b2-bde5-4652-aed7-9f53b1bd5d94',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120525) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 28 00:57:27.835649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8042f6f4-8484-4dea-a08b-c7157edd2323 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 408 microversion: 2.1 time: 1.097954 Aug 28 00:57:27.836299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 177/346] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Aug 28 00:57:26 2022] POST /compute/v2.1/servers => generated 408 bytes in 1099 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 28 00:57:27.864624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-abbed721-f683-43f0-b6e6-a3e5a301df88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:27.865864 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-abbed721-f683-43f0-b6e6-a3e5a301df88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:27.961940 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-abbed721-f683-43f0-b6e6-a3e5a301df88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:27.971216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-abbed721-f683-43f0-b6e6-a3e5a301df88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1245 microversion: 2.1 time: 0.124885 Aug 28 00:57:27.972387 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 170/347] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:27 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1245 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:28.300807 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-dc5969a1-8b41-44d7-9894-f05f5335dbf5 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:28.301608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-dc5969a1-8b41-44d7-9894-f05f5335dbf5 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:28.309596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc5969a1-8b41-44d7-9894-f05f5335dbf5 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:28.310481 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc5969a1-8b41-44d7-9894-f05f5335dbf5 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:28.311306 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc5969a1-8b41-44d7-9894-f05f5335dbf5 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:28.312543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc5969a1-8b41-44d7-9894-f05f5335dbf5 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:28.313359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc5969a1-8b41-44d7-9894-f05f5335dbf5 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:28.313864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc5969a1-8b41-44d7-9894-f05f5335dbf5 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:28.722564 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dc5969a1-8b41-44d7-9894-f05f5335dbf5 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:28.737377 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-dc5969a1-8b41-44d7-9894-f05f5335dbf5 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 200 len: 1312 microversion: 2.1 time: 0.439959 Aug 28 00:57:28.748669 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 178/348] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:28 2022] GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 1312 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:57:28.900996 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-0751b27d-37c8-4b2e-a387-bf32a1432747 req-73c742b0-a19a-4e3a-b1bf-5a0cf07a277a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "11430c0a-d74c-48c3-a7a9-9b9922804cca", "tag": "9addba86-2245-48de-a1af-5e38ae4ae619"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:57:28.921208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-0751b27d-37c8-4b2e-a387-bf32a1432747 req-73c742b0-a19a-4e3a-b1bf-5a0cf07a277a service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:28.922127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-0751b27d-37c8-4b2e-a387-bf32a1432747 req-73c742b0-a19a-4e3a-b1bf-5a0cf07a277a service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:28.922837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-0751b27d-37c8-4b2e-a387-bf32a1432747 req-73c742b0-a19a-4e3a-b1bf-5a0cf07a277a service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:28.948334 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-0751b27d-37c8-4b2e-a387-bf32a1432747 req-73c742b0-a19a-4e3a-b1bf-5a0cf07a277a service nova] Creating event network-changed:9addba86-2245-48de-a1af-5e38ae4ae619 for instance 11430c0a-d74c-48c3-a7a9-9b9922804cca on hv-cinder-maste Aug 28 00:57:28.958390 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-0751b27d-37c8-4b2e-a387-bf32a1432747 req-73c742b0-a19a-4e3a-b1bf-5a0cf07a277a service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068680 Aug 28 00:57:28.960242 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 171/349] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:57:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:57:28.998947 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-464dd9a0-f0c5-4130-8f91-52366991a8ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:29.000084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-464dd9a0-f0c5-4130-8f91-52366991a8ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:29.013147 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-464dd9a0-f0c5-4130-8f91-52366991a8ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:29.014058 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-464dd9a0-f0c5-4130-8f91-52366991a8ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:29.015054 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-464dd9a0-f0c5-4130-8f91-52366991a8ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:29.016330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-464dd9a0-f0c5-4130-8f91-52366991a8ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:29.017228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-464dd9a0-f0c5-4130-8f91-52366991a8ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:29.018003 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-464dd9a0-f0c5-4130-8f91-52366991a8ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:29.151105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-464dd9a0-f0c5-4130-8f91-52366991a8ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:29.167525 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-464dd9a0-f0c5-4130-8f91-52366991a8ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1297 microversion: 2.1 time: 0.176243 Aug 28 00:57:29.169356 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 179/350] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:28 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1297 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:29.761530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5183aaa9-97f6-4ef1-a82a-f156bc2a98df tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:29.762368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5183aaa9-97f6-4ef1-a82a-f156bc2a98df tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:29.769265 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5183aaa9-97f6-4ef1-a82a-f156bc2a98df tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:29.769886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5183aaa9-97f6-4ef1-a82a-f156bc2a98df tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:29.770588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5183aaa9-97f6-4ef1-a82a-f156bc2a98df tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:29.771682 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5183aaa9-97f6-4ef1-a82a-f156bc2a98df tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:29.772269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5183aaa9-97f6-4ef1-a82a-f156bc2a98df tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:29.772800 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5183aaa9-97f6-4ef1-a82a-f156bc2a98df tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:30.066309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5183aaa9-97f6-4ef1-a82a-f156bc2a98df tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:30.075325 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5183aaa9-97f6-4ef1-a82a-f156bc2a98df tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 200 len: 1312 microversion: 2.1 time: 0.318750 Aug 28 00:57:30.075919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 172/351] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:29 2022] GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 1312 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:30.206043 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b8eae545-0476-4ea9-9e05-9cf4176e54ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:30.206637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b8eae545-0476-4ea9-9e05-9cf4176e54ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:30.215400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8eae545-0476-4ea9-9e05-9cf4176e54ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:30.216036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8eae545-0476-4ea9-9e05-9cf4176e54ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:30.216675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8eae545-0476-4ea9-9e05-9cf4176e54ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:30.217805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8eae545-0476-4ea9-9e05-9cf4176e54ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:30.218375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8eae545-0476-4ea9-9e05-9cf4176e54ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:30.219014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8eae545-0476-4ea9-9e05-9cf4176e54ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:30.294350 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b8eae545-0476-4ea9-9e05-9cf4176e54ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:30.308122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b8eae545-0476-4ea9-9e05-9cf4176e54ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1303 microversion: 2.1 time: 0.106975 Aug 28 00:57:30.309177 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 180/352] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:30 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1303 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:57:31.111778 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-18dd3226-b984-4ca2-80e2-b5da08ce2e61 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:31.112859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-18dd3226-b984-4ca2-80e2-b5da08ce2e61 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:31.123846 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-18dd3226-b984-4ca2-80e2-b5da08ce2e61 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:31.124580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-18dd3226-b984-4ca2-80e2-b5da08ce2e61 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:31.125287 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-18dd3226-b984-4ca2-80e2-b5da08ce2e61 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:31.126578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-18dd3226-b984-4ca2-80e2-b5da08ce2e61 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:31.127391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-18dd3226-b984-4ca2-80e2-b5da08ce2e61 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:31.128140 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-18dd3226-b984-4ca2-80e2-b5da08ce2e61 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:31.149997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:57:31.294197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-18dd3226-b984-4ca2-80e2-b5da08ce2e61 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:31.306333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-18dd3226-b984-4ca2-80e2-b5da08ce2e61 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 200 len: 1312 microversion: 2.1 time: 0.199696 Aug 28 00:57:31.307499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 173/353] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:31 2022] GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 1312 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:31.345911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d6bf8c1f-c8c3-4c6c-8ff2-0cb86778fae9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:31.346908 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d6bf8c1f-c8c3-4c6c-8ff2-0cb86778fae9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:31.357574 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6bf8c1f-c8c3-4c6c-8ff2-0cb86778fae9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:31.358414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6bf8c1f-c8c3-4c6c-8ff2-0cb86778fae9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:31.359291 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6bf8c1f-c8c3-4c6c-8ff2-0cb86778fae9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:31.360545 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6bf8c1f-c8c3-4c6c-8ff2-0cb86778fae9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:31.361481 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6bf8c1f-c8c3-4c6c-8ff2-0cb86778fae9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:31.362256 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6bf8c1f-c8c3-4c6c-8ff2-0cb86778fae9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:31.445442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d6bf8c1f-c8c3-4c6c-8ff2-0cb86778fae9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:31.454109 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d6bf8c1f-c8c3-4c6c-8ff2-0cb86778fae9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1303 microversion: 2.1 time: 0.115320 Aug 28 00:57:31.454883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 181/354] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:31 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1303 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:32.341839 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b3799a8a-5760-44a3-ae03-90d2ff0be718 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:32.342722 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b3799a8a-5760-44a3-ae03-90d2ff0be718 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:32.351122 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3799a8a-5760-44a3-ae03-90d2ff0be718 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:32.351636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3799a8a-5760-44a3-ae03-90d2ff0be718 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:32.352211 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3799a8a-5760-44a3-ae03-90d2ff0be718 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:32.353056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3799a8a-5760-44a3-ae03-90d2ff0be718 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:32.353549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3799a8a-5760-44a3-ae03-90d2ff0be718 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:32.354041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3799a8a-5760-44a3-ae03-90d2ff0be718 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:32.479772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-dbcc302a-e645-46e9-822a-05f7382cee1b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:32.481048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-dbcc302a-e645-46e9-822a-05f7382cee1b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:32.492140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbcc302a-e645-46e9-822a-05f7382cee1b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:32.497864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbcc302a-e645-46e9-822a-05f7382cee1b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:32.498757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbcc302a-e645-46e9-822a-05f7382cee1b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:32.500029 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbcc302a-e645-46e9-822a-05f7382cee1b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:32.500856 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbcc302a-e645-46e9-822a-05f7382cee1b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:32.501656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbcc302a-e645-46e9-822a-05f7382cee1b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:32.533157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b3799a8a-5760-44a3-ae03-90d2ff0be718 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:32.547914 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b3799a8a-5760-44a3-ae03-90d2ff0be718 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 200 len: 1312 microversion: 2.1 time: 0.211169 Aug 28 00:57:32.549041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 174/355] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:32 2022] GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 1312 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:32.596908 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dbcc302a-e645-46e9-822a-05f7382cee1b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:32.606954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-dbcc302a-e645-46e9-822a-05f7382cee1b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1303 microversion: 2.1 time: 0.133835 Aug 28 00:57:32.607562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 182/356] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:32 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1303 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:57:33.584614 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1499ab36-7e78-4a04-9c35-2a470323d4fa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:33.585540 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1499ab36-7e78-4a04-9c35-2a470323d4fa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:33.596581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1499ab36-7e78-4a04-9c35-2a470323d4fa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:33.597482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1499ab36-7e78-4a04-9c35-2a470323d4fa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:33.598126 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1499ab36-7e78-4a04-9c35-2a470323d4fa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:33.599273 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1499ab36-7e78-4a04-9c35-2a470323d4fa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:33.599926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1499ab36-7e78-4a04-9c35-2a470323d4fa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:33.600929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1499ab36-7e78-4a04-9c35-2a470323d4fa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:33.635431 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-38ff751e-78cb-4b66-9bf4-03e8d776a5b1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:33.636664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-38ff751e-78cb-4b66-9bf4-03e8d776a5b1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:33.641247 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=7 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:57:33.651539 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38ff751e-78cb-4b66-9bf4-03e8d776a5b1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:33.652380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38ff751e-78cb-4b66-9bf4-03e8d776a5b1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:33.653171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38ff751e-78cb-4b66-9bf4-03e8d776a5b1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:33.654503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38ff751e-78cb-4b66-9bf4-03e8d776a5b1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:33.655376 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38ff751e-78cb-4b66-9bf4-03e8d776a5b1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:33.656374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38ff751e-78cb-4b66-9bf4-03e8d776a5b1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:34.091690 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-38ff751e-78cb-4b66-9bf4-03e8d776a5b1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:34.105762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-38ff751e-78cb-4b66-9bf4-03e8d776a5b1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1303 microversion: 2.1 time: 0.479992 Aug 28 00:57:34.107315 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 183/357] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:33 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1303 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:34.298172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1499ab36-7e78-4a04-9c35-2a470323d4fa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:34.316073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1499ab36-7e78-4a04-9c35-2a470323d4fa tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 200 len: 1312 microversion: 2.1 time: 0.736588 Aug 28 00:57:34.318188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 175/358] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:33 2022] GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 1312 bytes in 741 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:57:35.130388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d08b46a2-b69c-4ad7-bf22-06cda843bf9b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:35.130792 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d08b46a2-b69c-4ad7-bf22-06cda843bf9b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:35.137308 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d08b46a2-b69c-4ad7-bf22-06cda843bf9b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:35.137868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d08b46a2-b69c-4ad7-bf22-06cda843bf9b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:35.138305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d08b46a2-b69c-4ad7-bf22-06cda843bf9b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:35.139292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d08b46a2-b69c-4ad7-bf22-06cda843bf9b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:35.139753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d08b46a2-b69c-4ad7-bf22-06cda843bf9b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:35.140181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d08b46a2-b69c-4ad7-bf22-06cda843bf9b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:35.347003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d9a6b9ac-05c5-4a0d-9572-3c0a631a8a08 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:35.347627 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d9a6b9ac-05c5-4a0d-9572-3c0a631a8a08 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:35.350121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=2 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:57:35.356797 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d9a6b9ac-05c5-4a0d-9572-3c0a631a8a08 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:35.357666 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d9a6b9ac-05c5-4a0d-9572-3c0a631a8a08 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:35.358273 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d9a6b9ac-05c5-4a0d-9572-3c0a631a8a08 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:35.359212 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d9a6b9ac-05c5-4a0d-9572-3c0a631a8a08 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:35.359840 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d9a6b9ac-05c5-4a0d-9572-3c0a631a8a08 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:35.360462 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d9a6b9ac-05c5-4a0d-9572-3c0a631a8a08 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:35.521517 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d08b46a2-b69c-4ad7-bf22-06cda843bf9b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:35.531772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d08b46a2-b69c-4ad7-bf22-06cda843bf9b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1370 microversion: 2.1 time: 0.406409 Aug 28 00:57:35.533096 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 184/359] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:35 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1370 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:35.651533 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d9a6b9ac-05c5-4a0d-9572-3c0a631a8a08 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:35.667934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d9a6b9ac-05c5-4a0d-9572-3c0a631a8a08 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 200 len: 1312 microversion: 2.1 time: 0.325501 Aug 28 00:57:35.668977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 176/360] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:35 2022] GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 1312 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:36.001423 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-0d990dca-e7e2-445f-adbb-d60d8221e0d1 req-722094f4-2643-488f-9199-b7d72c35bec3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "03969ac5-934e-4201-8087-0927eae2bbd2", "tag": "b9bc50e7-87a3-480a-8c62-87fe75411e15"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:57:36.012045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-0d990dca-e7e2-445f-adbb-d60d8221e0d1 req-722094f4-2643-488f-9199-b7d72c35bec3 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:36.012604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-0d990dca-e7e2-445f-adbb-d60d8221e0d1 req-722094f4-2643-488f-9199-b7d72c35bec3 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:36.013237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-0d990dca-e7e2-445f-adbb-d60d8221e0d1 req-722094f4-2643-488f-9199-b7d72c35bec3 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:36.027193 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-0d990dca-e7e2-445f-adbb-d60d8221e0d1 req-722094f4-2643-488f-9199-b7d72c35bec3 service nova] Creating event network-changed:b9bc50e7-87a3-480a-8c62-87fe75411e15 for instance 03969ac5-934e-4201-8087-0927eae2bbd2 on hv-cinder-maste Aug 28 00:57:36.032858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-0d990dca-e7e2-445f-adbb-d60d8221e0d1 req-722094f4-2643-488f-9199-b7d72c35bec3 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036192 Aug 28 00:57:36.033721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 185/361] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:57:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:57:36.569112 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-166eb488-c6b6-4b4a-a9ea-9dd0b895aa1d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:36.570023 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-166eb488-c6b6-4b4a-a9ea-9dd0b895aa1d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:36.578256 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-166eb488-c6b6-4b4a-a9ea-9dd0b895aa1d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:36.578822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-166eb488-c6b6-4b4a-a9ea-9dd0b895aa1d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:36.579305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-166eb488-c6b6-4b4a-a9ea-9dd0b895aa1d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:36.580093 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-166eb488-c6b6-4b4a-a9ea-9dd0b895aa1d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:36.580581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-166eb488-c6b6-4b4a-a9ea-9dd0b895aa1d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:36.581135 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-166eb488-c6b6-4b4a-a9ea-9dd0b895aa1d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:36.704678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ebc7352f-11a7-4333-95c6-d9c9388e6e62 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:36.708864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ebc7352f-11a7-4333-95c6-d9c9388e6e62 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:36.721081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebc7352f-11a7-4333-95c6-d9c9388e6e62 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:36.722832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebc7352f-11a7-4333-95c6-d9c9388e6e62 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:36.723600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebc7352f-11a7-4333-95c6-d9c9388e6e62 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:36.725188 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebc7352f-11a7-4333-95c6-d9c9388e6e62 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:36.726271 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebc7352f-11a7-4333-95c6-d9c9388e6e62 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:36.727298 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebc7352f-11a7-4333-95c6-d9c9388e6e62 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:36.829676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-166eb488-c6b6-4b4a-a9ea-9dd0b895aa1d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:36.839876 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-166eb488-c6b6-4b4a-a9ea-9dd0b895aa1d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1370 microversion: 2.1 time: 0.275681 Aug 28 00:57:36.846294 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 177/362] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:36 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1370 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:36.958600 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-7d7af9e3-7c89-4ea6-9b77-778e8f1e66c0 req-17a0e9c0-733d-4e4e-865c-756380e52896 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "11430c0a-d74c-48c3-a7a9-9b9922804cca", "name": "network-vif-plugged", "status": "completed", "tag": "9addba86-2245-48de-a1af-5e38ae4ae619"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:57:36.978869 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-7d7af9e3-7c89-4ea6-9b77-778e8f1e66c0 req-17a0e9c0-733d-4e4e-865c-756380e52896 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:36.979715 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-7d7af9e3-7c89-4ea6-9b77-778e8f1e66c0 req-17a0e9c0-733d-4e4e-865c-756380e52896 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:36.980457 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ebc7352f-11a7-4333-95c6-d9c9388e6e62 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:36.980598 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-7d7af9e3-7c89-4ea6-9b77-778e8f1e66c0 req-17a0e9c0-733d-4e4e-865c-756380e52896 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:36.993526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ebc7352f-11a7-4333-95c6-d9c9388e6e62 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 200 len: 1312 microversion: 2.1 time: 0.296730 Aug 28 00:57:36.997052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 186/363] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:36 2022] GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 1312 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:37.008481 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-7d7af9e3-7c89-4ea6-9b77-778e8f1e66c0 req-17a0e9c0-733d-4e4e-865c-756380e52896 service nova] Creating event network-vif-plugged:9addba86-2245-48de-a1af-5e38ae4ae619 for instance 11430c0a-d74c-48c3-a7a9-9b9922804cca on hv-cinder-maste Aug 28 00:57:37.018431 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-7d7af9e3-7c89-4ea6-9b77-778e8f1e66c0 req-17a0e9c0-733d-4e4e-865c-756380e52896 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.066526 Aug 28 00:57:37.019487 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 178/364] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:57:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:57:37.864375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9a8debee-f9bb-44d6-ab93-a55e9be679e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:37.865079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9a8debee-f9bb-44d6-ab93-a55e9be679e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:37.867592 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=2 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:57:37.868497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=12 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:57:37.876761 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a8debee-f9bb-44d6-ab93-a55e9be679e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:37.877283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a8debee-f9bb-44d6-ab93-a55e9be679e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:37.877864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a8debee-f9bb-44d6-ab93-a55e9be679e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:37.878580 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a8debee-f9bb-44d6-ab93-a55e9be679e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:37.879031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a8debee-f9bb-44d6-ab93-a55e9be679e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:37.879448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a8debee-f9bb-44d6-ab93-a55e9be679e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:38.014565 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=12 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:57:38.023148 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ac7853bc-a47e-40da-be97-6f765b8dcf66 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:38.024460 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ac7853bc-a47e-40da-be97-6f765b8dcf66 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:38.038748 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac7853bc-a47e-40da-be97-6f765b8dcf66 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:38.040109 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac7853bc-a47e-40da-be97-6f765b8dcf66 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:38.041209 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac7853bc-a47e-40da-be97-6f765b8dcf66 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:38.042744 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac7853bc-a47e-40da-be97-6f765b8dcf66 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:38.043597 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac7853bc-a47e-40da-be97-6f765b8dcf66 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:38.044611 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac7853bc-a47e-40da-be97-6f765b8dcf66 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:38.048668 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9a8debee-f9bb-44d6-ab93-a55e9be679e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:38.058979 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9a8debee-f9bb-44d6-ab93-a55e9be679e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1370 microversion: 2.1 time: 0.199209 Aug 28 00:57:38.059627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 187/365] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:37 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1370 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:38.225405 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ac7853bc-a47e-40da-be97-6f765b8dcf66 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:38.234213 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ac7853bc-a47e-40da-be97-6f765b8dcf66 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 200 len: 1312 microversion: 2.1 time: 0.221200 Aug 28 00:57:38.234805 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 179/366] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:38 2022] GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 1312 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:39.084170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-061875a8-638b-4a41-a214-1dd3a2b614b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:39.084934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-061875a8-638b-4a41-a214-1dd3a2b614b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:39.092400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-061875a8-638b-4a41-a214-1dd3a2b614b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:39.092909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-061875a8-638b-4a41-a214-1dd3a2b614b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:39.093401 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-061875a8-638b-4a41-a214-1dd3a2b614b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:39.094139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-061875a8-638b-4a41-a214-1dd3a2b614b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:39.094588 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-061875a8-638b-4a41-a214-1dd3a2b614b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:39.095005 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-061875a8-638b-4a41-a214-1dd3a2b614b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:39.256072 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-064fe0a8-8a34-4d29-839a-25ec3e85f7f4 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:39.259919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-064fe0a8-8a34-4d29-839a-25ec3e85f7f4 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:39.266672 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-064fe0a8-8a34-4d29-839a-25ec3e85f7f4 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:39.267332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-064fe0a8-8a34-4d29-839a-25ec3e85f7f4 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:39.267907 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-064fe0a8-8a34-4d29-839a-25ec3e85f7f4 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:39.268460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-061875a8-638b-4a41-a214-1dd3a2b614b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:39.268692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-064fe0a8-8a34-4d29-839a-25ec3e85f7f4 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:39.269266 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-064fe0a8-8a34-4d29-839a-25ec3e85f7f4 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:39.269791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-064fe0a8-8a34-4d29-839a-25ec3e85f7f4 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:39.281161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-061875a8-638b-4a41-a214-1dd3a2b614b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1370 microversion: 2.1 time: 0.200831 Aug 28 00:57:39.281887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 188/367] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:39 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1370 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:39.426547 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-064fe0a8-8a34-4d29-839a-25ec3e85f7f4 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:39.435607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-064fe0a8-8a34-4d29-839a-25ec3e85f7f4 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 200 len: 1312 microversion: 2.1 time: 0.183389 Aug 28 00:57:39.436389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 180/368] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:39 2022] GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 1312 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:40.315485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2df7ddcd-ee01-466d-8ee7-12b01447bf07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:40.316090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2df7ddcd-ee01-466d-8ee7-12b01447bf07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:40.323282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2df7ddcd-ee01-466d-8ee7-12b01447bf07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:40.323777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2df7ddcd-ee01-466d-8ee7-12b01447bf07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:40.324206 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2df7ddcd-ee01-466d-8ee7-12b01447bf07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:40.324920 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2df7ddcd-ee01-466d-8ee7-12b01447bf07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:40.325400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2df7ddcd-ee01-466d-8ee7-12b01447bf07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:40.325816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2df7ddcd-ee01-466d-8ee7-12b01447bf07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:40.459669 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e8ea2b5d-8658-4433-a0bf-f47585ca3d16 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:40.460603 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e8ea2b5d-8658-4433-a0bf-f47585ca3d16 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:40.472838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8ea2b5d-8658-4433-a0bf-f47585ca3d16 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:40.473698 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8ea2b5d-8658-4433-a0bf-f47585ca3d16 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:40.474469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8ea2b5d-8658-4433-a0bf-f47585ca3d16 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:40.475683 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8ea2b5d-8658-4433-a0bf-f47585ca3d16 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:40.476320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8ea2b5d-8658-4433-a0bf-f47585ca3d16 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:40.476846 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8ea2b5d-8658-4433-a0bf-f47585ca3d16 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:40.489857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2df7ddcd-ee01-466d-8ee7-12b01447bf07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:40.501501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2df7ddcd-ee01-466d-8ee7-12b01447bf07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1370 microversion: 2.1 time: 0.189642 Aug 28 00:57:40.502669 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 189/369] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:40 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1370 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:40.774051 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e8ea2b5d-8658-4433-a0bf-f47585ca3d16 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:40.787667 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e8ea2b5d-8658-4433-a0bf-f47585ca3d16 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 200 len: 1312 microversion: 2.1 time: 0.333482 Aug 28 00:57:40.788645 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 181/370] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:40 2022] GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 1312 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:57:41.540680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-942a6130-6f1c-47e3-82ce-8817257e96a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:41.541652 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-942a6130-6f1c-47e3-82ce-8817257e96a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:41.549523 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-942a6130-6f1c-47e3-82ce-8817257e96a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:41.550164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-942a6130-6f1c-47e3-82ce-8817257e96a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:41.550708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-942a6130-6f1c-47e3-82ce-8817257e96a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:41.551669 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-942a6130-6f1c-47e3-82ce-8817257e96a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:41.552244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-942a6130-6f1c-47e3-82ce-8817257e96a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:41.552913 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-942a6130-6f1c-47e3-82ce-8817257e96a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:41.759595 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-942a6130-6f1c-47e3-82ce-8817257e96a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:41.775630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-942a6130-6f1c-47e3-82ce-8817257e96a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1370 microversion: 2.1 time: 0.238872 Aug 28 00:57:41.777089 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 190/371] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:41 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1370 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:41.824752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cc592b20-9a96-443c-8125-a5a12b4c115f tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:41.825453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cc592b20-9a96-443c-8125-a5a12b4c115f tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:41.832823 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc592b20-9a96-443c-8125-a5a12b4c115f tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:41.833356 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc592b20-9a96-443c-8125-a5a12b4c115f tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:41.833793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc592b20-9a96-443c-8125-a5a12b4c115f tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:41.834501 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc592b20-9a96-443c-8125-a5a12b4c115f tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:41.834968 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc592b20-9a96-443c-8125-a5a12b4c115f tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:41.835389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc592b20-9a96-443c-8125-a5a12b4c115f tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:41.988476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-cc592b20-9a96-443c-8125-a5a12b4c115f tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:41.997385 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cc592b20-9a96-443c-8125-a5a12b4c115f tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 200 len: 1312 microversion: 2.1 time: 0.176877 Aug 28 00:57:41.997986 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 182/372] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:41 2022] GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 1312 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:57:42.811044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-dbd1a489-3f4d-45e7-b27f-49a457f80af1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:42.811688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-dbd1a489-3f4d-45e7-b27f-49a457f80af1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:42.818933 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbd1a489-3f4d-45e7-b27f-49a457f80af1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:42.819423 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbd1a489-3f4d-45e7-b27f-49a457f80af1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:42.819858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbd1a489-3f4d-45e7-b27f-49a457f80af1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:42.820576 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbd1a489-3f4d-45e7-b27f-49a457f80af1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:42.821064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbd1a489-3f4d-45e7-b27f-49a457f80af1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:42.821491 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbd1a489-3f4d-45e7-b27f-49a457f80af1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:42.973414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dbd1a489-3f4d-45e7-b27f-49a457f80af1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:42.982066 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-dbd1a489-3f4d-45e7-b27f-49a457f80af1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1370 microversion: 2.1 time: 0.175507 Aug 28 00:57:42.982771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 191/373] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:42 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1370 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:43.019627 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-78a0a86d-0c69-4e52-8453-8f65921bad27 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:43.020260 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-78a0a86d-0c69-4e52-8453-8f65921bad27 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:43.022742 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=83,nova_api:UPDATE=1,nova_api:INSERT=4 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:57:43.028677 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78a0a86d-0c69-4e52-8453-8f65921bad27 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:43.029232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78a0a86d-0c69-4e52-8453-8f65921bad27 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:43.029663 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78a0a86d-0c69-4e52-8453-8f65921bad27 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:43.030374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78a0a86d-0c69-4e52-8453-8f65921bad27 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:43.030823 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78a0a86d-0c69-4e52-8453-8f65921bad27 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:43.031239 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78a0a86d-0c69-4e52-8453-8f65921bad27 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:43.216676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-78a0a86d-0c69-4e52-8453-8f65921bad27 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:43.225725 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-78a0a86d-0c69-4e52-8453-8f65921bad27 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 200 len: 1312 microversion: 2.1 time: 0.210557 Aug 28 00:57:43.226357 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 183/374] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:43 2022] GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 1312 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:57:43.612103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-651bc459-0b95-48e6-9ad3-545c8443cbbe req-80da8732-00fc-4b6d-a026-c940b0107c71 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "03969ac5-934e-4201-8087-0927eae2bbd2", "name": "network-vif-plugged", "status": "completed", "tag": "b9bc50e7-87a3-480a-8c62-87fe75411e15"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:57:43.623160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-651bc459-0b95-48e6-9ad3-545c8443cbbe req-80da8732-00fc-4b6d-a026-c940b0107c71 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:43.623406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-651bc459-0b95-48e6-9ad3-545c8443cbbe req-80da8732-00fc-4b6d-a026-c940b0107c71 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:43.624095 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-651bc459-0b95-48e6-9ad3-545c8443cbbe req-80da8732-00fc-4b6d-a026-c940b0107c71 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:43.641702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-651bc459-0b95-48e6-9ad3-545c8443cbbe req-80da8732-00fc-4b6d-a026-c940b0107c71 service nova] Creating event network-vif-plugged:b9bc50e7-87a3-480a-8c62-87fe75411e15 for instance 03969ac5-934e-4201-8087-0927eae2bbd2 on hv-cinder-maste Aug 28 00:57:43.648028 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-651bc459-0b95-48e6-9ad3-545c8443cbbe req-80da8732-00fc-4b6d-a026-c940b0107c71 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040944 Aug 28 00:57:43.648634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 192/375] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:57:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:57:44.004465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b09c781a-656a-4bbf-8437-99f2cb4894ac tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:44.004990 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b09c781a-656a-4bbf-8437-99f2cb4894ac tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:44.011867 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b09c781a-656a-4bbf-8437-99f2cb4894ac tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:44.012363 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b09c781a-656a-4bbf-8437-99f2cb4894ac tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:44.012800 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b09c781a-656a-4bbf-8437-99f2cb4894ac tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:44.013595 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b09c781a-656a-4bbf-8437-99f2cb4894ac tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:44.014089 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b09c781a-656a-4bbf-8437-99f2cb4894ac tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:44.014515 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b09c781a-656a-4bbf-8437-99f2cb4894ac tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:44.199751 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b09c781a-656a-4bbf-8437-99f2cb4894ac tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:44.209332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b09c781a-656a-4bbf-8437-99f2cb4894ac tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1370 microversion: 2.1 time: 0.209214 Aug 28 00:57:44.210144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 184/376] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:43 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1370 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:57:44.252560 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a337d1f7-a86e-4608-9462-1fb1322d8640 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:44.254000 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a337d1f7-a86e-4608-9462-1fb1322d8640 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:44.262131 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a337d1f7-a86e-4608-9462-1fb1322d8640 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:44.262914 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a337d1f7-a86e-4608-9462-1fb1322d8640 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:44.263782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a337d1f7-a86e-4608-9462-1fb1322d8640 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:44.265414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a337d1f7-a86e-4608-9462-1fb1322d8640 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:44.266265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a337d1f7-a86e-4608-9462-1fb1322d8640 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:44.266983 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a337d1f7-a86e-4608-9462-1fb1322d8640 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:44.287844 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:57:44.459343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a337d1f7-a86e-4608-9462-1fb1322d8640 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:44.469556 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a337d1f7-a86e-4608-9462-1fb1322d8640 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 200 len: 1496 microversion: 2.1 time: 0.224169 Aug 28 00:57:44.470508 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 193/377] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:44 2022] GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 1496 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:45.139784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0752e7a7-c144-436e-a978-06623eaadef6 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:45.140652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0752e7a7-c144-436e-a978-06623eaadef6 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:45.152016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0752e7a7-c144-436e-a978-06623eaadef6 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:45.152877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0752e7a7-c144-436e-a978-06623eaadef6 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:45.153639 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0752e7a7-c144-436e-a978-06623eaadef6 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:45.154842 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0752e7a7-c144-436e-a978-06623eaadef6 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:45.155636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0752e7a7-c144-436e-a978-06623eaadef6 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:45.156365 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0752e7a7-c144-436e-a978-06623eaadef6 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:45.186860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0752e7a7-c144-436e-a978-06623eaadef6 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Going to try to terminate instance {{(pid=120524) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 28 00:57:45.246335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-92cd6f66-5817-461a-8834-3340adc0a94d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:45.247157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-92cd6f66-5817-461a-8834-3340adc0a94d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:45.256942 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92cd6f66-5817-461a-8834-3340adc0a94d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:45.258052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92cd6f66-5817-461a-8834-3340adc0a94d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:45.258437 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92cd6f66-5817-461a-8834-3340adc0a94d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:45.259427 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92cd6f66-5817-461a-8834-3340adc0a94d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:45.260033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92cd6f66-5817-461a-8834-3340adc0a94d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:45.260550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92cd6f66-5817-461a-8834-3340adc0a94d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:45.278046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0752e7a7-c144-436e-a978-06623eaadef6 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "DELETE /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 204 len: 0 microversion: 2.1 time: 0.144454 Aug 28 00:57:45.281599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 185/378] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 00:57:45 2022] DELETE /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 0 bytes in 147 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 28 00:57:45.287602 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-143ef134-ea59-41bf-84ae-1c0c3058c08e tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:45.288214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-143ef134-ea59-41bf-84ae-1c0c3058c08e tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:45.297672 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-143ef134-ea59-41bf-84ae-1c0c3058c08e tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:45.297672 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-143ef134-ea59-41bf-84ae-1c0c3058c08e tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:45.297672 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-143ef134-ea59-41bf-84ae-1c0c3058c08e tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:45.298319 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-143ef134-ea59-41bf-84ae-1c0c3058c08e tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:45.298727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-143ef134-ea59-41bf-84ae-1c0c3058c08e tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:45.299216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-143ef134-ea59-41bf-84ae-1c0c3058c08e tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:45.499093 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-92cd6f66-5817-461a-8834-3340adc0a94d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:45.508857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-92cd6f66-5817-461a-8834-3340adc0a94d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1370 microversion: 2.1 time: 0.267380 Aug 28 00:57:45.509968 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 194/379] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:45 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1370 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:45.514756 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-143ef134-ea59-41bf-84ae-1c0c3058c08e tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:45.529513 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-143ef134-ea59-41bf-84ae-1c0c3058c08e tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 200 len: 1502 microversion: 2.1 time: 0.245860 Aug 28 00:57:45.530540 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 186/380] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:45 2022] GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 1502 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:46.532211 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-87d9149d-b8e9-4c58-9beb-863f444cfeef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:46.533143 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-87d9149d-b8e9-4c58-9beb-863f444cfeef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:46.540683 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87d9149d-b8e9-4c58-9beb-863f444cfeef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:46.541215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87d9149d-b8e9-4c58-9beb-863f444cfeef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:46.541630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87d9149d-b8e9-4c58-9beb-863f444cfeef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:46.542357 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87d9149d-b8e9-4c58-9beb-863f444cfeef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:46.542804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87d9149d-b8e9-4c58-9beb-863f444cfeef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:46.543213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87d9149d-b8e9-4c58-9beb-863f444cfeef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:46.554930 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-43c69d47-32d7-4b31-b519-d729212fd691 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:46.555991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-43c69d47-32d7-4b31-b519-d729212fd691 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:46.565065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43c69d47-32d7-4b31-b519-d729212fd691 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:46.565728 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43c69d47-32d7-4b31-b519-d729212fd691 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:46.566283 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43c69d47-32d7-4b31-b519-d729212fd691 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:46.567267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43c69d47-32d7-4b31-b519-d729212fd691 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:46.567857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43c69d47-32d7-4b31-b519-d729212fd691 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:46.568593 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43c69d47-32d7-4b31-b519-d729212fd691 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:46.777169 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-87d9149d-b8e9-4c58-9beb-863f444cfeef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:46.782676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-43c69d47-32d7-4b31-b519-d729212fd691 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:46.786285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-87d9149d-b8e9-4c58-9beb-863f444cfeef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1370 microversion: 2.1 time: 0.258847 Aug 28 00:57:46.786892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 195/381] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:46 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1370 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:46.797943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-43c69d47-32d7-4b31-b519-d729212fd691 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 200 len: 1502 microversion: 2.1 time: 0.246776 Aug 28 00:57:46.798860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 187/382] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:46 2022] GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 1502 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:47.673005 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-63fb0476-d0e5-4d13-b6de-b9dd6db446f9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-122693963", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1578003485", "security_groups": [{"name": "tempest-secgroup-smoke-1006421122"}], "block_device_mapping_v2": [{"uuid": "69e463b3-ca8b-4bc8-b32a-40b13075b24d", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "63ecdb07-b041-4401-9ffd-486e98bce044"}]}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:57:47.807143 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.network.neutron [None req-63fb0476-d0e5-4d13-b6de-b9dd6db446f9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] validate_networks() for [('63ecdb07-b041-4401-9ffd-486e98bce044', None, None, None, None, None)] {{(pid=120525) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 28 00:57:47.811727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-48e86074-3a70-459d-b04b-09175f7b6a07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:47.812726 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-48e86074-3a70-459d-b04b-09175f7b6a07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:47.820835 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48e86074-3a70-459d-b04b-09175f7b6a07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:47.821449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48e86074-3a70-459d-b04b-09175f7b6a07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:47.821948 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48e86074-3a70-459d-b04b-09175f7b6a07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:47.823100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48e86074-3a70-459d-b04b-09175f7b6a07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:47.823637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48e86074-3a70-459d-b04b-09175f7b6a07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:47.824348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48e86074-3a70-459d-b04b-09175f7b6a07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:48.077327 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-48e86074-3a70-459d-b04b-09175f7b6a07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:48.085781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-48e86074-3a70-459d-b04b-09175f7b6a07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1565 microversion: 2.1 time: 0.281571 Aug 28 00:57:48.086380 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 188/383] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:47 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1565 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:57:48.093215 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ca1fd92b-4341-4d62-b5ec-13e867c15983 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:48.093215 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ca1fd92b-4341-4d62-b5ec-13e867c15983 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:48.105450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ca1fd92b-4341-4d62-b5ec-13e867c15983 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:48.106535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ca1fd92b-4341-4d62-b5ec-13e867c15983 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:48.107473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ca1fd92b-4341-4d62-b5ec-13e867c15983 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:48.108939 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ca1fd92b-4341-4d62-b5ec-13e867c15983 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:48.111291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ca1fd92b-4341-4d62-b5ec-13e867c15983 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:48.112255 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ca1fd92b-4341-4d62-b5ec-13e867c15983 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:48.121824 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:57:48.412510 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ca1fd92b-4341-4d62-b5ec-13e867c15983 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:48.423898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ca1fd92b-4341-4d62-b5ec-13e867c15983 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 200 len: 1502 microversion: 2.1 time: 0.336123 Aug 28 00:57:48.424707 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 189/384] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:48 2022] GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 1502 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:48.433946 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0063c5aa-981a-4649-be93-7fd77030ff20 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:48.435210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0063c5aa-981a-4649-be93-7fd77030ff20 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:48.450773 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0063c5aa-981a-4649-be93-7fd77030ff20 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:48.451587 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0063c5aa-981a-4649-be93-7fd77030ff20 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:48.452128 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0063c5aa-981a-4649-be93-7fd77030ff20 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:48.453055 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0063c5aa-981a-4649-be93-7fd77030ff20 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:48.453618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0063c5aa-981a-4649-be93-7fd77030ff20 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:48.454201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0063c5aa-981a-4649-be93-7fd77030ff20 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:48.469819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-63fb0476-d0e5-4d13-b6de-b9dd6db446f9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Flavor limits 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 28 00:57:48.470503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-63fb0476-d0e5-4d13-b6de-b9dd6db446f9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Image limits 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 28 00:57:48.471284 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-63fb0476-d0e5-4d13-b6de-b9dd6db446f9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Flavor pref 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 28 00:57:48.472502 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-63fb0476-d0e5-4d13-b6de-b9dd6db446f9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Image pref 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 28 00:57:48.473241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-63fb0476-d0e5-4d13-b6de-b9dd6db446f9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 28 00:57:48.490132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63fb0476-d0e5-4d13-b6de-b9dd6db446f9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:48.491415 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63fb0476-d0e5-4d13-b6de-b9dd6db446f9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:48.492442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63fb0476-d0e5-4d13-b6de-b9dd6db446f9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:48.493978 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63fb0476-d0e5-4d13-b6de-b9dd6db446f9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:48.494632 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63fb0476-d0e5-4d13-b6de-b9dd6db446f9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:48.495274 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63fb0476-d0e5-4d13-b6de-b9dd6db446f9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:48.519994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-63fb0476-d0e5-4d13-b6de-b9dd6db446f9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Getting quotas for project 287c855ef8e241978a4d1e7cb3c7a16f. Resources: {'ram', 'instances', 'cores'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 00:57:48.528745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-63fb0476-d0e5-4d13-b6de-b9dd6db446f9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Getting quotas for user cac1ab72dcee40b19bcff4dd5a75d82d and project 287c855ef8e241978a4d1e7cb3c7a16f. Resources: {'ram', 'instances', 'cores'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 00:57:48.613681 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-63fb0476-d0e5-4d13-b6de-b9dd6db446f9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Going to run 1 instances... {{(pid=120525) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 28 00:57:48.616519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-63fb0476-d0e5-4d13-b6de-b9dd6db446f9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='69e463b3-ca8b-4bc8-b32a-40b13075b24d',volume_size=None,volume_type=None)] {{(pid=120525) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 28 00:57:48.623659 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0063c5aa-981a-4649-be93-7fd77030ff20 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.196858 Aug 28 00:57:48.624742 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 190/385] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Aug 28 00:57:48 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-interface => generated 285 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:57:48.827401 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-63fb0476-d0e5-4d13-b6de-b9dd6db446f9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.158243 Aug 28 00:57:48.829157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 196/386] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Aug 28 00:57:47 2022] POST /compute/v2.1/servers => generated 406 bytes in 1161 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Aug 28 00:57:48.845698 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-eb48f4cd-5413-4395-887f-a9d4161f5bc6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:48.847125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-eb48f4cd-5413-4395-887f-a9d4161f5bc6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:48.974873 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-eb48f4cd-5413-4395-887f-a9d4161f5bc6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:48.985621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-eb48f4cd-5413-4395-887f-a9d4161f5bc6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1094 microversion: 2.1 time: 0.144297 Aug 28 00:57:48.986552 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 191/387] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:48 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1094 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:49.315561 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-be2689b4-d26f-4860-9937-e21f9fe56cf3 req-12fb7b73-228c-441f-904f-59f533251233 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "11430c0a-d74c-48c3-a7a9-9b9922804cca", "name": "network-vif-deleted", "tag": "9addba86-2245-48de-a1af-5e38ae4ae619"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:57:49.334365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-be2689b4-d26f-4860-9937-e21f9fe56cf3 req-12fb7b73-228c-441f-904f-59f533251233 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:49.335094 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-be2689b4-d26f-4860-9937-e21f9fe56cf3 req-12fb7b73-228c-441f-904f-59f533251233 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:49.335864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-be2689b4-d26f-4860-9937-e21f9fe56cf3 req-12fb7b73-228c-441f-904f-59f533251233 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:49.360627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-be2689b4-d26f-4860-9937-e21f9fe56cf3 req-12fb7b73-228c-441f-904f-59f533251233 service nova] Creating event network-vif-deleted:9addba86-2245-48de-a1af-5e38ae4ae619 for instance 11430c0a-d74c-48c3-a7a9-9b9922804cca on hv-cinder-maste Aug 28 00:57:49.370215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-be2689b4-d26f-4860-9937-e21f9fe56cf3 req-12fb7b73-228c-441f-904f-59f533251233 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062409 Aug 28 00:57:49.371739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 197/388] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:57:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:57:49.456520 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0aa75c63-9230-40c3-be5a-85696ac48f3b tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:49.457467 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0aa75c63-9230-40c3-be5a-85696ac48f3b tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:49.464869 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0aa75c63-9230-40c3-be5a-85696ac48f3b tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:49.465792 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0aa75c63-9230-40c3-be5a-85696ac48f3b tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:49.466597 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0aa75c63-9230-40c3-be5a-85696ac48f3b tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:49.468889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0aa75c63-9230-40c3-be5a-85696ac48f3b tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:49.469752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0aa75c63-9230-40c3-be5a-85696ac48f3b tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:49.470274 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0aa75c63-9230-40c3-be5a-85696ac48f3b tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:49.568668 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0aa75c63-9230-40c3-be5a-85696ac48f3b tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:49.583208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0aa75c63-9230-40c3-be5a-85696ac48f3b tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 200 len: 1460 microversion: 2.1 time: 0.131247 Aug 28 00:57:49.584231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 192/389] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:49 2022] GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 1460 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:50.025877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-adf4f495-1cf1-43df-9b41-8f1ce6f280f2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:50.027019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-adf4f495-1cf1-43df-9b41-8f1ce6f280f2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:50.037054 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-adf4f495-1cf1-43df-9b41-8f1ce6f280f2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:50.038248 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-adf4f495-1cf1-43df-9b41-8f1ce6f280f2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:50.039306 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-adf4f495-1cf1-43df-9b41-8f1ce6f280f2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:50.040632 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-adf4f495-1cf1-43df-9b41-8f1ce6f280f2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:50.041485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-adf4f495-1cf1-43df-9b41-8f1ce6f280f2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:50.042263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-adf4f495-1cf1-43df-9b41-8f1ce6f280f2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:50.135641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-adf4f495-1cf1-43df-9b41-8f1ce6f280f2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:50.149107 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-adf4f495-1cf1-43df-9b41-8f1ce6f280f2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1136 microversion: 2.1 time: 0.131342 Aug 28 00:57:50.150215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 198/390] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:50 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1136 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:50.391309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-2b8eb69e-96ad-4e9a-9118-086595c0138b req-6e8452a6-864b-4235-88f0-dbbbfa792033 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "03969ac5-934e-4201-8087-0927eae2bbd2", "tag": "b9bc50e7-87a3-480a-8c62-87fe75411e15"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:57:50.410140 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-2b8eb69e-96ad-4e9a-9118-086595c0138b req-6e8452a6-864b-4235-88f0-dbbbfa792033 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:50.411125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-2b8eb69e-96ad-4e9a-9118-086595c0138b req-6e8452a6-864b-4235-88f0-dbbbfa792033 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:50.411872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-2b8eb69e-96ad-4e9a-9118-086595c0138b req-6e8452a6-864b-4235-88f0-dbbbfa792033 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:50.425197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-2b8eb69e-96ad-4e9a-9118-086595c0138b req-6e8452a6-864b-4235-88f0-dbbbfa792033 service nova] Creating event network-changed:b9bc50e7-87a3-480a-8c62-87fe75411e15 for instance 03969ac5-934e-4201-8087-0927eae2bbd2 on hv-cinder-maste Aug 28 00:57:50.432005 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-2b8eb69e-96ad-4e9a-9118-086595c0138b req-6e8452a6-864b-4235-88f0-dbbbfa792033 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047644 Aug 28 00:57:50.433266 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 193/391] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:57:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:57:50.612446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a011eb41-fc00-44fd-af19-a05b92e48615 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:50.614261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a011eb41-fc00-44fd-af19-a05b92e48615 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:50.623041 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a011eb41-fc00-44fd-af19-a05b92e48615 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:50.623679 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a011eb41-fc00-44fd-af19-a05b92e48615 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:50.624218 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a011eb41-fc00-44fd-af19-a05b92e48615 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:50.625104 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a011eb41-fc00-44fd-af19-a05b92e48615 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:50.625660 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a011eb41-fc00-44fd-af19-a05b92e48615 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:50.626186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a011eb41-fc00-44fd-af19-a05b92e48615 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:50.671571 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a011eb41-fc00-44fd-af19-a05b92e48615 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:50.672406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a011eb41-fc00-44fd-af19-a05b92e48615 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:50.673176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a011eb41-fc00-44fd-af19-a05b92e48615 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:50.673956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.objects.instance [None req-a011eb41-fc00-44fd-af19-a05b92e48615 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lazy-loading 'fault' on Instance uuid 11430c0a-d74c-48c3-a7a9-9b9922804cca {{(pid=120525) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 00:57:51.013810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a011eb41-fc00-44fd-af19-a05b92e48615 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '59b04859082547149bf2776aefa63b43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315d6d72ee5844edb203b740ab2d08ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:51.028614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a011eb41-fc00-44fd-af19-a05b92e48615 tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 200 len: 1302 microversion: 2.1 time: 0.421144 Aug 28 00:57:51.049681 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 199/392] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:50 2022] GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 1302 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:51.188490 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c30f05ed-5e6e-4676-bc20-d4e05a6751fe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:51.189741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c30f05ed-5e6e-4676-bc20-d4e05a6751fe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:51.201506 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c30f05ed-5e6e-4676-bc20-d4e05a6751fe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:51.202779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c30f05ed-5e6e-4676-bc20-d4e05a6751fe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:51.203816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c30f05ed-5e6e-4676-bc20-d4e05a6751fe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:51.205436 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c30f05ed-5e6e-4676-bc20-d4e05a6751fe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:51.206553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c30f05ed-5e6e-4676-bc20-d4e05a6751fe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:51.207529 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c30f05ed-5e6e-4676-bc20-d4e05a6751fe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:51.299226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c30f05ed-5e6e-4676-bc20-d4e05a6751fe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:51.309985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c30f05ed-5e6e-4676-bc20-d4e05a6751fe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1136 microversion: 2.1 time: 0.129654 Aug 28 00:57:51.310795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 194/393] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:51 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1136 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:52.059791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2918766a-e51b-4eec-b037-cac4216d302c tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:52.061302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2918766a-e51b-4eec-b037-cac4216d302c tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] [instance: 11430c0a-d74c-48c3-a7a9-9b9922804cca] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:52.070161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2918766a-e51b-4eec-b037-cac4216d302c tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:52.070992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2918766a-e51b-4eec-b037-cac4216d302c tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:52.071701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2918766a-e51b-4eec-b037-cac4216d302c tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:52.072991 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2918766a-e51b-4eec-b037-cac4216d302c tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:52.073707 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2918766a-e51b-4eec-b037-cac4216d302c tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:52.074264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2918766a-e51b-4eec-b037-cac4216d302c tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:52.101160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-2918766a-e51b-4eec-b037-cac4216d302c tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] HTTP exception thrown: Instance 11430c0a-d74c-48c3-a7a9-9b9922804cca could not be found. Aug 28 00:57:52.104074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2918766a-e51b-4eec-b037-cac4216d302c tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Request method failure captured: Aug 28 00:57:52.104074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca HTTP/1.1 Aug 28 00:57:52.104074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 00:57:52.104074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 00:57:52.104074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 00:57:52.104074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 00:57:52.104074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 00:57:52.104074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 00:57:52.104074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 00:57:52.104074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 00:57:52.104074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCr1cMoPUe0bkvwrgCXK1x-PXWG4dAWe3fu6cufayhImfmbLqvwWIogu97x1WAtOd9qulHdsmmTUCY1T5fZyOq4f_W03DfmAUWrbP9Hg7mLNiCZWh8JrBi7GlyYKfOqvl609NQaqGzXIftLkBDRoFi2q9yigsmw0U52NRuE3tVYaIv-k Aug 28 00:57:52.104074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 00:57:52.104074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 00:57:52.104074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 00:57:52.104074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 00:57:52.104074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 00:57:52.104074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 00:57:52.104074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 315d6d72ee5844edb203b740ab2d08ce Aug 28 00:57:52.104074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-VolumesActionsTest-1664214821 Aug 28 00:57:52.104074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 00:57:52.104074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 00:57:52.107081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_315d6d72ee5844edb203b740ab2d08ce", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/315d6d72ee5844edb203b740ab2d08ce"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/315d6d72ee5844edb203b740ab2d08ce"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/315d6d72ee5844edb203b740ab2d08ce"}]}] Aug 28 00:57:52.107081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-VolumesActionsTest-1664214821 Aug 28 00:57:52.107081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 315d6d72ee5844edb203b740ab2d08ce Aug 28 00:57:52.107081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-VolumesActionsTest-1664214821 Aug 28 00:57:52.107081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-VolumesActionsTest-1664214821-project Aug 28 00:57:52.107081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 00:57:52.107081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 00:57:52.107081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: 59b04859082547149bf2776aefa63b43 Aug 28 00:57:52.107081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-VolumesActionsTest-1664214821-project Aug 28 00:57:52.107081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 00:57:52.107081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Instance 11430c0a-d74c-48c3-a7a9-9b9922804cca could not be found. Aug 28 00:57:52.107081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'id': '11430c0a-d74c-48c3-a7a9-9b9922804cca'} Aug 28 00:57:52.108688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 00:57:52.108688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:57:52.108688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 00:57:52.108688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 00:57:52.108688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 00:57:52.108688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 00:57:52.108688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 00:57:52.108688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 00:57:52.108688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 00:57:52.108688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 00:57:52.108688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 11430c0a-d74c-48c3-a7a9-9b9922804cca could not be found. Aug 28 00:57:52.108688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:57:52.108688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:57:52.108688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:57:52.108688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:57:52.108688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:57:52.108688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:57:52.108688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 00:57:52.108688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 00:57:52.108688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 00:57:52.111360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 00:57:52.111360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 00:57:52.111360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 00:57:52.111360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 00:57:52.111360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 00:57:52.111360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 00:57:52.111360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 00:57:52.111360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 11430c0a-d74c-48c3-a7a9-9b9922804cca could not be found. Aug 28 00:57:52.111360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:57:52.111360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 00:57:52.111360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:57:52.111360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 00:57:52.111360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 00:57:52.111360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 00:57:52.111360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 00:57:52.111360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 00:57:52.111360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 11430c0a-d74c-48c3-a7a9-9b9922804cca could not be found. Aug 28 00:57:52.111360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 00:57:52.113782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2918766a-e51b-4eec-b037-cac4216d302c tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] Returning 404 to user: Instance 11430c0a-d74c-48c3-a7a9-9b9922804cca could not be found. {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 00:57:52.113782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2918766a-e51b-4eec-b037-cac4216d302c tempest-VolumesActionsTest-1664214821 tempest-VolumesActionsTest-1664214821-project] 10.222.0.42 "GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca" status: 404 len: 111 microversion: 2.1 time: 0.052871 Aug 28 00:57:52.113782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 200/394] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:52 2022] GET /compute/v2.1/servers/11430c0a-d74c-48c3-a7a9-9b9922804cca => generated 111 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 28 00:57:52.334272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ecae9d35-5905-4f7f-9f6f-4d7824f84f0b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:52.336082 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ecae9d35-5905-4f7f-9f6f-4d7824f84f0b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:52.350708 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ecae9d35-5905-4f7f-9f6f-4d7824f84f0b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:52.351258 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ecae9d35-5905-4f7f-9f6f-4d7824f84f0b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:52.351838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ecae9d35-5905-4f7f-9f6f-4d7824f84f0b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:52.353230 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ecae9d35-5905-4f7f-9f6f-4d7824f84f0b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:52.353815 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ecae9d35-5905-4f7f-9f6f-4d7824f84f0b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:52.354505 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ecae9d35-5905-4f7f-9f6f-4d7824f84f0b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:52.450376 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ecae9d35-5905-4f7f-9f6f-4d7824f84f0b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:52.460708 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ecae9d35-5905-4f7f-9f6f-4d7824f84f0b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1192 microversion: 2.1 time: 0.131511 Aug 28 00:57:52.462001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 195/395] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:52 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1192 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:53.496226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-71dbefc1-37aa-4e07-b853-148c315f88a5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:53.497257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-71dbefc1-37aa-4e07-b853-148c315f88a5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:53.506425 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-71dbefc1-37aa-4e07-b853-148c315f88a5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:53.507051 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-71dbefc1-37aa-4e07-b853-148c315f88a5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:53.507653 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-71dbefc1-37aa-4e07-b853-148c315f88a5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:53.508935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-71dbefc1-37aa-4e07-b853-148c315f88a5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:53.509720 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-71dbefc1-37aa-4e07-b853-148c315f88a5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:53.510247 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-71dbefc1-37aa-4e07-b853-148c315f88a5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:53.608228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-71dbefc1-37aa-4e07-b853-148c315f88a5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:53.622957 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-71dbefc1-37aa-4e07-b853-148c315f88a5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1210 microversion: 2.1 time: 0.132302 Aug 28 00:57:53.623984 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 201/396] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:53 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1210 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:54.647145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-af55a8b3-f04a-40fa-a730-1682f14c7086 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:54.647831 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-af55a8b3-f04a-40fa-a730-1682f14c7086 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:54.654939 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af55a8b3-f04a-40fa-a730-1682f14c7086 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:54.655429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af55a8b3-f04a-40fa-a730-1682f14c7086 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:54.655921 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af55a8b3-f04a-40fa-a730-1682f14c7086 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:54.656850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af55a8b3-f04a-40fa-a730-1682f14c7086 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:54.657341 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af55a8b3-f04a-40fa-a730-1682f14c7086 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:54.657769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af55a8b3-f04a-40fa-a730-1682f14c7086 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:54.830518 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-af55a8b3-f04a-40fa-a730-1682f14c7086 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:54.839371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-af55a8b3-f04a-40fa-a730-1682f14c7086 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1278 microversion: 2.1 time: 0.196446 Aug 28 00:57:54.839960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 196/397] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:54 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1278 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:55.865894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ac5af5a6-90f4-46c5-ae90-37815548fbb8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:55.866936 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ac5af5a6-90f4-46c5-ae90-37815548fbb8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:55.878136 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac5af5a6-90f4-46c5-ae90-37815548fbb8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:55.878979 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac5af5a6-90f4-46c5-ae90-37815548fbb8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:55.879743 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac5af5a6-90f4-46c5-ae90-37815548fbb8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:55.881089 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac5af5a6-90f4-46c5-ae90-37815548fbb8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:55.881875 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac5af5a6-90f4-46c5-ae90-37815548fbb8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:55.882596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac5af5a6-90f4-46c5-ae90-37815548fbb8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:56.049461 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ac5af5a6-90f4-46c5-ae90-37815548fbb8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:56.058122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ac5af5a6-90f4-46c5-ae90-37815548fbb8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1278 microversion: 2.1 time: 0.200320 Aug 28 00:57:56.058725 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 202/398] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:55 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1278 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:57.081226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-da6701ec-d5a3-44bc-b288-ae249f05e0a6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:57.082128 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-da6701ec-d5a3-44bc-b288-ae249f05e0a6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:57.090275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da6701ec-d5a3-44bc-b288-ae249f05e0a6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:57.090786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da6701ec-d5a3-44bc-b288-ae249f05e0a6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:57.091247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da6701ec-d5a3-44bc-b288-ae249f05e0a6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:57.092085 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da6701ec-d5a3-44bc-b288-ae249f05e0a6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:57.092547 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da6701ec-d5a3-44bc-b288-ae249f05e0a6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:57.093068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da6701ec-d5a3-44bc-b288-ae249f05e0a6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:57.276999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-da6701ec-d5a3-44bc-b288-ae249f05e0a6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:57.285765 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-da6701ec-d5a3-44bc-b288-ae249f05e0a6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1278 microversion: 2.1 time: 0.208517 Aug 28 00:57:57.286411 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 197/399] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:57 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1278 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:58.306591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2b4c1fdc-6ffd-42de-9a87-34cd1d06bdbe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:58.307369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2b4c1fdc-6ffd-42de-9a87-34cd1d06bdbe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:58.314820 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2b4c1fdc-6ffd-42de-9a87-34cd1d06bdbe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:58.315328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2b4c1fdc-6ffd-42de-9a87-34cd1d06bdbe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:58.315759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2b4c1fdc-6ffd-42de-9a87-34cd1d06bdbe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:58.316477 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2b4c1fdc-6ffd-42de-9a87-34cd1d06bdbe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:58.316937 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2b4c1fdc-6ffd-42de-9a87-34cd1d06bdbe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:58.317375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2b4c1fdc-6ffd-42de-9a87-34cd1d06bdbe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:58.507407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=2 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:57:58.541576 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=6 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:57:58.647416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2b4c1fdc-6ffd-42de-9a87-34cd1d06bdbe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:57:58.657232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2b4c1fdc-6ffd-42de-9a87-34cd1d06bdbe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1278 microversion: 2.1 time: 0.354526 Aug 28 00:57:58.658232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 203/400] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:58 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1278 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:57:58.758757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-807d6a13-d5c8-44cc-bfe6-c3144ec52768 req-933001ab-8bcc-46a3-a8c2-989fbd8a7692 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1cc5e59c-500c-421e-920c-c08b8ae3a206", "tag": "de0f5772-f2b1-42a3-84e4-e24d81b36597"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:57:58.769517 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-807d6a13-d5c8-44cc-bfe6-c3144ec52768 req-933001ab-8bcc-46a3-a8c2-989fbd8a7692 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:58.769999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-807d6a13-d5c8-44cc-bfe6-c3144ec52768 req-933001ab-8bcc-46a3-a8c2-989fbd8a7692 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:58.770426 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-807d6a13-d5c8-44cc-bfe6-c3144ec52768 req-933001ab-8bcc-46a3-a8c2-989fbd8a7692 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:58.782905 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-807d6a13-d5c8-44cc-bfe6-c3144ec52768 req-933001ab-8bcc-46a3-a8c2-989fbd8a7692 service nova] Creating event network-changed:de0f5772-f2b1-42a3-84e4-e24d81b36597 for instance 1cc5e59c-500c-421e-920c-c08b8ae3a206 on hv-cinder-maste Aug 28 00:57:58.788866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-807d6a13-d5c8-44cc-bfe6-c3144ec52768 req-933001ab-8bcc-46a3-a8c2-989fbd8a7692 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033893 Aug 28 00:57:58.789526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 198/401] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:57:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:57:59.698574 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f148fcf5-e660-420a-a123-5c9b0aaa7d80 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:57:59.700079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f148fcf5-e660-420a-a123-5c9b0aaa7d80 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:57:59.723519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f148fcf5-e660-420a-a123-5c9b0aaa7d80 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:59.724344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f148fcf5-e660-420a-a123-5c9b0aaa7d80 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:59.725163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f148fcf5-e660-420a-a123-5c9b0aaa7d80 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:57:59.726802 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f148fcf5-e660-420a-a123-5c9b0aaa7d80 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:57:59.728163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f148fcf5-e660-420a-a123-5c9b0aaa7d80 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:57:59.728930 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f148fcf5-e660-420a-a123-5c9b0aaa7d80 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:00.201246 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f148fcf5-e660-420a-a123-5c9b0aaa7d80 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:00.220268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f148fcf5-e660-420a-a123-5c9b0aaa7d80 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1266 microversion: 2.1 time: 0.530713 Aug 28 00:58:00.222288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 204/402] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:57:59 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1266 bytes in 534 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:58:01.248101 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=2 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:58:01.270141 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-dc22d16e-7396-4ec3-baca-cecb5cce4375 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:01.272533 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-dc22d16e-7396-4ec3-baca-cecb5cce4375 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:01.283249 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dc22d16e-7396-4ec3-baca-cecb5cce4375 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:01.284630 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dc22d16e-7396-4ec3-baca-cecb5cce4375 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:01.285599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dc22d16e-7396-4ec3-baca-cecb5cce4375 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:01.286955 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dc22d16e-7396-4ec3-baca-cecb5cce4375 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:01.287802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dc22d16e-7396-4ec3-baca-cecb5cce4375 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:01.288730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dc22d16e-7396-4ec3-baca-cecb5cce4375 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:01.522965 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-dc22d16e-7396-4ec3-baca-cecb5cce4375 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:01.535857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-dc22d16e-7396-4ec3-baca-cecb5cce4375 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1266 microversion: 2.1 time: 0.289721 Aug 28 00:58:01.536856 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 199/403] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:01 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1266 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:02.580712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-43875b37-71cb-4ed1-9ddd-99dddb212ef9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:02.583414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-43875b37-71cb-4ed1-9ddd-99dddb212ef9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:02.596760 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-43875b37-71cb-4ed1-9ddd-99dddb212ef9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:02.598703 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-43875b37-71cb-4ed1-9ddd-99dddb212ef9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:02.599845 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-43875b37-71cb-4ed1-9ddd-99dddb212ef9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:02.601772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-43875b37-71cb-4ed1-9ddd-99dddb212ef9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:02.603439 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-43875b37-71cb-4ed1-9ddd-99dddb212ef9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:02.604878 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-43875b37-71cb-4ed1-9ddd-99dddb212ef9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:02.889924 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-43875b37-71cb-4ed1-9ddd-99dddb212ef9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:02.916174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-43875b37-71cb-4ed1-9ddd-99dddb212ef9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1266 microversion: 2.1 time: 0.344556 Aug 28 00:58:02.918789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 205/404] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:02 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1266 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:03.955833 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bed0b44b-1566-4873-9c73-96f6cc6f539f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:03.959323 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bed0b44b-1566-4873-9c73-96f6cc6f539f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:03.975239 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bed0b44b-1566-4873-9c73-96f6cc6f539f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:03.976608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bed0b44b-1566-4873-9c73-96f6cc6f539f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:03.977473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bed0b44b-1566-4873-9c73-96f6cc6f539f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:03.979125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bed0b44b-1566-4873-9c73-96f6cc6f539f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:03.980076 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bed0b44b-1566-4873-9c73-96f6cc6f539f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:03.980894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bed0b44b-1566-4873-9c73-96f6cc6f539f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:04.268383 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bed0b44b-1566-4873-9c73-96f6cc6f539f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:04.277581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bed0b44b-1566-4873-9c73-96f6cc6f539f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1266 microversion: 2.1 time: 0.334045 Aug 28 00:58:04.278204 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 200/405] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:03 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1266 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:04.594376 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6f0cb242-065c-4d7c-bf29-64bbbc05e46a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:04.596392 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6f0cb242-065c-4d7c-bf29-64bbbc05e46a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:04.601915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=61,nova_api:INSERT=3 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:58:04.610401 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6f0cb242-065c-4d7c-bf29-64bbbc05e46a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:04.611787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6f0cb242-065c-4d7c-bf29-64bbbc05e46a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:04.612932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6f0cb242-065c-4d7c-bf29-64bbbc05e46a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:04.614233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6f0cb242-065c-4d7c-bf29-64bbbc05e46a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:04.615052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6f0cb242-065c-4d7c-bf29-64bbbc05e46a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:04.615950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6f0cb242-065c-4d7c-bf29-64bbbc05e46a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:04.659600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6f0cb242-065c-4d7c-bf29-64bbbc05e46a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/ips" status: 200 len: 151 microversion: 2.20 time: 0.072631 Aug 28 00:58:04.667427 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 206/406] 10.222.0.42 () {60 vars in 1289 bytes} [Sun Aug 28 00:58:04 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/ips => generated 151 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 00:58:05.314799 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5472d194-838c-418c-a587-4787b5f84ae4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:05.315780 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5472d194-838c-418c-a587-4787b5f84ae4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:05.326696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5472d194-838c-418c-a587-4787b5f84ae4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:05.327745 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5472d194-838c-418c-a587-4787b5f84ae4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:05.328406 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5472d194-838c-418c-a587-4787b5f84ae4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:05.329681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5472d194-838c-418c-a587-4787b5f84ae4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:05.330441 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5472d194-838c-418c-a587-4787b5f84ae4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:05.331175 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5472d194-838c-418c-a587-4787b5f84ae4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:05.538825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5472d194-838c-418c-a587-4787b5f84ae4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:05.553087 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5472d194-838c-418c-a587-4787b5f84ae4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1266 microversion: 2.1 time: 0.245502 Aug 28 00:58:05.564993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 201/407] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:05 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1266 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:06.582767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e9454d00-f4bf-4035-98fa-4f9b654d66b0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:06.584467 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e9454d00-f4bf-4035-98fa-4f9b654d66b0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:06.593364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e9454d00-f4bf-4035-98fa-4f9b654d66b0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:06.597809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e9454d00-f4bf-4035-98fa-4f9b654d66b0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:06.598721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e9454d00-f4bf-4035-98fa-4f9b654d66b0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:06.600164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e9454d00-f4bf-4035-98fa-4f9b654d66b0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:06.601141 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e9454d00-f4bf-4035-98fa-4f9b654d66b0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:06.602269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e9454d00-f4bf-4035-98fa-4f9b654d66b0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:06.830569 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e9454d00-f4bf-4035-98fa-4f9b654d66b0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:06.839880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e9454d00-f4bf-4035-98fa-4f9b654d66b0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1266 microversion: 2.1 time: 0.261370 Aug 28 00:58:06.840650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 207/408] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:06 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1266 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:07.077391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6b92dd03-ecfd-4b1b-9013-105fa10069c1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:58:07.078292 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6b92dd03-ecfd-4b1b-9013-105fa10069c1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:07.086692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b92dd03-ecfd-4b1b-9013-105fa10069c1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:07.087183 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b92dd03-ecfd-4b1b-9013-105fa10069c1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:07.087614 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b92dd03-ecfd-4b1b-9013-105fa10069c1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:07.088365 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b92dd03-ecfd-4b1b-9013-105fa10069c1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:07.088830 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b92dd03-ecfd-4b1b-9013-105fa10069c1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:07.089288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b92dd03-ecfd-4b1b-9013-105fa10069c1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:07.120511 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.objects.instance [None req-6b92dd03-ecfd-4b1b-9013-105fa10069c1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lazy-loading 'flavor' on Instance uuid dd61123b-7a96-4c9e-936c-5794a4fdd7df {{(pid=120524) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 00:58:07.400301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6b92dd03-ecfd-4b1b-9013-105fa10069c1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "POST /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/action" status: 202 len: 0 microversion: 2.20 time: 0.328205 Aug 28 00:58:07.400936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 202/409] 10.222.0.42 () {62 vars in 1319 bytes} [Sun Aug 28 00:58:07 2022] POST /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/action => generated 0 bytes in 330 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Aug 28 00:58:07.412926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ab071168-117e-4276-9743-51d6caa6c3bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:07.413989 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ab071168-117e-4276-9743-51d6caa6c3bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:07.423580 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab071168-117e-4276-9743-51d6caa6c3bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:07.424411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab071168-117e-4276-9743-51d6caa6c3bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:07.425197 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab071168-117e-4276-9743-51d6caa6c3bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:07.426371 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab071168-117e-4276-9743-51d6caa6c3bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:07.427144 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab071168-117e-4276-9743-51d6caa6c3bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:07.427860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab071168-117e-4276-9743-51d6caa6c3bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:07.664767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ab071168-117e-4276-9743-51d6caa6c3bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:07.681161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ab071168-117e-4276-9743-51d6caa6c3bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:07.685103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ab071168-117e-4276-9743-51d6caa6c3bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:07.687909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ab071168-117e-4276-9743-51d6caa6c3bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1721 microversion: 2.20 time: 0.281093 Aug 28 00:58:07.689375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 208/410] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:07 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1721 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:07.875531 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bebc7075-1c1a-4432-a466-b42acb170242 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:07.876336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bebc7075-1c1a-4432-a466-b42acb170242 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:07.884108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bebc7075-1c1a-4432-a466-b42acb170242 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:07.884656 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bebc7075-1c1a-4432-a466-b42acb170242 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:07.885127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bebc7075-1c1a-4432-a466-b42acb170242 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:07.885866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bebc7075-1c1a-4432-a466-b42acb170242 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:07.886361 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bebc7075-1c1a-4432-a466-b42acb170242 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:07.886791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bebc7075-1c1a-4432-a466-b42acb170242 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:08.152160 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bebc7075-1c1a-4432-a466-b42acb170242 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:08.163393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bebc7075-1c1a-4432-a466-b42acb170242 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1266 microversion: 2.1 time: 0.292491 Aug 28 00:58:08.163999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 203/411] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:07 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1266 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:08.726933 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5b9d0ee1-0f09-4ec5-af4c-43b93d7300f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:08.729095 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5b9d0ee1-0f09-4ec5-af4c-43b93d7300f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:08.742501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b9d0ee1-0f09-4ec5-af4c-43b93d7300f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:08.742803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b9d0ee1-0f09-4ec5-af4c-43b93d7300f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:08.743322 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b9d0ee1-0f09-4ec5-af4c-43b93d7300f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:08.743779 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b9d0ee1-0f09-4ec5-af4c-43b93d7300f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:08.744420 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b9d0ee1-0f09-4ec5-af4c-43b93d7300f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:08.745025 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b9d0ee1-0f09-4ec5-af4c-43b93d7300f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:08.975964 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5b9d0ee1-0f09-4ec5-af4c-43b93d7300f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:08.992855 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5b9d0ee1-0f09-4ec5-af4c-43b93d7300f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:08.995030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5b9d0ee1-0f09-4ec5-af4c-43b93d7300f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:08.996668 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5b9d0ee1-0f09-4ec5-af4c-43b93d7300f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1721 microversion: 2.20 time: 0.276206 Aug 28 00:58:08.997827 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 209/412] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:08 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1721 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:09.188649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c37237ff-cb62-49ee-8999-4cf4aac346c5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:09.189993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c37237ff-cb62-49ee-8999-4cf4aac346c5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:09.201450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c37237ff-cb62-49ee-8999-4cf4aac346c5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:09.202724 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c37237ff-cb62-49ee-8999-4cf4aac346c5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:09.203804 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c37237ff-cb62-49ee-8999-4cf4aac346c5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:09.205324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c37237ff-cb62-49ee-8999-4cf4aac346c5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:09.206490 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c37237ff-cb62-49ee-8999-4cf4aac346c5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:09.207358 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c37237ff-cb62-49ee-8999-4cf4aac346c5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:09.444547 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c37237ff-cb62-49ee-8999-4cf4aac346c5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:09.459484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c37237ff-cb62-49ee-8999-4cf4aac346c5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1266 microversion: 2.1 time: 0.276729 Aug 28 00:58:09.460474 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 204/413] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:09 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1266 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:10.025730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b1b11f6d-6cf3-4574-8ae0-00703f64bb93 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:10.029673 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b1b11f6d-6cf3-4574-8ae0-00703f64bb93 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:10.053484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b1b11f6d-6cf3-4574-8ae0-00703f64bb93 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:10.053484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b1b11f6d-6cf3-4574-8ae0-00703f64bb93 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:10.053484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b1b11f6d-6cf3-4574-8ae0-00703f64bb93 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:10.054997 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b1b11f6d-6cf3-4574-8ae0-00703f64bb93 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:10.055860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b1b11f6d-6cf3-4574-8ae0-00703f64bb93 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:10.056548 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b1b11f6d-6cf3-4574-8ae0-00703f64bb93 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:10.479350 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b1b11f6d-6cf3-4574-8ae0-00703f64bb93 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:10.498158 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-813168e9-df09-4cba-ba26-a8e2f4f832c7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:10.501580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-813168e9-df09-4cba-ba26-a8e2f4f832c7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:10.514987 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-813168e9-df09-4cba-ba26-a8e2f4f832c7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:10.516115 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-813168e9-df09-4cba-ba26-a8e2f4f832c7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:10.517184 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-813168e9-df09-4cba-ba26-a8e2f4f832c7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:10.518980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-813168e9-df09-4cba-ba26-a8e2f4f832c7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:10.520447 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-813168e9-df09-4cba-ba26-a8e2f4f832c7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:10.521780 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-813168e9-df09-4cba-ba26-a8e2f4f832c7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:10.524471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b1b11f6d-6cf3-4574-8ae0-00703f64bb93 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:10.527598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b1b11f6d-6cf3-4574-8ae0-00703f64bb93 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:10.530048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b1b11f6d-6cf3-4574-8ae0-00703f64bb93 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1721 microversion: 2.20 time: 0.512066 Aug 28 00:58:10.533685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 210/414] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:10 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1721 bytes in 517 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:10.792271 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-813168e9-df09-4cba-ba26-a8e2f4f832c7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:10.808641 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-813168e9-df09-4cba-ba26-a8e2f4f832c7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1266 microversion: 2.1 time: 0.319549 Aug 28 00:58:10.829008 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 205/415] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:10 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1266 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:58:11.566931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d6ed9ed8-4805-47c3-ac77-5d7f429d7b37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:11.568023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d6ed9ed8-4805-47c3-ac77-5d7f429d7b37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:11.578244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6ed9ed8-4805-47c3-ac77-5d7f429d7b37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:11.579395 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6ed9ed8-4805-47c3-ac77-5d7f429d7b37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:11.580004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6ed9ed8-4805-47c3-ac77-5d7f429d7b37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:11.580878 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6ed9ed8-4805-47c3-ac77-5d7f429d7b37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:11.581514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6ed9ed8-4805-47c3-ac77-5d7f429d7b37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:11.582074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6ed9ed8-4805-47c3-ac77-5d7f429d7b37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:11.803445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d6ed9ed8-4805-47c3-ac77-5d7f429d7b37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:11.820267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d6ed9ed8-4805-47c3-ac77-5d7f429d7b37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:11.823643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d6ed9ed8-4805-47c3-ac77-5d7f429d7b37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:11.826034 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d6ed9ed8-4805-47c3-ac77-5d7f429d7b37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1721 microversion: 2.20 time: 0.265488 Aug 28 00:58:11.827075 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 211/416] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:11 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1721 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:11.853934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c0d3a018-832d-4cc9-aacd-88c53d5cdb1e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:11.856740 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c0d3a018-832d-4cc9-aacd-88c53d5cdb1e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:11.867338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0d3a018-832d-4cc9-aacd-88c53d5cdb1e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:11.868558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0d3a018-832d-4cc9-aacd-88c53d5cdb1e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:11.869444 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0d3a018-832d-4cc9-aacd-88c53d5cdb1e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:11.871503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0d3a018-832d-4cc9-aacd-88c53d5cdb1e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:11.873045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0d3a018-832d-4cc9-aacd-88c53d5cdb1e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:11.874147 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0d3a018-832d-4cc9-aacd-88c53d5cdb1e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:11.909934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:58:11.925714 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3484652f-cd40-481e-9ddf-dad957917277 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:11.926986 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3484652f-cd40-481e-9ddf-dad957917277 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:11.940151 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3484652f-cd40-481e-9ddf-dad957917277 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:11.941722 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3484652f-cd40-481e-9ddf-dad957917277 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:11.942921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3484652f-cd40-481e-9ddf-dad957917277 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:11.944429 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3484652f-cd40-481e-9ddf-dad957917277 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:11.945767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3484652f-cd40-481e-9ddf-dad957917277 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:11.946815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3484652f-cd40-481e-9ddf-dad957917277 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:11.956780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:58:11.997433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3484652f-cd40-481e-9ddf-dad957917277 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/ips" status: 200 len: 151 microversion: 2.1 time: 0.079013 Aug 28 00:58:12.008106 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 212/417] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Aug 28 00:58:11 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/ips => generated 151 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:58:12.178714 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c0d3a018-832d-4cc9-aacd-88c53d5cdb1e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:12.192946 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c0d3a018-832d-4cc9-aacd-88c53d5cdb1e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1266 microversion: 2.1 time: 0.347352 Aug 28 00:58:12.195677 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 206/418] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:11 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1266 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:12.870371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-31c7102e-8919-43a3-8329-ca115e355cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:12.874625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b09cebfa-2273-4bcd-a587-4a70be2fad74 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:12.877235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-31c7102e-8919-43a3-8329-ca115e355cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:12.878570 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b09cebfa-2273-4bcd-a587-4a70be2fad74 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:12.891333 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b09cebfa-2273-4bcd-a587-4a70be2fad74 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:12.892013 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b09cebfa-2273-4bcd-a587-4a70be2fad74 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:12.892570 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b09cebfa-2273-4bcd-a587-4a70be2fad74 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:12.893447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b09cebfa-2273-4bcd-a587-4a70be2fad74 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:12.893880 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31c7102e-8919-43a3-8329-ca115e355cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:12.894115 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b09cebfa-2273-4bcd-a587-4a70be2fad74 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:12.895433 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31c7102e-8919-43a3-8329-ca115e355cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:12.895653 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31c7102e-8919-43a3-8329-ca115e355cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:12.896880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b09cebfa-2273-4bcd-a587-4a70be2fad74 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:12.897226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31c7102e-8919-43a3-8329-ca115e355cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:12.898086 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31c7102e-8919-43a3-8329-ca115e355cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:12.898842 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31c7102e-8919-43a3-8329-ca115e355cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:12.929325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b09cebfa-2273-4bcd-a587-4a70be2fad74 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/ips" status: 200 len: 145 microversion: 2.1 time: 0.075332 Aug 28 00:58:12.930122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 213/419] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Aug 28 00:58:12 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/ips => generated 145 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:58:12.937712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-35a7df76-025f-4955-9128-f84fe8b3e5e5 req-3231ae8f-be7d-46d3-b0d4-3b8483cc3536 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1cc5e59c-500c-421e-920c-c08b8ae3a206", "name": "network-vif-plugged", "status": "completed", "tag": "de0f5772-f2b1-42a3-84e4-e24d81b36597"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:58:12.949695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-35a7df76-025f-4955-9128-f84fe8b3e5e5 req-3231ae8f-be7d-46d3-b0d4-3b8483cc3536 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:12.950379 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-35a7df76-025f-4955-9128-f84fe8b3e5e5 req-3231ae8f-be7d-46d3-b0d4-3b8483cc3536 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:12.951208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-35a7df76-025f-4955-9128-f84fe8b3e5e5 req-3231ae8f-be7d-46d3-b0d4-3b8483cc3536 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:12.968887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-35a7df76-025f-4955-9128-f84fe8b3e5e5 req-3231ae8f-be7d-46d3-b0d4-3b8483cc3536 service nova] Creating event network-vif-plugged:de0f5772-f2b1-42a3-84e4-e24d81b36597 for instance 1cc5e59c-500c-421e-920c-c08b8ae3a206 on hv-cinder-maste Aug 28 00:58:12.974957 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-35a7df76-025f-4955-9128-f84fe8b3e5e5 req-3231ae8f-be7d-46d3-b0d4-3b8483cc3536 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041046 Aug 28 00:58:12.975579 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 214/420] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:58:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:58:13.131116 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-31c7102e-8919-43a3-8329-ca115e355cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:13.150162 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-31c7102e-8919-43a3-8329-ca115e355cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:13.152235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-31c7102e-8919-43a3-8329-ca115e355cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:13.154323 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-31c7102e-8919-43a3-8329-ca115e355cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1721 microversion: 2.20 time: 0.293074 Aug 28 00:58:13.162194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 207/421] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:12 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1721 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:13.229975 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bf70b107-4b43-4e0c-97ff-a4a1ee7a4fab tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:13.231689 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bf70b107-4b43-4e0c-97ff-a4a1ee7a4fab tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:13.255302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf70b107-4b43-4e0c-97ff-a4a1ee7a4fab tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:13.256137 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf70b107-4b43-4e0c-97ff-a4a1ee7a4fab tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:13.256892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf70b107-4b43-4e0c-97ff-a4a1ee7a4fab tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:13.258409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf70b107-4b43-4e0c-97ff-a4a1ee7a4fab tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:13.259261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf70b107-4b43-4e0c-97ff-a4a1ee7a4fab tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:13.261473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf70b107-4b43-4e0c-97ff-a4a1ee7a4fab tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:13.588804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bf70b107-4b43-4e0c-97ff-a4a1ee7a4fab tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:13.598752 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bf70b107-4b43-4e0c-97ff-a4a1ee7a4fab tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1266 microversion: 2.1 time: 0.377689 Aug 28 00:58:13.599866 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 215/422] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:13 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1266 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:14.190707 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-88fcc327-50a5-4d80-b167-f97e54b92657 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:14.191970 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-88fcc327-50a5-4d80-b167-f97e54b92657 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:14.199361 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=57,nova_api:UPDATE=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:58:14.201950 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88fcc327-50a5-4d80-b167-f97e54b92657 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:14.203044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88fcc327-50a5-4d80-b167-f97e54b92657 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:14.203705 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88fcc327-50a5-4d80-b167-f97e54b92657 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:14.204771 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88fcc327-50a5-4d80-b167-f97e54b92657 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:14.205938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88fcc327-50a5-4d80-b167-f97e54b92657 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:14.211197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88fcc327-50a5-4d80-b167-f97e54b92657 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:14.427445 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-88fcc327-50a5-4d80-b167-f97e54b92657 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:14.437298 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-88fcc327-50a5-4d80-b167-f97e54b92657 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:14.438870 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-88fcc327-50a5-4d80-b167-f97e54b92657 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:14.440311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-88fcc327-50a5-4d80-b167-f97e54b92657 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1721 microversion: 2.20 time: 0.256824 Aug 28 00:58:14.442223 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 208/423] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:14 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1721 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:14.634802 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4a57bd20-4af7-4bc3-8a9a-b059321dfc7c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:14.635861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4a57bd20-4af7-4bc3-8a9a-b059321dfc7c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:14.646243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a57bd20-4af7-4bc3-8a9a-b059321dfc7c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:14.646913 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a57bd20-4af7-4bc3-8a9a-b059321dfc7c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:14.647581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a57bd20-4af7-4bc3-8a9a-b059321dfc7c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:14.648516 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a57bd20-4af7-4bc3-8a9a-b059321dfc7c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:14.649177 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a57bd20-4af7-4bc3-8a9a-b059321dfc7c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:14.649795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a57bd20-4af7-4bc3-8a9a-b059321dfc7c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:14.961506 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-63137a84-48ae-400d-98a3-0ec351cc5b4c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "1dcdecb9-22d2-4941-9359-084d3178b65a"}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:58:14.965801 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-63137a84-48ae-400d-98a3-0ec351cc5b4c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:14.991736 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-63137a84-48ae-400d-98a3-0ec351cc5b4c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:14.992544 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-63137a84-48ae-400d-98a3-0ec351cc5b4c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:14.993277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-63137a84-48ae-400d-98a3-0ec351cc5b4c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:14.994478 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-63137a84-48ae-400d-98a3-0ec351cc5b4c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:15.001146 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-63137a84-48ae-400d-98a3-0ec351cc5b4c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:15.001146 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-63137a84-48ae-400d-98a3-0ec351cc5b4c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:15.033143 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4a57bd20-4af7-4bc3-8a9a-b059321dfc7c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:15.049808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4a57bd20-4af7-4bc3-8a9a-b059321dfc7c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1266 microversion: 2.1 time: 0.424070 Aug 28 00:58:15.051673 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 216/424] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:14 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1266 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:15.469259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-90709b76-9d93-442a-85fa-4d21c2737538 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:15.470392 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-90709b76-9d93-442a-85fa-4d21c2737538 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:15.480676 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90709b76-9d93-442a-85fa-4d21c2737538 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:15.481593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90709b76-9d93-442a-85fa-4d21c2737538 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:15.482594 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90709b76-9d93-442a-85fa-4d21c2737538 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:15.483770 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90709b76-9d93-442a-85fa-4d21c2737538 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:15.484740 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90709b76-9d93-442a-85fa-4d21c2737538 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:15.485476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90709b76-9d93-442a-85fa-4d21c2737538 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:15.508859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-63137a84-48ae-400d-98a3-0ec351cc5b4c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "POST /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.553577 Aug 28 00:58:15.511156 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 209/425] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Aug 28 00:58:14 2022] POST /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 194 bytes in 558 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:58:15.737359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-90709b76-9d93-442a-85fa-4d21c2737538 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:15.748051 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-90709b76-9d93-442a-85fa-4d21c2737538 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:15.749804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-90709b76-9d93-442a-85fa-4d21c2737538 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:15.751321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-90709b76-9d93-442a-85fa-4d21c2737538 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1721 microversion: 2.20 time: 0.289802 Aug 28 00:58:15.752256 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 217/426] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:15 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1721 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:58:16.092474 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-38ffd99b-1464-4956-9bf4-13aadf37815d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:16.099048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-38ffd99b-1464-4956-9bf4-13aadf37815d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:16.127814 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38ffd99b-1464-4956-9bf4-13aadf37815d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:16.129190 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38ffd99b-1464-4956-9bf4-13aadf37815d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:16.130938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38ffd99b-1464-4956-9bf4-13aadf37815d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:16.132929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38ffd99b-1464-4956-9bf4-13aadf37815d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:16.134096 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38ffd99b-1464-4956-9bf4-13aadf37815d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:16.135167 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38ffd99b-1464-4956-9bf4-13aadf37815d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:16.427201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-38ffd99b-1464-4956-9bf4-13aadf37815d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:16.457585 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-38ffd99b-1464-4956-9bf4-13aadf37815d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1451 microversion: 2.1 time: 0.373817 Aug 28 00:58:16.460249 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 210/427] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:16 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1451 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:16.483835 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-00087401-d0d3-4d00-a68a-1b09f3cd6e31 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:16.484939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-00087401-d0d3-4d00-a68a-1b09f3cd6e31 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:16.516086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-00087401-d0d3-4d00-a68a-1b09f3cd6e31 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:16.522423 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-00087401-d0d3-4d00-a68a-1b09f3cd6e31 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:16.523334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-00087401-d0d3-4d00-a68a-1b09f3cd6e31 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:16.525007 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-00087401-d0d3-4d00-a68a-1b09f3cd6e31 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:16.526142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-00087401-d0d3-4d00-a68a-1b09f3cd6e31 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:16.527226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-00087401-d0d3-4d00-a68a-1b09f3cd6e31 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:16.793407 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-929037dc-4763-41d2-a85c-e8096ea0b2dc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:16.793407 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-929037dc-4763-41d2-a85c-e8096ea0b2dc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:16.809345 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-929037dc-4763-41d2-a85c-e8096ea0b2dc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:16.809972 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-929037dc-4763-41d2-a85c-e8096ea0b2dc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:16.810678 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-929037dc-4763-41d2-a85c-e8096ea0b2dc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:16.811960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-929037dc-4763-41d2-a85c-e8096ea0b2dc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:16.812615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-929037dc-4763-41d2-a85c-e8096ea0b2dc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:16.813402 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-929037dc-4763-41d2-a85c-e8096ea0b2dc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:16.823480 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-00087401-d0d3-4d00-a68a-1b09f3cd6e31 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:16.838573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-00087401-d0d3-4d00-a68a-1b09f3cd6e31 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1451 microversion: 2.1 time: 0.361981 Aug 28 00:58:16.839512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 218/428] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:16 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1451 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:17.044369 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-929037dc-4763-41d2-a85c-e8096ea0b2dc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:17.065939 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-929037dc-4763-41d2-a85c-e8096ea0b2dc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:17.069766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-929037dc-4763-41d2-a85c-e8096ea0b2dc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:17.083348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-929037dc-4763-41d2-a85c-e8096ea0b2dc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1721 microversion: 2.20 time: 0.297410 Aug 28 00:58:17.087036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 211/429] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:16 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1721 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:58:17.267772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-da0d0eb2-66e4-44ad-9c0b-9647ebcadedf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2d7e8fa1-c167-4381-a38c-015d9ac4cf10"}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:58:17.277055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-da0d0eb2-66e4-44ad-9c0b-9647ebcadedf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:17.291725 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da0d0eb2-66e4-44ad-9c0b-9647ebcadedf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:17.292673 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da0d0eb2-66e4-44ad-9c0b-9647ebcadedf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:17.294124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da0d0eb2-66e4-44ad-9c0b-9647ebcadedf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:17.296304 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da0d0eb2-66e4-44ad-9c0b-9647ebcadedf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:17.297338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da0d0eb2-66e4-44ad-9c0b-9647ebcadedf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:17.298063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da0d0eb2-66e4-44ad-9c0b-9647ebcadedf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:18.010304 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-da0d0eb2-66e4-44ad-9c0b-9647ebcadedf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "POST /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.750437 Aug 28 00:58:18.011907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 219/430] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Aug 28 00:58:17 2022] POST /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 194 bytes in 754 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:58:18.116144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2fd30eb1-ed03-4076-89b0-72277932f565 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:18.125345 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2fd30eb1-ed03-4076-89b0-72277932f565 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:18.139682 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fd30eb1-ed03-4076-89b0-72277932f565 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:18.141796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fd30eb1-ed03-4076-89b0-72277932f565 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:18.143234 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fd30eb1-ed03-4076-89b0-72277932f565 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:18.144739 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fd30eb1-ed03-4076-89b0-72277932f565 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:18.146561 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fd30eb1-ed03-4076-89b0-72277932f565 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:18.147949 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fd30eb1-ed03-4076-89b0-72277932f565 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:18.422311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2fd30eb1-ed03-4076-89b0-72277932f565 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:18.435446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2fd30eb1-ed03-4076-89b0-72277932f565 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:18.437037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2fd30eb1-ed03-4076-89b0-72277932f565 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:18.438700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2fd30eb1-ed03-4076-89b0-72277932f565 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1721 microversion: 2.20 time: 0.331478 Aug 28 00:58:18.439759 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 212/431] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:18 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1721 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:19.488921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ec6affb5-0117-4b42-a3ba-7d0f6daa2c99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:19.491238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ec6affb5-0117-4b42-a3ba-7d0f6daa2c99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:19.506568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec6affb5-0117-4b42-a3ba-7d0f6daa2c99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:19.508420 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec6affb5-0117-4b42-a3ba-7d0f6daa2c99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:19.509273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec6affb5-0117-4b42-a3ba-7d0f6daa2c99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:19.515210 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec6affb5-0117-4b42-a3ba-7d0f6daa2c99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:19.517740 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec6affb5-0117-4b42-a3ba-7d0f6daa2c99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:19.519866 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec6affb5-0117-4b42-a3ba-7d0f6daa2c99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:19.927527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ec6affb5-0117-4b42-a3ba-7d0f6daa2c99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:19.942665 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ec6affb5-0117-4b42-a3ba-7d0f6daa2c99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:19.945962 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ec6affb5-0117-4b42-a3ba-7d0f6daa2c99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:19.948859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ec6affb5-0117-4b42-a3ba-7d0f6daa2c99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1721 microversion: 2.20 time: 0.487802 Aug 28 00:58:19.950469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 220/432] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:19 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1721 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:58:20.456430 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-c7334687-20c6-453f-86e5-92d14325c333 req-83509696-7d82-456e-9f42-47f1cd2d07cc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1cc5e59c-500c-421e-920c-c08b8ae3a206", "tag": "de0f5772-f2b1-42a3-84e4-e24d81b36597"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:58:20.476333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-c7334687-20c6-453f-86e5-92d14325c333 req-83509696-7d82-456e-9f42-47f1cd2d07cc service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:20.476733 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-c7334687-20c6-453f-86e5-92d14325c333 req-83509696-7d82-456e-9f42-47f1cd2d07cc service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:20.477900 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-c7334687-20c6-453f-86e5-92d14325c333 req-83509696-7d82-456e-9f42-47f1cd2d07cc service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:20.507334 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-c7334687-20c6-453f-86e5-92d14325c333 req-83509696-7d82-456e-9f42-47f1cd2d07cc service nova] Creating event network-changed:de0f5772-f2b1-42a3-84e4-e24d81b36597 for instance 1cc5e59c-500c-421e-920c-c08b8ae3a206 on hv-cinder-maste Aug 28 00:58:20.518849 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-c7334687-20c6-453f-86e5-92d14325c333 req-83509696-7d82-456e-9f42-47f1cd2d07cc service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.069323 Aug 28 00:58:20.520442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 213/433] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:58:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:58:20.977874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8248f91b-f94d-4ab3-99f2-2f719f9e9e39 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:20.979027 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8248f91b-f94d-4ab3-99f2-2f719f9e9e39 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:20.993027 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8248f91b-f94d-4ab3-99f2-2f719f9e9e39 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:20.993874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8248f91b-f94d-4ab3-99f2-2f719f9e9e39 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:20.994632 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8248f91b-f94d-4ab3-99f2-2f719f9e9e39 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:20.996323 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8248f91b-f94d-4ab3-99f2-2f719f9e9e39 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:20.997754 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8248f91b-f94d-4ab3-99f2-2f719f9e9e39 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:20.998817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8248f91b-f94d-4ab3-99f2-2f719f9e9e39 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:21.337770 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8248f91b-f94d-4ab3-99f2-2f719f9e9e39 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:21.355771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8248f91b-f94d-4ab3-99f2-2f719f9e9e39 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:21.357443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8248f91b-f94d-4ab3-99f2-2f719f9e9e39 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:21.358822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8248f91b-f94d-4ab3-99f2-2f719f9e9e39 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1721 microversion: 2.20 time: 0.389527 Aug 28 00:58:21.359496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 221/434] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:20 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1721 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:22.384210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a8cdc23a-894b-4cbc-875f-0d7fa9d9d4a2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:22.385057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a8cdc23a-894b-4cbc-875f-0d7fa9d9d4a2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:22.392145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a8cdc23a-894b-4cbc-875f-0d7fa9d9d4a2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:22.392681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a8cdc23a-894b-4cbc-875f-0d7fa9d9d4a2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:22.393154 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a8cdc23a-894b-4cbc-875f-0d7fa9d9d4a2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:22.393925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a8cdc23a-894b-4cbc-875f-0d7fa9d9d4a2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:22.394371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a8cdc23a-894b-4cbc-875f-0d7fa9d9d4a2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:22.394788 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a8cdc23a-894b-4cbc-875f-0d7fa9d9d4a2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:22.574335 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a8cdc23a-894b-4cbc-875f-0d7fa9d9d4a2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:22.584875 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a8cdc23a-894b-4cbc-875f-0d7fa9d9d4a2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:22.586770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a8cdc23a-894b-4cbc-875f-0d7fa9d9d4a2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:22.588075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a8cdc23a-894b-4cbc-875f-0d7fa9d9d4a2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1721 microversion: 2.20 time: 0.208918 Aug 28 00:58:22.588721 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 214/435] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:22 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1721 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:58:23.621905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f0b44356-e09d-4671-a823-bc069c79e690 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:23.623604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f0b44356-e09d-4671-a823-bc069c79e690 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:23.635899 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0b44356-e09d-4671-a823-bc069c79e690 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:23.636760 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0b44356-e09d-4671-a823-bc069c79e690 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:23.637515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0b44356-e09d-4671-a823-bc069c79e690 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:23.649010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0b44356-e09d-4671-a823-bc069c79e690 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:23.649010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0b44356-e09d-4671-a823-bc069c79e690 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:23.649010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0b44356-e09d-4671-a823-bc069c79e690 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:23.902254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f0b44356-e09d-4671-a823-bc069c79e690 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:23.911800 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f0b44356-e09d-4671-a823-bc069c79e690 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:23.913406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f0b44356-e09d-4671-a823-bc069c79e690 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:23.914918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f0b44356-e09d-4671-a823-bc069c79e690 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1721 microversion: 2.20 time: 0.304091 Aug 28 00:58:23.915581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 222/436] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:23 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1721 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:24.692468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-24c404f3-7e0c-47ce-bd31-17d6719dc307 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Action: 'create', calling method: .version_select at 0x7f0e6e9bb820>, body: {"keypair": {"name": "tempest-keypair-669625462", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDD7/NROcHRNLZ0l7cNXQjRQTc1O4VXGijEMMEl1PAkiPMMVj0V8BR4NAQGHAvF8s/hm6PEwD4jy7i4tLpUbpIKUx6Hiifp47FAluzefqXMVx5K/GZrgrW5zFUAH83JRxg=="}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:58:24.722094 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-24c404f3-7e0c-47ce-bd31-17d6719dc307 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Getting quotas for project 95df81e14a8549fca0851234f4a00b89. Resources: {'key_pairs'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 00:58:24.731296 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-24c404f3-7e0c-47ce-bd31-17d6719dc307 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Getting quotas for user d209174d0db04b3584556824c6d0a3d4 and project 95df81e14a8549fca0851234f4a00b89. Resources: {'key_pairs'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 00:58:24.787867 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-24c404f3-7e0c-47ce-bd31-17d6719dc307 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.098891 Aug 28 00:58:24.789858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 215/437] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Aug 28 00:58:24 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:58:24.941927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-66b1c4fe-e97a-4578-aa62-6a685b556ca4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:24.943025 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-66b1c4fe-e97a-4578-aa62-6a685b556ca4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:24.953321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66b1c4fe-e97a-4578-aa62-6a685b556ca4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:24.954139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66b1c4fe-e97a-4578-aa62-6a685b556ca4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:24.954648 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66b1c4fe-e97a-4578-aa62-6a685b556ca4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:24.955492 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66b1c4fe-e97a-4578-aa62-6a685b556ca4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:24.956163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66b1c4fe-e97a-4578-aa62-6a685b556ca4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:24.956600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66b1c4fe-e97a-4578-aa62-6a685b556ca4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:25.284489 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-66b1c4fe-e97a-4578-aa62-6a685b556ca4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:25.296143 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-66b1c4fe-e97a-4578-aa62-6a685b556ca4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:25.297948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-66b1c4fe-e97a-4578-aa62-6a685b556ca4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:25.299586 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-66b1c4fe-e97a-4578-aa62-6a685b556ca4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1721 microversion: 2.20 time: 0.364298 Aug 28 00:58:25.300626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 223/438] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:24 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1721 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:58:26.325238 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7dcca220-37e4-4d5b-b629-c449d53e12d5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:26.325923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7dcca220-37e4-4d5b-b629-c449d53e12d5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:26.335379 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7dcca220-37e4-4d5b-b629-c449d53e12d5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:26.336243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7dcca220-37e4-4d5b-b629-c449d53e12d5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:26.336691 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7dcca220-37e4-4d5b-b629-c449d53e12d5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:26.340045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7dcca220-37e4-4d5b-b629-c449d53e12d5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:26.340705 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7dcca220-37e4-4d5b-b629-c449d53e12d5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:26.341779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7dcca220-37e4-4d5b-b629-c449d53e12d5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:26.619610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7dcca220-37e4-4d5b-b629-c449d53e12d5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:26.629306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7dcca220-37e4-4d5b-b629-c449d53e12d5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:26.631660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7dcca220-37e4-4d5b-b629-c449d53e12d5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:26.633038 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7dcca220-37e4-4d5b-b629-c449d53e12d5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1742 microversion: 2.20 time: 0.313582 Aug 28 00:58:26.633613 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 216/439] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:26 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1742 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:27.071323 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-096b51a9-6f7d-4723-820d-4f4be14d2852 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1608317588", "imageRef": "2c2585b2-bde5-4652-aed7-9f53b1bd5d94", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "e3dfdb02-2ba5-4b22-9a96-a212c53e9d60"}], "security_groups": [{"name": "tempest-securitygroup--120082760"}], "key_name": "tempest-keypair-669625462", "user_data": "}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:58:27.189838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.network.neutron [None req-096b51a9-6f7d-4723-820d-4f4be14d2852 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] validate_networks() for [('e3dfdb02-2ba5-4b22-9a96-a212c53e9d60', None, None, None, None, None)] {{(pid=120525) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 28 00:58:27.676371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f4d94a9f-0c50-429d-834f-571aa8882d96 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:27.677715 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f4d94a9f-0c50-429d-834f-571aa8882d96 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:27.691280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4d94a9f-0c50-429d-834f-571aa8882d96 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:27.693103 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4d94a9f-0c50-429d-834f-571aa8882d96 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:27.693886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4d94a9f-0c50-429d-834f-571aa8882d96 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:27.695170 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4d94a9f-0c50-429d-834f-571aa8882d96 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:27.696098 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4d94a9f-0c50-429d-834f-571aa8882d96 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:27.697694 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4d94a9f-0c50-429d-834f-571aa8882d96 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:27.707372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-096b51a9-6f7d-4723-820d-4f4be14d2852 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Flavor limits 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 28 00:58:27.707779 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-096b51a9-6f7d-4723-820d-4f4be14d2852 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Image limits 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 28 00:58:27.708600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-096b51a9-6f7d-4723-820d-4f4be14d2852 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Flavor pref 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 28 00:58:27.709327 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-096b51a9-6f7d-4723-820d-4f4be14d2852 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Image pref 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 28 00:58:27.710294 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-096b51a9-6f7d-4723-820d-4f4be14d2852 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 28 00:58:27.725905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-096b51a9-6f7d-4723-820d-4f4be14d2852 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:27.726841 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-096b51a9-6f7d-4723-820d-4f4be14d2852 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:27.728270 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-096b51a9-6f7d-4723-820d-4f4be14d2852 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:27.728663 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-096b51a9-6f7d-4723-820d-4f4be14d2852 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:27.729782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-096b51a9-6f7d-4723-820d-4f4be14d2852 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:27.730913 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-096b51a9-6f7d-4723-820d-4f4be14d2852 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:27.757901 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-096b51a9-6f7d-4723-820d-4f4be14d2852 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Getting quotas for project 95df81e14a8549fca0851234f4a00b89. Resources: {'ram', 'instances', 'cores'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 00:58:27.763718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-096b51a9-6f7d-4723-820d-4f4be14d2852 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Getting quotas for user d209174d0db04b3584556824c6d0a3d4 and project 95df81e14a8549fca0851234f4a00b89. Resources: {'ram', 'instances', 'cores'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 00:58:27.780291 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-096b51a9-6f7d-4723-820d-4f4be14d2852 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Going to run 1 instances... {{(pid=120525) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 28 00:58:27.783214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-096b51a9-6f7d-4723-820d-4f4be14d2852 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2c2585b2-bde5-4652-aed7-9f53b1bd5d94',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120525) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 28 00:58:27.850625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-096b51a9-6f7d-4723-820d-4f4be14d2852 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.783617 Aug 28 00:58:27.861619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 224/440] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Aug 28 00:58:27 2022] POST /compute/v2.1/servers => generated 401 bytes in 785 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 28 00:58:27.872920 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-597ec4c8-8e18-40de-be87-2be67ac2f8cb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:27.874194 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-597ec4c8-8e18-40de-be87-2be67ac2f8cb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:27.920148 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f4d94a9f-0c50-429d-834f-571aa8882d96 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:27.937463 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f4d94a9f-0c50-429d-834f-571aa8882d96 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:27.940415 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f4d94a9f-0c50-429d-834f-571aa8882d96 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:27.942859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f4d94a9f-0c50-429d-834f-571aa8882d96 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1742 microversion: 2.20 time: 0.273339 Aug 28 00:58:27.944390 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 217/441] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:27 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1742 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:58:27.967734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-597ec4c8-8e18-40de-be87-2be67ac2f8cb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:27.978350 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-597ec4c8-8e18-40de-be87-2be67ac2f8cb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1241 microversion: 2.1 time: 0.113608 Aug 28 00:58:27.979533 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 225/442] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:27 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1241 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:28.882168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-ba1fdcf3-dead-4545-876f-240c184a7c8d req-dfd73a2d-4ac8-4752-8e19-7e7411efefee service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dd61123b-7a96-4c9e-936c-5794a4fdd7df", "name": "network-vif-plugged", "status": "completed", "tag": "522d4a1b-32bc-4106-afb0-5995aecc53c0"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:58:28.895828 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-ba1fdcf3-dead-4545-876f-240c184a7c8d req-dfd73a2d-4ac8-4752-8e19-7e7411efefee service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:28.896763 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-ba1fdcf3-dead-4545-876f-240c184a7c8d req-dfd73a2d-4ac8-4752-8e19-7e7411efefee service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:28.897550 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-ba1fdcf3-dead-4545-876f-240c184a7c8d req-dfd73a2d-4ac8-4752-8e19-7e7411efefee service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:28.911439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-ba1fdcf3-dead-4545-876f-240c184a7c8d req-dfd73a2d-4ac8-4752-8e19-7e7411efefee service nova] Creating event network-vif-plugged:522d4a1b-32bc-4106-afb0-5995aecc53c0 for instance dd61123b-7a96-4c9e-936c-5794a4fdd7df on hv-cinder-maste Aug 28 00:58:28.917654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-ba1fdcf3-dead-4545-876f-240c184a7c8d req-dfd73a2d-4ac8-4752-8e19-7e7411efefee service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039137 Aug 28 00:58:28.919090 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 218/443] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:58:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:58:28.975569 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b7bff767-1b95-4492-bf8c-aa18a7780dfb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:28.976577 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b7bff767-1b95-4492-bf8c-aa18a7780dfb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:28.989966 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7bff767-1b95-4492-bf8c-aa18a7780dfb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:28.990850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7bff767-1b95-4492-bf8c-aa18a7780dfb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:28.991641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7bff767-1b95-4492-bf8c-aa18a7780dfb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:28.992927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7bff767-1b95-4492-bf8c-aa18a7780dfb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:28.993802 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7bff767-1b95-4492-bf8c-aa18a7780dfb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:28.994564 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7bff767-1b95-4492-bf8c-aa18a7780dfb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:29.019341 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-229090f3-1394-4f6d-b9ff-b1ee61c38696 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:29.021588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-229090f3-1394-4f6d-b9ff-b1ee61c38696 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:29.035962 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-229090f3-1394-4f6d-b9ff-b1ee61c38696 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:29.036919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-229090f3-1394-4f6d-b9ff-b1ee61c38696 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:29.037760 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-229090f3-1394-4f6d-b9ff-b1ee61c38696 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:29.038814 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-229090f3-1394-4f6d-b9ff-b1ee61c38696 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:29.039477 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-229090f3-1394-4f6d-b9ff-b1ee61c38696 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:29.040203 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-229090f3-1394-4f6d-b9ff-b1ee61c38696 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:29.180757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-229090f3-1394-4f6d-b9ff-b1ee61c38696 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:29.193850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-229090f3-1394-4f6d-b9ff-b1ee61c38696 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1293 microversion: 2.1 time: 0.184628 Aug 28 00:58:29.195506 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 219/444] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:29 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1293 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:29.273948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b7bff767-1b95-4492-bf8c-aa18a7780dfb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:29.291794 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b7bff767-1b95-4492-bf8c-aa18a7780dfb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:29.293509 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b7bff767-1b95-4492-bf8c-aa18a7780dfb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:29.294827 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b7bff767-1b95-4492-bf8c-aa18a7780dfb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1742 microversion: 2.20 time: 0.326828 Aug 28 00:58:29.295429 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 226/445] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:28 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1742 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:30.222473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-547ffbca-7cf0-4f11-8bfe-001cf44e92dc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:30.223643 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-547ffbca-7cf0-4f11-8bfe-001cf44e92dc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:30.239158 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-547ffbca-7cf0-4f11-8bfe-001cf44e92dc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:30.240070 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-547ffbca-7cf0-4f11-8bfe-001cf44e92dc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:30.241252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-547ffbca-7cf0-4f11-8bfe-001cf44e92dc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:30.242582 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-547ffbca-7cf0-4f11-8bfe-001cf44e92dc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:30.243774 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-547ffbca-7cf0-4f11-8bfe-001cf44e92dc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:30.244537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-547ffbca-7cf0-4f11-8bfe-001cf44e92dc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:30.334050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a90e64a2-09fe-4b95-9db8-a70d8b4ddb2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:30.335545 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a90e64a2-09fe-4b95-9db8-a70d8b4ddb2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:30.341700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-547ffbca-7cf0-4f11-8bfe-001cf44e92dc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:30.348359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a90e64a2-09fe-4b95-9db8-a70d8b4ddb2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:30.349511 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a90e64a2-09fe-4b95-9db8-a70d8b4ddb2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:30.350402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a90e64a2-09fe-4b95-9db8-a70d8b4ddb2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:30.351567 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a90e64a2-09fe-4b95-9db8-a70d8b4ddb2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:30.351833 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-547ffbca-7cf0-4f11-8bfe-001cf44e92dc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1293 microversion: 2.1 time: 0.137263 Aug 28 00:58:30.352315 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a90e64a2-09fe-4b95-9db8-a70d8b4ddb2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:30.353145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 220/446] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:30 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1293 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:30.360748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a90e64a2-09fe-4b95-9db8-a70d8b4ddb2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:30.558456 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a90e64a2-09fe-4b95-9db8-a70d8b4ddb2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:30.568617 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a90e64a2-09fe-4b95-9db8-a70d8b4ddb2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:30.570479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a90e64a2-09fe-4b95-9db8-a70d8b4ddb2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:30.572524 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a90e64a2-09fe-4b95-9db8-a70d8b4ddb2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1742 microversion: 2.20 time: 0.245423 Aug 28 00:58:30.573372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 227/447] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:30 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1742 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:58:31.388811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c8c8f0b1-eb3b-4cca-93fb-178ed55f44e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:31.390105 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c8c8f0b1-eb3b-4cca-93fb-178ed55f44e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:31.399081 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8c8f0b1-eb3b-4cca-93fb-178ed55f44e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:31.399476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8c8f0b1-eb3b-4cca-93fb-178ed55f44e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:31.400045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8c8f0b1-eb3b-4cca-93fb-178ed55f44e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:31.401040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8c8f0b1-eb3b-4cca-93fb-178ed55f44e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:31.401554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8c8f0b1-eb3b-4cca-93fb-178ed55f44e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:31.401984 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8c8f0b1-eb3b-4cca-93fb-178ed55f44e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:31.480173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c8c8f0b1-eb3b-4cca-93fb-178ed55f44e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:31.495621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c8c8f0b1-eb3b-4cca-93fb-178ed55f44e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1293 microversion: 2.1 time: 0.114322 Aug 28 00:58:31.496738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 221/448] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:31 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1293 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:31.606075 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-affcd652-ad96-4852-855a-127591770296 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:31.618806 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-affcd652-ad96-4852-855a-127591770296 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:31.634829 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-affcd652-ad96-4852-855a-127591770296 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:31.635843 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-affcd652-ad96-4852-855a-127591770296 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:31.637026 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-affcd652-ad96-4852-855a-127591770296 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:31.638483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-affcd652-ad96-4852-855a-127591770296 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:31.639454 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-affcd652-ad96-4852-855a-127591770296 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:31.640162 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-affcd652-ad96-4852-855a-127591770296 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:32.023460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-affcd652-ad96-4852-855a-127591770296 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:32.045708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-affcd652-ad96-4852-855a-127591770296 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:32.053387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-affcd652-ad96-4852-855a-127591770296 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:32.060361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-affcd652-ad96-4852-855a-127591770296 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1742 microversion: 2.20 time: 0.462309 Aug 28 00:58:32.063506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 228/449] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:31 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1742 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:32.521875 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-20257481-3f29-4914-8d5f-f7c6baa345ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:32.524350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-20257481-3f29-4914-8d5f-f7c6baa345ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:32.540086 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20257481-3f29-4914-8d5f-f7c6baa345ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:32.540952 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20257481-3f29-4914-8d5f-f7c6baa345ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:32.541771 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20257481-3f29-4914-8d5f-f7c6baa345ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:32.543104 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20257481-3f29-4914-8d5f-f7c6baa345ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:32.544326 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20257481-3f29-4914-8d5f-f7c6baa345ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:32.545632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20257481-3f29-4914-8d5f-f7c6baa345ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:32.557008 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1a8f7612-b3d8-468a-bac3-ea0e6c2f4d78 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:58:32.565988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a8f7612-b3d8-468a-bac3-ea0e6c2f4d78 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:32.566896 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a8f7612-b3d8-468a-bac3-ea0e6c2f4d78 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:32.567500 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a8f7612-b3d8-468a-bac3-ea0e6c2f4d78 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:32.655949 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1a8f7612-b3d8-468a-bac3-ea0e6c2f4d78 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Going to try to stop instance {{(pid=120525) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Aug 28 00:58:32.668522 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-20257481-3f29-4914-8d5f-f7c6baa345ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:32.691913 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=3,nova_cell1:INSERT=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:58:32.707132 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-20257481-3f29-4914-8d5f-f7c6baa345ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1299 microversion: 2.1 time: 0.192277 Aug 28 00:58:32.713019 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 222/450] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:32 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1299 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:32.747588 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1a8f7612-b3d8-468a-bac3-ea0e6c2f4d78 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "POST /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/action" status: 202 len: 0 microversion: 2.1 time: 0.193752 Aug 28 00:58:32.748853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 229/451] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Aug 28 00:58:32 2022] POST /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/action => generated 0 bytes in 197 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 28 00:58:32.762629 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4a7ac29f-25e3-4d85-b74b-9db1e9cd8ecf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:32.764266 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4a7ac29f-25e3-4d85-b74b-9db1e9cd8ecf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:32.778948 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a7ac29f-25e3-4d85-b74b-9db1e9cd8ecf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:32.780871 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a7ac29f-25e3-4d85-b74b-9db1e9cd8ecf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:32.782414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a7ac29f-25e3-4d85-b74b-9db1e9cd8ecf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:32.784605 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a7ac29f-25e3-4d85-b74b-9db1e9cd8ecf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:32.785895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a7ac29f-25e3-4d85-b74b-9db1e9cd8ecf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:32.787171 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a7ac29f-25e3-4d85-b74b-9db1e9cd8ecf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:33.072037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4a7ac29f-25e3-4d85-b74b-9db1e9cd8ecf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:33.083994 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4a7ac29f-25e3-4d85-b74b-9db1e9cd8ecf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1724 microversion: 2.1 time: 0.329067 Aug 28 00:58:33.085638 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 223/452] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:32 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1724 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:33.104286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-926a3327-5ff7-41ec-8a75-0b11b5048be9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:33.105889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-926a3327-5ff7-41ec-8a75-0b11b5048be9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:33.123111 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-926a3327-5ff7-41ec-8a75-0b11b5048be9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:33.123962 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-926a3327-5ff7-41ec-8a75-0b11b5048be9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:33.124757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-926a3327-5ff7-41ec-8a75-0b11b5048be9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:33.126284 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-926a3327-5ff7-41ec-8a75-0b11b5048be9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:33.127094 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-926a3327-5ff7-41ec-8a75-0b11b5048be9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:33.127879 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-926a3327-5ff7-41ec-8a75-0b11b5048be9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:33.406034 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-926a3327-5ff7-41ec-8a75-0b11b5048be9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:33.416045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-926a3327-5ff7-41ec-8a75-0b11b5048be9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:33.417763 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-926a3327-5ff7-41ec-8a75-0b11b5048be9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:33.419109 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-926a3327-5ff7-41ec-8a75-0b11b5048be9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1737 microversion: 2.20 time: 0.335058 Aug 28 00:58:33.419692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 230/453] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:33 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1737 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:58:33.742155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-63e0d7b1-0f7b-44f7-bd60-8e2c9190e380 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:33.743916 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-63e0d7b1-0f7b-44f7-bd60-8e2c9190e380 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:33.756714 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-63e0d7b1-0f7b-44f7-bd60-8e2c9190e380 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:33.757986 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-63e0d7b1-0f7b-44f7-bd60-8e2c9190e380 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:33.759262 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-63e0d7b1-0f7b-44f7-bd60-8e2c9190e380 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:33.761187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-63e0d7b1-0f7b-44f7-bd60-8e2c9190e380 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:33.762410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-63e0d7b1-0f7b-44f7-bd60-8e2c9190e380 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:33.763371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-63e0d7b1-0f7b-44f7-bd60-8e2c9190e380 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:33.773158 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:58:34.126078 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9efe861b-b37f-468d-85f8-0107e2acd036 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:34.126910 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9efe861b-b37f-468d-85f8-0107e2acd036 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:34.143601 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9efe861b-b37f-468d-85f8-0107e2acd036 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:34.144365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9efe861b-b37f-468d-85f8-0107e2acd036 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:34.144815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9efe861b-b37f-468d-85f8-0107e2acd036 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:34.147056 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9efe861b-b37f-468d-85f8-0107e2acd036 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:34.148223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9efe861b-b37f-468d-85f8-0107e2acd036 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:34.153466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9efe861b-b37f-468d-85f8-0107e2acd036 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:34.322977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-63e0d7b1-0f7b-44f7-bd60-8e2c9190e380 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:34.334861 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-63e0d7b1-0f7b-44f7-bd60-8e2c9190e380 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1366 microversion: 2.1 time: 0.601249 Aug 28 00:58:34.336110 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 224/454] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:33 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1366 bytes in 604 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:58:34.477936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-335f8047-ec70-42c8-bd99-fc63d130c171 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:34.479189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-335f8047-ec70-42c8-bd99-fc63d130c171 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:34.493332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-335f8047-ec70-42c8-bd99-fc63d130c171 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:34.494639 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-335f8047-ec70-42c8-bd99-fc63d130c171 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:34.495731 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-335f8047-ec70-42c8-bd99-fc63d130c171 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:34.497179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-335f8047-ec70-42c8-bd99-fc63d130c171 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:34.498264 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-335f8047-ec70-42c8-bd99-fc63d130c171 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:34.499418 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-335f8047-ec70-42c8-bd99-fc63d130c171 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:34.536701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9efe861b-b37f-468d-85f8-0107e2acd036 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:34.564274 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9efe861b-b37f-468d-85f8-0107e2acd036 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1724 microversion: 2.1 time: 0.455842 Aug 28 00:58:34.565798 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 231/455] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:34 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1724 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:58:34.754649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=4 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:58:34.824408 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-3d3fcd6a-a887-4e0f-a327-efa5e21a98ad req-0c343c6e-66a2-42c8-8e7e-90c2662c8b95 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "27028458-f872-49aa-8d37-1a886d566c00", "tag": "59c8cffa-e3e1-4a46-979a-94e0428e33d6"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:58:34.841592 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=67,nova_api:INSERT=3 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:58:34.869063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-3d3fcd6a-a887-4e0f-a327-efa5e21a98ad req-0c343c6e-66a2-42c8-8e7e-90c2662c8b95 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:34.873815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-3d3fcd6a-a887-4e0f-a327-efa5e21a98ad req-0c343c6e-66a2-42c8-8e7e-90c2662c8b95 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:34.875275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-3d3fcd6a-a887-4e0f-a327-efa5e21a98ad req-0c343c6e-66a2-42c8-8e7e-90c2662c8b95 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:34.914333 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-3d3fcd6a-a887-4e0f-a327-efa5e21a98ad req-0c343c6e-66a2-42c8-8e7e-90c2662c8b95 service nova] Creating event network-changed:59c8cffa-e3e1-4a46-979a-94e0428e33d6 for instance 27028458-f872-49aa-8d37-1a886d566c00 on hv-cinder-maste Aug 28 00:58:34.925163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-3d3fcd6a-a887-4e0f-a327-efa5e21a98ad req-0c343c6e-66a2-42c8-8e7e-90c2662c8b95 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.108733 Aug 28 00:58:34.927582 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 232/456] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:58:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:58:35.005494 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-335f8047-ec70-42c8-bd99-fc63d130c171 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:35.021902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-335f8047-ec70-42c8-bd99-fc63d130c171 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:35.025119 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-335f8047-ec70-42c8-bd99-fc63d130c171 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:35.027503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-335f8047-ec70-42c8-bd99-fc63d130c171 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1737 microversion: 2.20 time: 0.565047 Aug 28 00:58:35.033693 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 225/457] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:34 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1737 bytes in 573 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:35.090985 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e3506e09-c711-4d28-a03e-79fdfb5c9843 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-197298675", "imageRef": "2c2585b2-bde5-4652-aed7-9f53b1bd5d94", "flavorRef": "452", "networks": [{"uuid": "aca2a4f4-367a-4fc9-8160-ba764301efb4"}]}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:58:35.221117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.network.neutron [None req-e3506e09-c711-4d28-a03e-79fdfb5c9843 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] validate_networks() for [('aca2a4f4-367a-4fc9-8160-ba764301efb4', None, None, None, None, None)] {{(pid=120525) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 28 00:58:35.362139 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9a0879ba-e9a7-4ae4-9baf-652e56a111e8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:35.363768 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9a0879ba-e9a7-4ae4-9baf-652e56a111e8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:35.374743 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9a0879ba-e9a7-4ae4-9baf-652e56a111e8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:35.375865 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9a0879ba-e9a7-4ae4-9baf-652e56a111e8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:35.377115 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9a0879ba-e9a7-4ae4-9baf-652e56a111e8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:35.378613 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9a0879ba-e9a7-4ae4-9baf-652e56a111e8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:35.379791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9a0879ba-e9a7-4ae4-9baf-652e56a111e8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:35.380760 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9a0879ba-e9a7-4ae4-9baf-652e56a111e8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:36.141856 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9a0879ba-e9a7-4ae4-9baf-652e56a111e8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:36.156104 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9a0879ba-e9a7-4ae4-9baf-652e56a111e8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1366 microversion: 2.1 time: 0.802592 Aug 28 00:58:36.169914 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 226/458] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:35 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1366 bytes in 817 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:36.175399 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-43caaf6a-9ee1-40fc-aa1f-1604f665a19d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:36.176027 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-43caaf6a-9ee1-40fc-aa1f-1604f665a19d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:36.182886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43caaf6a-9ee1-40fc-aa1f-1604f665a19d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:36.183694 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43caaf6a-9ee1-40fc-aa1f-1604f665a19d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:36.184361 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43caaf6a-9ee1-40fc-aa1f-1604f665a19d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:36.185433 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43caaf6a-9ee1-40fc-aa1f-1604f665a19d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:36.186065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43caaf6a-9ee1-40fc-aa1f-1604f665a19d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:36.186653 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43caaf6a-9ee1-40fc-aa1f-1604f665a19d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:36.546943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-43caaf6a-9ee1-40fc-aa1f-1604f665a19d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:36.580745 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-43caaf6a-9ee1-40fc-aa1f-1604f665a19d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1724 microversion: 2.1 time: 0.409323 Aug 28 00:58:36.584657 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 227/459] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:36 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1724 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:36.600890 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-76ce451a-95ab-4c0b-a8e2-f3638c3cbd7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:36.603910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-76ce451a-95ab-4c0b-a8e2-f3638c3cbd7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:36.613134 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-76ce451a-95ab-4c0b-a8e2-f3638c3cbd7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:36.613675 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-76ce451a-95ab-4c0b-a8e2-f3638c3cbd7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:36.614146 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-76ce451a-95ab-4c0b-a8e2-f3638c3cbd7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:36.615270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-76ce451a-95ab-4c0b-a8e2-f3638c3cbd7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:36.616019 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-76ce451a-95ab-4c0b-a8e2-f3638c3cbd7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:36.616457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-76ce451a-95ab-4c0b-a8e2-f3638c3cbd7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:36.727591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-e3506e09-c711-4d28-a03e-79fdfb5c9843 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Flavor limits 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 28 00:58:36.728403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-e3506e09-c711-4d28-a03e-79fdfb5c9843 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Image limits 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 28 00:58:36.729587 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-e3506e09-c711-4d28-a03e-79fdfb5c9843 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Flavor pref 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 28 00:58:36.731390 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-e3506e09-c711-4d28-a03e-79fdfb5c9843 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Image pref 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 28 00:58:36.732273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-e3506e09-c711-4d28-a03e-79fdfb5c9843 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 28 00:58:36.751033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e3506e09-c711-4d28-a03e-79fdfb5c9843 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:36.752745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e3506e09-c711-4d28-a03e-79fdfb5c9843 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:36.754257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e3506e09-c711-4d28-a03e-79fdfb5c9843 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:36.756251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e3506e09-c711-4d28-a03e-79fdfb5c9843 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:36.757140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e3506e09-c711-4d28-a03e-79fdfb5c9843 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:36.758666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e3506e09-c711-4d28-a03e-79fdfb5c9843 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:36.795269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-e3506e09-c711-4d28-a03e-79fdfb5c9843 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Getting quotas for project 89dfee143ed147abb6f2339958412e7e. Resources: {'ram', 'instances', 'cores'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 00:58:36.803699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-e3506e09-c711-4d28-a03e-79fdfb5c9843 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Getting quotas for user 93ee4e8157364093bc9cdbe980bde4bb and project 89dfee143ed147abb6f2339958412e7e. Resources: {'ram', 'instances', 'cores'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 00:58:36.824502 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e3506e09-c711-4d28-a03e-79fdfb5c9843 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Going to run 1 instances... {{(pid=120525) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 28 00:58:36.829337 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e3506e09-c711-4d28-a03e-79fdfb5c9843 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2c2585b2-bde5-4652-aed7-9f53b1bd5d94',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120525) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 28 00:58:36.846628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-76ce451a-95ab-4c0b-a8e2-f3638c3cbd7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:36.867440 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-76ce451a-95ab-4c0b-a8e2-f3638c3cbd7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:36.870686 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-76ce451a-95ab-4c0b-a8e2-f3638c3cbd7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:36.873688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-76ce451a-95ab-4c0b-a8e2-f3638c3cbd7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1737 microversion: 2.20 time: 0.286654 Aug 28 00:58:36.875162 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 228/460] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:36 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1737 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:36.925702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e3506e09-c711-4d28-a03e-79fdfb5c9843 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.842749 Aug 28 00:58:36.937114 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 233/461] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Aug 28 00:58:35 2022] POST /compute/v2.1/servers => generated 380 bytes in 1856 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 28 00:58:36.951615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d663f8b6-f49f-429a-a7ae-f0862f3683ed tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:36.953621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d663f8b6-f49f-429a-a7ae-f0862f3683ed tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:37.099338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d663f8b6-f49f-429a-a7ae-f0862f3683ed tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:37.135936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d663f8b6-f49f-429a-a7ae-f0862f3683ed tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1218 microversion: 2.1 time: 0.195561 Aug 28 00:58:37.159029 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 229/462] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:36 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1218 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:37.185985 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-44e9930b-d80c-4227-b6c5-d87cd3597ef7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:37.187064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-44e9930b-d80c-4227-b6c5-d87cd3597ef7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:37.201024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-44e9930b-d80c-4227-b6c5-d87cd3597ef7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:37.202412 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-44e9930b-d80c-4227-b6c5-d87cd3597ef7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:37.203261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-44e9930b-d80c-4227-b6c5-d87cd3597ef7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:37.204288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-44e9930b-d80c-4227-b6c5-d87cd3597ef7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:37.204895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-44e9930b-d80c-4227-b6c5-d87cd3597ef7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:37.205473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-44e9930b-d80c-4227-b6c5-d87cd3597ef7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:37.421991 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-44e9930b-d80c-4227-b6c5-d87cd3597ef7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:37.434148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-44e9930b-d80c-4227-b6c5-d87cd3597ef7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1366 microversion: 2.1 time: 0.255874 Aug 28 00:58:37.435956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 234/463] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:37 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1366 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:37.622618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1771288a-38d1-4a90-b195-1f4296b0ac8d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:37.624140 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1771288a-38d1-4a90-b195-1f4296b0ac8d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:37.636261 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1771288a-38d1-4a90-b195-1f4296b0ac8d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:37.637368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1771288a-38d1-4a90-b195-1f4296b0ac8d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:37.639546 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1771288a-38d1-4a90-b195-1f4296b0ac8d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:37.641617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1771288a-38d1-4a90-b195-1f4296b0ac8d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:37.643126 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1771288a-38d1-4a90-b195-1f4296b0ac8d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:37.644512 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1771288a-38d1-4a90-b195-1f4296b0ac8d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:37.896001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1771288a-38d1-4a90-b195-1f4296b0ac8d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:37.915850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1771288a-38d1-4a90-b195-1f4296b0ac8d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1724 microversion: 2.1 time: 0.300823 Aug 28 00:58:37.916875 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 230/464] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:37 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1724 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:37.918100 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-aeb641f1-2357-4a23-9d50-49d2ddfb5b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:37.920404 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-aeb641f1-2357-4a23-9d50-49d2ddfb5b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:37.938226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aeb641f1-2357-4a23-9d50-49d2ddfb5b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:37.940148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aeb641f1-2357-4a23-9d50-49d2ddfb5b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:37.941344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aeb641f1-2357-4a23-9d50-49d2ddfb5b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:37.943819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aeb641f1-2357-4a23-9d50-49d2ddfb5b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:37.945142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aeb641f1-2357-4a23-9d50-49d2ddfb5b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:37.946146 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aeb641f1-2357-4a23-9d50-49d2ddfb5b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:38.175171 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a075306a-5b7a-461f-8ac6-704e341cce35 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:38.178005 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a075306a-5b7a-461f-8ac6-704e341cce35 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:38.201249 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a075306a-5b7a-461f-8ac6-704e341cce35 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:38.202546 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a075306a-5b7a-461f-8ac6-704e341cce35 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:38.203568 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a075306a-5b7a-461f-8ac6-704e341cce35 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:38.205190 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a075306a-5b7a-461f-8ac6-704e341cce35 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:38.206136 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a075306a-5b7a-461f-8ac6-704e341cce35 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:38.207202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a075306a-5b7a-461f-8ac6-704e341cce35 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:38.215412 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-aeb641f1-2357-4a23-9d50-49d2ddfb5b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:38.231579 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-aeb641f1-2357-4a23-9d50-49d2ddfb5b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:38.234785 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-aeb641f1-2357-4a23-9d50-49d2ddfb5b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:38.238232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-aeb641f1-2357-4a23-9d50-49d2ddfb5b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1737 microversion: 2.20 time: 0.325062 Aug 28 00:58:38.240156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 235/465] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:37 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1737 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:58:38.332458 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a075306a-5b7a-461f-8ac6-704e341cce35 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:38.374111 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a075306a-5b7a-461f-8ac6-704e341cce35 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1222 microversion: 2.1 time: 0.207471 Aug 28 00:58:38.389400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 231/466] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:38 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1222 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:58:38.475091 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-80964ea9-fe30-4fcf-86a7-141b0ccedd01 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:38.476453 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-80964ea9-fe30-4fcf-86a7-141b0ccedd01 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:38.490638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-80964ea9-fe30-4fcf-86a7-141b0ccedd01 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:38.492245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-80964ea9-fe30-4fcf-86a7-141b0ccedd01 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:38.493310 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-80964ea9-fe30-4fcf-86a7-141b0ccedd01 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:38.495158 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-80964ea9-fe30-4fcf-86a7-141b0ccedd01 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:38.496380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-80964ea9-fe30-4fcf-86a7-141b0ccedd01 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:38.497314 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-80964ea9-fe30-4fcf-86a7-141b0ccedd01 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:38.714506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-80964ea9-fe30-4fcf-86a7-141b0ccedd01 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:38.725059 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-80964ea9-fe30-4fcf-86a7-141b0ccedd01 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1366 microversion: 2.1 time: 0.258206 Aug 28 00:58:38.725943 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 236/467] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:38 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1366 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:38.954980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0cd3b944-1cf9-45c1-af77-d952a921f5ac tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:38.955527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0cd3b944-1cf9-45c1-af77-d952a921f5ac tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:38.963705 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0cd3b944-1cf9-45c1-af77-d952a921f5ac tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:38.964191 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0cd3b944-1cf9-45c1-af77-d952a921f5ac tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:38.965070 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0cd3b944-1cf9-45c1-af77-d952a921f5ac tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:38.966009 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0cd3b944-1cf9-45c1-af77-d952a921f5ac tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:38.966798 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0cd3b944-1cf9-45c1-af77-d952a921f5ac tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:38.967617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0cd3b944-1cf9-45c1-af77-d952a921f5ac tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:39.212305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0cd3b944-1cf9-45c1-af77-d952a921f5ac tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:39.225944 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0cd3b944-1cf9-45c1-af77-d952a921f5ac tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1724 microversion: 2.1 time: 0.275491 Aug 28 00:58:39.227353 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 232/468] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:38 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1724 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:39.268196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9538cf73-26e6-433b-b5ad-58b2cdc252f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:39.269297 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9538cf73-26e6-433b-b5ad-58b2cdc252f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:39.280704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9538cf73-26e6-433b-b5ad-58b2cdc252f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:39.281458 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9538cf73-26e6-433b-b5ad-58b2cdc252f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:39.282201 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9538cf73-26e6-433b-b5ad-58b2cdc252f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:39.283697 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9538cf73-26e6-433b-b5ad-58b2cdc252f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:39.284879 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9538cf73-26e6-433b-b5ad-58b2cdc252f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:39.285551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9538cf73-26e6-433b-b5ad-58b2cdc252f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:39.412607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2534451f-cd9f-4b45-a37b-033b6bb42b0a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:39.413417 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2534451f-cd9f-4b45-a37b-033b6bb42b0a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:39.425162 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2534451f-cd9f-4b45-a37b-033b6bb42b0a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:39.425162 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2534451f-cd9f-4b45-a37b-033b6bb42b0a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:39.425162 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2534451f-cd9f-4b45-a37b-033b6bb42b0a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:39.425982 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2534451f-cd9f-4b45-a37b-033b6bb42b0a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:39.426389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2534451f-cd9f-4b45-a37b-033b6bb42b0a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:39.426978 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2534451f-cd9f-4b45-a37b-033b6bb42b0a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:39.589666 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2534451f-cd9f-4b45-a37b-033b6bb42b0a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:39.600303 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2534451f-cd9f-4b45-a37b-033b6bb42b0a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1214 microversion: 2.1 time: 0.195035 Aug 28 00:58:39.618901 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 233/469] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:39 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1214 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:39.767758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1255463c-9cb2-46ba-ae12-7e46a983457b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:39.768472 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1255463c-9cb2-46ba-ae12-7e46a983457b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:39.781092 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1255463c-9cb2-46ba-ae12-7e46a983457b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:39.781730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1255463c-9cb2-46ba-ae12-7e46a983457b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:39.782277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1255463c-9cb2-46ba-ae12-7e46a983457b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:39.790060 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1255463c-9cb2-46ba-ae12-7e46a983457b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:39.791013 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1255463c-9cb2-46ba-ae12-7e46a983457b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:39.791591 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1255463c-9cb2-46ba-ae12-7e46a983457b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:39.823952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9538cf73-26e6-433b-b5ad-58b2cdc252f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:39.837506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9538cf73-26e6-433b-b5ad-58b2cdc252f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:39.844067 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9538cf73-26e6-433b-b5ad-58b2cdc252f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:39.848328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9538cf73-26e6-433b-b5ad-58b2cdc252f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1737 microversion: 2.20 time: 0.586994 Aug 28 00:58:39.849129 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 237/470] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:39 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1737 bytes in 589 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:40.104686 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1255463c-9cb2-46ba-ae12-7e46a983457b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:40.142096 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1255463c-9cb2-46ba-ae12-7e46a983457b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1366 microversion: 2.1 time: 0.378138 Aug 28 00:58:40.143881 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 234/471] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:39 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1366 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:58:40.254859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-013d5543-c0ea-4b09-968d-23f70e1e759f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:40.255485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-013d5543-c0ea-4b09-968d-23f70e1e759f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:40.263343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-013d5543-c0ea-4b09-968d-23f70e1e759f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:40.264128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-013d5543-c0ea-4b09-968d-23f70e1e759f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:40.264783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-013d5543-c0ea-4b09-968d-23f70e1e759f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:40.265838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-013d5543-c0ea-4b09-968d-23f70e1e759f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:40.266666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-013d5543-c0ea-4b09-968d-23f70e1e759f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:40.267261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-013d5543-c0ea-4b09-968d-23f70e1e759f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:40.507778 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-013d5543-c0ea-4b09-968d-23f70e1e759f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:40.519983 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-013d5543-c0ea-4b09-968d-23f70e1e759f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1724 microversion: 2.1 time: 0.269044 Aug 28 00:58:40.533049 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 238/472] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:40 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1724 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:40.633234 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a5dcbfff-f15f-44cb-a92d-ac88a8c4a1b4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:40.635928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a5dcbfff-f15f-44cb-a92d-ac88a8c4a1b4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:40.651309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5dcbfff-f15f-44cb-a92d-ac88a8c4a1b4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:40.652618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5dcbfff-f15f-44cb-a92d-ac88a8c4a1b4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:40.653796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5dcbfff-f15f-44cb-a92d-ac88a8c4a1b4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:40.658440 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5dcbfff-f15f-44cb-a92d-ac88a8c4a1b4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:40.658440 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5dcbfff-f15f-44cb-a92d-ac88a8c4a1b4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:40.658440 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5dcbfff-f15f-44cb-a92d-ac88a8c4a1b4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:40.806401 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a5dcbfff-f15f-44cb-a92d-ac88a8c4a1b4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:40.835507 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a5dcbfff-f15f-44cb-a92d-ac88a8c4a1b4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1270 microversion: 2.1 time: 0.208256 Aug 28 00:58:40.841912 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 235/473] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:40 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1270 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:40.913098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b9ecc934-9c0e-4a6b-ad6a-e54f28a3938b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:40.914675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b9ecc934-9c0e-4a6b-ad6a-e54f28a3938b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:40.925523 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9ecc934-9c0e-4a6b-ad6a-e54f28a3938b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:40.926871 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9ecc934-9c0e-4a6b-ad6a-e54f28a3938b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:40.927916 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9ecc934-9c0e-4a6b-ad6a-e54f28a3938b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:40.929463 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9ecc934-9c0e-4a6b-ad6a-e54f28a3938b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:40.930541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9ecc934-9c0e-4a6b-ad6a-e54f28a3938b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:40.931570 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9ecc934-9c0e-4a6b-ad6a-e54f28a3938b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:41.169226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f95aec9d-2a1e-41df-b0a6-d9125ab79888 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:41.170586 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f95aec9d-2a1e-41df-b0a6-d9125ab79888 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:41.192174 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f95aec9d-2a1e-41df-b0a6-d9125ab79888 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:41.193573 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f95aec9d-2a1e-41df-b0a6-d9125ab79888 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:41.194647 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f95aec9d-2a1e-41df-b0a6-d9125ab79888 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:41.196237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f95aec9d-2a1e-41df-b0a6-d9125ab79888 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:41.197765 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f95aec9d-2a1e-41df-b0a6-d9125ab79888 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:41.202170 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f95aec9d-2a1e-41df-b0a6-d9125ab79888 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:41.462653 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b9ecc934-9c0e-4a6b-ad6a-e54f28a3938b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:41.478286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b9ecc934-9c0e-4a6b-ad6a-e54f28a3938b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:41.480365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b9ecc934-9c0e-4a6b-ad6a-e54f28a3938b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:41.481884 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b9ecc934-9c0e-4a6b-ad6a-e54f28a3938b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1737 microversion: 2.20 time: 0.578821 Aug 28 00:58:41.483476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 239/474] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:40 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1737 bytes in 582 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:41.558014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fdddfbb1-cb5f-428a-9cf9-f36759d091f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:41.559008 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fdddfbb1-cb5f-428a-9cf9-f36759d091f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:41.572270 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fdddfbb1-cb5f-428a-9cf9-f36759d091f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:41.572996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fdddfbb1-cb5f-428a-9cf9-f36759d091f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:41.573644 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fdddfbb1-cb5f-428a-9cf9-f36759d091f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:41.574326 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fdddfbb1-cb5f-428a-9cf9-f36759d091f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:41.574826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fdddfbb1-cb5f-428a-9cf9-f36759d091f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:41.579223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fdddfbb1-cb5f-428a-9cf9-f36759d091f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:41.754590 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f95aec9d-2a1e-41df-b0a6-d9125ab79888 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:41.773809 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f95aec9d-2a1e-41df-b0a6-d9125ab79888 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1366 microversion: 2.1 time: 0.610225 Aug 28 00:58:41.782387 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 236/475] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:41 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1366 bytes in 620 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:58:41.921960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-120aa186-ba18-446e-b95b-e2e9bb446989 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:41.923051 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-120aa186-ba18-446e-b95b-e2e9bb446989 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:41.930844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-120aa186-ba18-446e-b95b-e2e9bb446989 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:41.931941 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-120aa186-ba18-446e-b95b-e2e9bb446989 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:41.932610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-120aa186-ba18-446e-b95b-e2e9bb446989 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:41.933679 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-120aa186-ba18-446e-b95b-e2e9bb446989 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:41.934317 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-120aa186-ba18-446e-b95b-e2e9bb446989 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:41.935188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-120aa186-ba18-446e-b95b-e2e9bb446989 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:42.086211 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-120aa186-ba18-446e-b95b-e2e9bb446989 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:42.107595 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fdddfbb1-cb5f-428a-9cf9-f36759d091f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:42.112223 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-120aa186-ba18-446e-b95b-e2e9bb446989 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1276 microversion: 2.1 time: 0.193553 Aug 28 00:58:42.112223 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 237/476] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:41 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1276 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:58:42.166591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fdddfbb1-cb5f-428a-9cf9-f36759d091f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1724 microversion: 2.1 time: 0.612185 Aug 28 00:58:42.188940 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 240/477] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:41 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1724 bytes in 636 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:58:42.528517 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-53b5e3da-3cce-4a78-ac08-2264d417f1e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:42.529739 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-53b5e3da-3cce-4a78-ac08-2264d417f1e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:42.538524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53b5e3da-3cce-4a78-ac08-2264d417f1e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:42.539280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53b5e3da-3cce-4a78-ac08-2264d417f1e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:42.539915 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53b5e3da-3cce-4a78-ac08-2264d417f1e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:42.540658 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53b5e3da-3cce-4a78-ac08-2264d417f1e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:42.541139 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53b5e3da-3cce-4a78-ac08-2264d417f1e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:42.541747 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53b5e3da-3cce-4a78-ac08-2264d417f1e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:42.810748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-db110f60-cd87-45c4-b0d7-5570023b66ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:42.814346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-db110f60-cd87-45c4-b0d7-5570023b66ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:42.829142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db110f60-cd87-45c4-b0d7-5570023b66ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:42.831430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db110f60-cd87-45c4-b0d7-5570023b66ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:42.832468 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db110f60-cd87-45c4-b0d7-5570023b66ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:42.833971 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db110f60-cd87-45c4-b0d7-5570023b66ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:42.834972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db110f60-cd87-45c4-b0d7-5570023b66ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:42.835923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db110f60-cd87-45c4-b0d7-5570023b66ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:42.887996 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-53b5e3da-3cce-4a78-ac08-2264d417f1e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:42.925126 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-53b5e3da-3cce-4a78-ac08-2264d417f1e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:42.932833 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-53b5e3da-3cce-4a78-ac08-2264d417f1e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:42.934686 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-53b5e3da-3cce-4a78-ac08-2264d417f1e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1737 microversion: 2.20 time: 0.410516 Aug 28 00:58:42.935404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 238/478] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:42 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1737 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:43.133884 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-db110f60-cd87-45c4-b0d7-5570023b66ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:43.145403 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3d8ba382-e281-44fa-80f9-6c773240e939 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:43.146477 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3d8ba382-e281-44fa-80f9-6c773240e939 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:43.162865 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-db110f60-cd87-45c4-b0d7-5570023b66ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1366 microversion: 2.1 time: 0.359886 Aug 28 00:58:43.164877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 241/479] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:42 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1366 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:58:43.188468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d8ba382-e281-44fa-80f9-6c773240e939 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:43.189400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d8ba382-e281-44fa-80f9-6c773240e939 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:43.190068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d8ba382-e281-44fa-80f9-6c773240e939 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:43.191249 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d8ba382-e281-44fa-80f9-6c773240e939 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:43.192063 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d8ba382-e281-44fa-80f9-6c773240e939 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:43.192857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d8ba382-e281-44fa-80f9-6c773240e939 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:43.217778 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-40b2fe16-f0f8-487c-ad50-a9f07399468b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:43.218827 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-40b2fe16-f0f8-487c-ad50-a9f07399468b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:43.237150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-40b2fe16-f0f8-487c-ad50-a9f07399468b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:43.237457 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-40b2fe16-f0f8-487c-ad50-a9f07399468b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:43.237879 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-40b2fe16-f0f8-487c-ad50-a9f07399468b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:43.238624 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-40b2fe16-f0f8-487c-ad50-a9f07399468b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:43.239095 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-40b2fe16-f0f8-487c-ad50-a9f07399468b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:43.239525 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-40b2fe16-f0f8-487c-ad50-a9f07399468b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:43.636034 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3d8ba382-e281-44fa-80f9-6c773240e939 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:43.663943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3d8ba382-e281-44fa-80f9-6c773240e939 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1318 microversion: 2.1 time: 0.522466 Aug 28 00:58:43.666267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 239/480] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:43 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1318 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:58:43.692459 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-40b2fe16-f0f8-487c-ad50-a9f07399468b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:43.743409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-40b2fe16-f0f8-487c-ad50-a9f07399468b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1724 microversion: 2.1 time: 0.525516 Aug 28 00:58:43.746564 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 242/481] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:43 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1724 bytes in 538 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:58:43.963361 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1166caf9-895d-4a6e-bd83-8cb633dd1248 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:43.965636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1166caf9-895d-4a6e-bd83-8cb633dd1248 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:43.980420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1166caf9-895d-4a6e-bd83-8cb633dd1248 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:43.981292 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1166caf9-895d-4a6e-bd83-8cb633dd1248 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:43.981943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1166caf9-895d-4a6e-bd83-8cb633dd1248 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:43.985006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1166caf9-895d-4a6e-bd83-8cb633dd1248 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:43.985736 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1166caf9-895d-4a6e-bd83-8cb633dd1248 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:43.986985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1166caf9-895d-4a6e-bd83-8cb633dd1248 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:44.044251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-9e0bd8de-187f-4064-86b1-21cee14962ab req-ed681295-661b-4778-a2d4-f0fa3d517684 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "27028458-f872-49aa-8d37-1a886d566c00", "name": "network-vif-plugged", "status": "completed", "tag": "59c8cffa-e3e1-4a46-979a-94e0428e33d6"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:58:44.093586 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-9e0bd8de-187f-4064-86b1-21cee14962ab req-ed681295-661b-4778-a2d4-f0fa3d517684 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:44.094073 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-9e0bd8de-187f-4064-86b1-21cee14962ab req-ed681295-661b-4778-a2d4-f0fa3d517684 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:44.094503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-9e0bd8de-187f-4064-86b1-21cee14962ab req-ed681295-661b-4778-a2d4-f0fa3d517684 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:44.121177 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-9e0bd8de-187f-4064-86b1-21cee14962ab req-ed681295-661b-4778-a2d4-f0fa3d517684 service nova] Creating event network-vif-plugged:59c8cffa-e3e1-4a46-979a-94e0428e33d6 for instance 27028458-f872-49aa-8d37-1a886d566c00 on hv-cinder-maste Aug 28 00:58:44.130749 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-9e0bd8de-187f-4064-86b1-21cee14962ab req-ed681295-661b-4778-a2d4-f0fa3d517684 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.099137 Aug 28 00:58:44.137104 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 243/482] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:58:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:58:44.212614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-64692bf0-329a-4e7d-93c6-b8aef839fc81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:44.214287 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-64692bf0-329a-4e7d-93c6-b8aef839fc81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:44.243979 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64692bf0-329a-4e7d-93c6-b8aef839fc81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:44.245295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64692bf0-329a-4e7d-93c6-b8aef839fc81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:44.246437 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64692bf0-329a-4e7d-93c6-b8aef839fc81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:44.247936 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64692bf0-329a-4e7d-93c6-b8aef839fc81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:44.250364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64692bf0-329a-4e7d-93c6-b8aef839fc81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:44.251201 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64692bf0-329a-4e7d-93c6-b8aef839fc81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:44.260674 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1166caf9-895d-4a6e-bd83-8cb633dd1248 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:44.274188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1166caf9-895d-4a6e-bd83-8cb633dd1248 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:44.275924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1166caf9-895d-4a6e-bd83-8cb633dd1248 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:44.278487 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1166caf9-895d-4a6e-bd83-8cb633dd1248 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1737 microversion: 2.20 time: 0.319342 Aug 28 00:58:44.279078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 240/483] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:43 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1737 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:44.693145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-24b7020c-bc18-4bec-beb9-4e20e97d2932 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:44.702843 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-24b7020c-bc18-4bec-beb9-4e20e97d2932 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:44.708843 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=77,nova_api:INSERT=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:58:44.717150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24b7020c-bc18-4bec-beb9-4e20e97d2932 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:44.718403 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24b7020c-bc18-4bec-beb9-4e20e97d2932 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:44.719372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24b7020c-bc18-4bec-beb9-4e20e97d2932 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:44.721887 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24b7020c-bc18-4bec-beb9-4e20e97d2932 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:44.723191 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24b7020c-bc18-4bec-beb9-4e20e97d2932 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:44.724205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24b7020c-bc18-4bec-beb9-4e20e97d2932 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:44.734296 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-64692bf0-329a-4e7d-93c6-b8aef839fc81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:44.760808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-64692bf0-329a-4e7d-93c6-b8aef839fc81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1366 microversion: 2.1 time: 0.560938 Aug 28 00:58:44.761816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 244/484] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:44 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1366 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:44.787492 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bf03633a-5e02-4015-a655-d334a95473a1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:44.788458 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bf03633a-5e02-4015-a655-d334a95473a1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:44.811909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf03633a-5e02-4015-a655-d334a95473a1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:44.814848 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf03633a-5e02-4015-a655-d334a95473a1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:44.815888 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf03633a-5e02-4015-a655-d334a95473a1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:44.817577 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf03633a-5e02-4015-a655-d334a95473a1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:44.818891 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf03633a-5e02-4015-a655-d334a95473a1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:44.820113 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf03633a-5e02-4015-a655-d334a95473a1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:45.484726 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-24b7020c-bc18-4bec-beb9-4e20e97d2932 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:45.487603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bf03633a-5e02-4015-a655-d334a95473a1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:45.491584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:58:45.507515 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-24b7020c-bc18-4bec-beb9-4e20e97d2932 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1318 microversion: 2.1 time: 0.821422 Aug 28 00:58:45.510297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 241/485] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:44 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1318 bytes in 826 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:45.517488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bf03633a-5e02-4015-a655-d334a95473a1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1724 microversion: 2.1 time: 0.734236 Aug 28 00:58:45.522030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 245/486] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:44 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1724 bytes in 748 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:45.536684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-07105068-3295-43ce-b42f-58dd94c9e933 req-00091325-23f2-429a-a6b1-05587420f1a6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2cbc0806-fe4b-4018-93bf-b4cf3c292bdc", "tag": "a816f8eb-89c2-45b9-b35d-cf9bca6f4a7c"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:58:45.538119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-540cae70-84d8-4a47-8f1f-a8ed4ad3bf78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:45.538800 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-540cae70-84d8-4a47-8f1f-a8ed4ad3bf78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:45.547422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-540cae70-84d8-4a47-8f1f-a8ed4ad3bf78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:45.547925 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-540cae70-84d8-4a47-8f1f-a8ed4ad3bf78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:45.548366 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-540cae70-84d8-4a47-8f1f-a8ed4ad3bf78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:45.555039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-540cae70-84d8-4a47-8f1f-a8ed4ad3bf78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:45.555448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-540cae70-84d8-4a47-8f1f-a8ed4ad3bf78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:45.556076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-540cae70-84d8-4a47-8f1f-a8ed4ad3bf78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:45.561291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-07105068-3295-43ce-b42f-58dd94c9e933 req-00091325-23f2-429a-a6b1-05587420f1a6 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:45.562219 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-07105068-3295-43ce-b42f-58dd94c9e933 req-00091325-23f2-429a-a6b1-05587420f1a6 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:45.564044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-07105068-3295-43ce-b42f-58dd94c9e933 req-00091325-23f2-429a-a6b1-05587420f1a6 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:45.587688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-07105068-3295-43ce-b42f-58dd94c9e933 req-00091325-23f2-429a-a6b1-05587420f1a6 service nova] Creating event network-changed:a816f8eb-89c2-45b9-b35d-cf9bca6f4a7c for instance 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc on hv-cinder-maste Aug 28 00:58:45.598792 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-07105068-3295-43ce-b42f-58dd94c9e933 req-00091325-23f2-429a-a6b1-05587420f1a6 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.085872 Aug 28 00:58:45.601277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 242/487] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:58:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:58:45.799103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-540cae70-84d8-4a47-8f1f-a8ed4ad3bf78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:45.806541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9b5ec34e-eae7-4d01-9336-43aef8ab7dd8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:45.810947 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9b5ec34e-eae7-4d01-9336-43aef8ab7dd8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:45.816610 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-540cae70-84d8-4a47-8f1f-a8ed4ad3bf78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:45.826012 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-540cae70-84d8-4a47-8f1f-a8ed4ad3bf78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:45.828490 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-540cae70-84d8-4a47-8f1f-a8ed4ad3bf78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1737 microversion: 2.20 time: 0.304457 Aug 28 00:58:45.838692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 246/488] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:45 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1737 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:45.839271 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b5ec34e-eae7-4d01-9336-43aef8ab7dd8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:45.841419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b5ec34e-eae7-4d01-9336-43aef8ab7dd8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:45.844820 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b5ec34e-eae7-4d01-9336-43aef8ab7dd8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:45.850849 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b5ec34e-eae7-4d01-9336-43aef8ab7dd8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:45.852235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b5ec34e-eae7-4d01-9336-43aef8ab7dd8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:45.853389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b5ec34e-eae7-4d01-9336-43aef8ab7dd8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:46.089780 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9b5ec34e-eae7-4d01-9336-43aef8ab7dd8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:46.100409 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9b5ec34e-eae7-4d01-9336-43aef8ab7dd8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1366 microversion: 2.1 time: 0.302309 Aug 28 00:58:46.101396 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 243/489] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:45 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1366 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:46.541656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e2e867b5-3973-4069-bec0-6527bf07de71 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:46.542685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e2e867b5-3973-4069-bec0-6527bf07de71 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:46.550978 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2e867b5-3973-4069-bec0-6527bf07de71 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:46.551492 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2e867b5-3973-4069-bec0-6527bf07de71 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:46.551915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2e867b5-3973-4069-bec0-6527bf07de71 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:46.552646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2e867b5-3973-4069-bec0-6527bf07de71 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:46.553219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2e867b5-3973-4069-bec0-6527bf07de71 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:46.553649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2e867b5-3973-4069-bec0-6527bf07de71 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:46.556092 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6dd69b98-e0a1-4901-860e-43efed2a6e2e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:46.557307 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6dd69b98-e0a1-4901-860e-43efed2a6e2e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:46.567178 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6dd69b98-e0a1-4901-860e-43efed2a6e2e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:46.567848 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6dd69b98-e0a1-4901-860e-43efed2a6e2e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:46.568492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6dd69b98-e0a1-4901-860e-43efed2a6e2e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:46.569595 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6dd69b98-e0a1-4901-860e-43efed2a6e2e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:46.570329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6dd69b98-e0a1-4901-860e-43efed2a6e2e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:46.571052 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6dd69b98-e0a1-4901-860e-43efed2a6e2e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:46.774757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=4 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:58:46.789333 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e2e867b5-3973-4069-bec0-6527bf07de71 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:46.793431 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6dd69b98-e0a1-4901-860e-43efed2a6e2e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:46.798276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e2e867b5-3973-4069-bec0-6527bf07de71 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1318 microversion: 2.1 time: 0.261363 Aug 28 00:58:46.798877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 247/490] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:46 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1318 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:46.805739 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6dd69b98-e0a1-4901-860e-43efed2a6e2e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1724 microversion: 2.1 time: 0.256930 Aug 28 00:58:46.806848 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 244/491] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:46 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1724 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:46.868670 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7687bb3c-1b98-4a27-803e-921f164f1294 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:46.869499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7687bb3c-1b98-4a27-803e-921f164f1294 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:46.871569 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=14 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:58:46.877581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7687bb3c-1b98-4a27-803e-921f164f1294 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:46.878067 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7687bb3c-1b98-4a27-803e-921f164f1294 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:46.878527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7687bb3c-1b98-4a27-803e-921f164f1294 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:46.879334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7687bb3c-1b98-4a27-803e-921f164f1294 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:46.879718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7687bb3c-1b98-4a27-803e-921f164f1294 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:46.880145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7687bb3c-1b98-4a27-803e-921f164f1294 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:46.885260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:58:47.036200 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7687bb3c-1b98-4a27-803e-921f164f1294 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:47.045418 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7687bb3c-1b98-4a27-803e-921f164f1294 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:47.046912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7687bb3c-1b98-4a27-803e-921f164f1294 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:47.048107 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7687bb3c-1b98-4a27-803e-921f164f1294 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1737 microversion: 2.20 time: 0.183696 Aug 28 00:58:47.048686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 248/492] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:46 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1737 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:47.122911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d6a71de1-4c6f-4bac-8288-2a9fa9986991 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:47.123904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d6a71de1-4c6f-4bac-8288-2a9fa9986991 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:47.132757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6a71de1-4c6f-4bac-8288-2a9fa9986991 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:47.133630 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6a71de1-4c6f-4bac-8288-2a9fa9986991 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:47.134449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6a71de1-4c6f-4bac-8288-2a9fa9986991 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:47.135121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6a71de1-4c6f-4bac-8288-2a9fa9986991 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:47.135588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6a71de1-4c6f-4bac-8288-2a9fa9986991 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:47.136125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6a71de1-4c6f-4bac-8288-2a9fa9986991 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:47.137117 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=2 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:58:47.295227 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d6a71de1-4c6f-4bac-8288-2a9fa9986991 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:47.304058 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d6a71de1-4c6f-4bac-8288-2a9fa9986991 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1557 microversion: 2.1 time: 0.185091 Aug 28 00:58:47.304658 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 245/493] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:47 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1557 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:47.325941 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0818dc06-4041-493b-911e-0f6adfa4f48b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:47.326443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0818dc06-4041-493b-911e-0f6adfa4f48b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:47.333249 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0818dc06-4041-493b-911e-0f6adfa4f48b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:47.333742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0818dc06-4041-493b-911e-0f6adfa4f48b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:47.334181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0818dc06-4041-493b-911e-0f6adfa4f48b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:47.334912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0818dc06-4041-493b-911e-0f6adfa4f48b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:47.335378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0818dc06-4041-493b-911e-0f6adfa4f48b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:47.335803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0818dc06-4041-493b-911e-0f6adfa4f48b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:47.421404 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0818dc06-4041-493b-911e-0f6adfa4f48b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.100049 Aug 28 00:58:47.422177 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 249/494] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Aug 28 00:58:47 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00/os-interface => generated 285 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:58:47.830622 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-48424d33-486a-49be-83c8-7ef2fd212c5d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:47.831304 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-48424d33-486a-49be-83c8-7ef2fd212c5d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:47.837700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b32404b7-f915-46a5-8e94-2c82b2ee3dcf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:47.838521 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b32404b7-f915-46a5-8e94-2c82b2ee3dcf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:47.852612 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b32404b7-f915-46a5-8e94-2c82b2ee3dcf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:47.855398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48424d33-486a-49be-83c8-7ef2fd212c5d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:47.855914 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48424d33-486a-49be-83c8-7ef2fd212c5d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:47.856515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48424d33-486a-49be-83c8-7ef2fd212c5d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:47.857524 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48424d33-486a-49be-83c8-7ef2fd212c5d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:47.858300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48424d33-486a-49be-83c8-7ef2fd212c5d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:47.859011 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48424d33-486a-49be-83c8-7ef2fd212c5d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:47.860444 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b32404b7-f915-46a5-8e94-2c82b2ee3dcf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:47.861140 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b32404b7-f915-46a5-8e94-2c82b2ee3dcf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:47.862352 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b32404b7-f915-46a5-8e94-2c82b2ee3dcf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:47.863139 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b32404b7-f915-46a5-8e94-2c82b2ee3dcf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:47.863911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b32404b7-f915-46a5-8e94-2c82b2ee3dcf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:48.155926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-48424d33-486a-49be-83c8-7ef2fd212c5d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:48.165919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b32404b7-f915-46a5-8e94-2c82b2ee3dcf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:48.174322 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-48424d33-486a-49be-83c8-7ef2fd212c5d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1724 microversion: 2.1 time: 0.349151 Aug 28 00:58:48.175254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 250/495] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:47 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1724 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:58:48.179955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-92fca35b-c06c-4c21-af7e-0beaba4fac32 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:48.180600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-92fca35b-c06c-4c21-af7e-0beaba4fac32 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:48.180767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b32404b7-f915-46a5-8e94-2c82b2ee3dcf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1318 microversion: 2.1 time: 0.360794 Aug 28 00:58:48.181784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 246/496] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:47 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1318 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:58:48.187355 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92fca35b-c06c-4c21-af7e-0beaba4fac32 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:48.187852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92fca35b-c06c-4c21-af7e-0beaba4fac32 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:48.188300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92fca35b-c06c-4c21-af7e-0beaba4fac32 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:48.189202 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92fca35b-c06c-4c21-af7e-0beaba4fac32 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:48.189514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92fca35b-c06c-4c21-af7e-0beaba4fac32 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:48.189941 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92fca35b-c06c-4c21-af7e-0beaba4fac32 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:48.380658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-92fca35b-c06c-4c21-af7e-0beaba4fac32 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:48.390602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-92fca35b-c06c-4c21-af7e-0beaba4fac32 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:48.392285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-92fca35b-c06c-4c21-af7e-0beaba4fac32 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:48.393799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-92fca35b-c06c-4c21-af7e-0beaba4fac32 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1719 microversion: 2.20 time: 0.217781 Aug 28 00:58:48.394940 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 251/497] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:48 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1719 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:48.643824 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-1a6b0d4d-3ad6-4e77-be68-b5613f61fa57 req-95a464d7-d9e7-4063-944e-77b79a48db5b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "27028458-f872-49aa-8d37-1a886d566c00", "tag": "59c8cffa-e3e1-4a46-979a-94e0428e33d6"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:58:48.654663 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-1a6b0d4d-3ad6-4e77-be68-b5613f61fa57 req-95a464d7-d9e7-4063-944e-77b79a48db5b service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:48.655156 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-1a6b0d4d-3ad6-4e77-be68-b5613f61fa57 req-95a464d7-d9e7-4063-944e-77b79a48db5b service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:48.655630 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-1a6b0d4d-3ad6-4e77-be68-b5613f61fa57 req-95a464d7-d9e7-4063-944e-77b79a48db5b service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:48.668938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-1a6b0d4d-3ad6-4e77-be68-b5613f61fa57 req-95a464d7-d9e7-4063-944e-77b79a48db5b service nova] Creating event network-changed:59c8cffa-e3e1-4a46-979a-94e0428e33d6 for instance 27028458-f872-49aa-8d37-1a886d566c00 on hv-cinder-maste Aug 28 00:58:48.674620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-1a6b0d4d-3ad6-4e77-be68-b5613f61fa57 req-95a464d7-d9e7-4063-944e-77b79a48db5b service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034512 Aug 28 00:58:48.675253 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 247/498] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:58:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:58:49.196293 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-70483630-d4fe-4c9d-a474-a49b6d7b21eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:49.197217 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-70483630-d4fe-4c9d-a474-a49b6d7b21eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:49.204403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70483630-d4fe-4c9d-a474-a49b6d7b21eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:49.204863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70483630-d4fe-4c9d-a474-a49b6d7b21eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:49.205695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70483630-d4fe-4c9d-a474-a49b6d7b21eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:49.206437 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70483630-d4fe-4c9d-a474-a49b6d7b21eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:49.206920 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70483630-d4fe-4c9d-a474-a49b6d7b21eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:49.207347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70483630-d4fe-4c9d-a474-a49b6d7b21eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:49.211513 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3c88b8f1-710d-42e7-87f8-91dd6559491d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:49.213042 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3c88b8f1-710d-42e7-87f8-91dd6559491d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:49.221381 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c88b8f1-710d-42e7-87f8-91dd6559491d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:49.222160 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c88b8f1-710d-42e7-87f8-91dd6559491d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:49.222778 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c88b8f1-710d-42e7-87f8-91dd6559491d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:49.223497 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c88b8f1-710d-42e7-87f8-91dd6559491d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:49.223992 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c88b8f1-710d-42e7-87f8-91dd6559491d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:49.224412 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c88b8f1-710d-42e7-87f8-91dd6559491d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:49.412776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-70483630-d4fe-4c9d-a474-a49b6d7b21eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:49.423475 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-70483630-d4fe-4c9d-a474-a49b6d7b21eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1724 microversion: 2.1 time: 0.231079 Aug 28 00:58:49.437554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 252/499] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:49 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1724 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:49.437554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4bff77c6-30a8-4eec-840c-b9a2437edc4b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:49.437554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4bff77c6-30a8-4eec-840c-b9a2437edc4b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:49.437554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bff77c6-30a8-4eec-840c-b9a2437edc4b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:49.438192 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bff77c6-30a8-4eec-840c-b9a2437edc4b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:49.438192 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bff77c6-30a8-4eec-840c-b9a2437edc4b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:49.438447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bff77c6-30a8-4eec-840c-b9a2437edc4b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:49.438866 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bff77c6-30a8-4eec-840c-b9a2437edc4b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:49.439437 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bff77c6-30a8-4eec-840c-b9a2437edc4b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:49.468118 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3c88b8f1-710d-42e7-87f8-91dd6559491d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:49.482922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3c88b8f1-710d-42e7-87f8-91dd6559491d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1318 microversion: 2.1 time: 0.275096 Aug 28 00:58:49.483940 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 248/500] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:49 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1318 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:49.644360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4bff77c6-30a8-4eec-840c-b9a2437edc4b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:49.654178 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4bff77c6-30a8-4eec-840c-b9a2437edc4b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:49.655693 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4bff77c6-30a8-4eec-840c-b9a2437edc4b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:49.657006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4bff77c6-30a8-4eec-840c-b9a2437edc4b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1719 microversion: 2.20 time: 0.231873 Aug 28 00:58:49.657795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 253/501] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:49 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1719 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:50.456368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e8cb1574-7254-467e-b01c-c592c3b9e368 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:50.457169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e8cb1574-7254-467e-b01c-c592c3b9e368 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:50.466398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8cb1574-7254-467e-b01c-c592c3b9e368 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:50.466890 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8cb1574-7254-467e-b01c-c592c3b9e368 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:50.467320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8cb1574-7254-467e-b01c-c592c3b9e368 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:50.468030 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8cb1574-7254-467e-b01c-c592c3b9e368 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:50.468481 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8cb1574-7254-467e-b01c-c592c3b9e368 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:50.468902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8cb1574-7254-467e-b01c-c592c3b9e368 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:50.515091 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cdeb4d96-71ef-44f8-956f-fa4e94bbfb6e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:50.515770 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cdeb4d96-71ef-44f8-956f-fa4e94bbfb6e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:50.526011 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cdeb4d96-71ef-44f8-956f-fa4e94bbfb6e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:50.526577 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cdeb4d96-71ef-44f8-956f-fa4e94bbfb6e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:50.527182 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cdeb4d96-71ef-44f8-956f-fa4e94bbfb6e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:50.528076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cdeb4d96-71ef-44f8-956f-fa4e94bbfb6e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:50.528745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cdeb4d96-71ef-44f8-956f-fa4e94bbfb6e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:50.529262 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cdeb4d96-71ef-44f8-956f-fa4e94bbfb6e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:50.688631 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e8cb1574-7254-467e-b01c-c592c3b9e368 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:50.701537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e8cb1574-7254-467e-b01c-c592c3b9e368 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1724 microversion: 2.1 time: 0.248873 Aug 28 00:58:50.702616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 249/502] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:50 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1724 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:50.711320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-86191a6b-ce2f-4895-b87b-56b477dffd79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:50.712404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-86191a6b-ce2f-4895-b87b-56b477dffd79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:50.723473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86191a6b-ce2f-4895-b87b-56b477dffd79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:50.724100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86191a6b-ce2f-4895-b87b-56b477dffd79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:50.724862 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86191a6b-ce2f-4895-b87b-56b477dffd79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:50.726427 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86191a6b-ce2f-4895-b87b-56b477dffd79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:50.727190 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86191a6b-ce2f-4895-b87b-56b477dffd79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:50.727902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86191a6b-ce2f-4895-b87b-56b477dffd79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:50.734735 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-cdeb4d96-71ef-44f8-956f-fa4e94bbfb6e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:50.750156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cdeb4d96-71ef-44f8-956f-fa4e94bbfb6e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1318 microversion: 2.1 time: 0.237725 Aug 28 00:58:50.751409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 254/503] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:50 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1318 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:58:51.060088 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-86191a6b-ce2f-4895-b87b-56b477dffd79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:51.076157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-86191a6b-ce2f-4895-b87b-56b477dffd79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:51.079044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-86191a6b-ce2f-4895-b87b-56b477dffd79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:51.081053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-86191a6b-ce2f-4895-b87b-56b477dffd79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1719 microversion: 2.20 time: 0.376676 Aug 28 00:58:51.081838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 250/504] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:50 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1719 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:51.730839 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1d10dd4c-4bfd-495b-9123-9e75b2504cb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:51.734208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1d10dd4c-4bfd-495b-9123-9e75b2504cb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:51.754846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1d10dd4c-4bfd-495b-9123-9e75b2504cb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:51.756555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1d10dd4c-4bfd-495b-9123-9e75b2504cb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:51.757577 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1d10dd4c-4bfd-495b-9123-9e75b2504cb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:51.760051 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1d10dd4c-4bfd-495b-9123-9e75b2504cb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:51.761583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1d10dd4c-4bfd-495b-9123-9e75b2504cb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:51.763421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1d10dd4c-4bfd-495b-9123-9e75b2504cb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:51.783177 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a1fba11a-e0bc-4615-8eef-feb58e07344f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:51.787565 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a1fba11a-e0bc-4615-8eef-feb58e07344f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:51.809604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a1fba11a-e0bc-4615-8eef-feb58e07344f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:51.811053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a1fba11a-e0bc-4615-8eef-feb58e07344f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:51.812088 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a1fba11a-e0bc-4615-8eef-feb58e07344f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:51.813893 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a1fba11a-e0bc-4615-8eef-feb58e07344f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:51.815297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a1fba11a-e0bc-4615-8eef-feb58e07344f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:51.816651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a1fba11a-e0bc-4615-8eef-feb58e07344f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:52.277705 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1d10dd4c-4bfd-495b-9123-9e75b2504cb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:52.291024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1d10dd4c-4bfd-495b-9123-9e75b2504cb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1701 microversion: 2.1 time: 0.570541 Aug 28 00:58:52.292534 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 255/505] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:51 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1701 bytes in 573 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:58:52.299927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-2bf02794-e35f-4845-8129-764e8d62d99d req-6e39dfe4-5f8d-4d20-9fce-ace021df73e2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dd61123b-7a96-4c9e-936c-5794a4fdd7df", "name": "network-vif-unplugged", "status": "completed", "tag": "522d4a1b-32bc-4106-afb0-5995aecc53c0"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:58:52.329117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-2bf02794-e35f-4845-8129-764e8d62d99d req-6e39dfe4-5f8d-4d20-9fce-ace021df73e2 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:52.329880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-2bf02794-e35f-4845-8129-764e8d62d99d req-6e39dfe4-5f8d-4d20-9fce-ace021df73e2 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:52.330620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-2bf02794-e35f-4845-8129-764e8d62d99d req-6e39dfe4-5f8d-4d20-9fce-ace021df73e2 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:52.353369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-2bf02794-e35f-4845-8129-764e8d62d99d req-6e39dfe4-5f8d-4d20-9fce-ace021df73e2 service nova] Creating event network-vif-unplugged:522d4a1b-32bc-4106-afb0-5995aecc53c0 for instance dd61123b-7a96-4c9e-936c-5794a4fdd7df on hv-cinder-maste Aug 28 00:58:52.362444 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-2bf02794-e35f-4845-8129-764e8d62d99d req-6e39dfe4-5f8d-4d20-9fce-ace021df73e2 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.068169 Aug 28 00:58:52.365857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 256/506] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:58:52 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 00:58:52.367116 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a1fba11a-e0bc-4615-8eef-feb58e07344f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:52.377398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b554ac7c-daf3-46ad-ad82-6a6f0a646d36 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:52.378688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b554ac7c-daf3-46ad-ad82-6a6f0a646d36 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:52.386534 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a1fba11a-e0bc-4615-8eef-feb58e07344f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1318 microversion: 2.1 time: 0.612583 Aug 28 00:58:52.387859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 251/507] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:51 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1318 bytes in 616 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:58:52.392741 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b554ac7c-daf3-46ad-ad82-6a6f0a646d36 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:52.393892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b554ac7c-daf3-46ad-ad82-6a6f0a646d36 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:52.395613 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b554ac7c-daf3-46ad-ad82-6a6f0a646d36 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:52.396510 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b554ac7c-daf3-46ad-ad82-6a6f0a646d36 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:52.397777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b554ac7c-daf3-46ad-ad82-6a6f0a646d36 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:52.398619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b554ac7c-daf3-46ad-ad82-6a6f0a646d36 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:52.407924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-804cad2b-0c30-4ad4-a00d-261d8508485e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:58:52.421268 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-804cad2b-0c30-4ad4-a00d-261d8508485e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:52.422533 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-804cad2b-0c30-4ad4-a00d-261d8508485e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:52.423825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-804cad2b-0c30-4ad4-a00d-261d8508485e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:52.552576 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-804cad2b-0c30-4ad4-a00d-261d8508485e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Going to try to start instance {{(pid=120524) start /opt/stack/nova/nova/compute/api.py:2785}} Aug 28 00:58:52.641970 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-804cad2b-0c30-4ad4-a00d-261d8508485e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "POST /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/action" status: 202 len: 0 microversion: 2.1 time: 0.246783 Aug 28 00:58:52.648392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 252/508] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Aug 28 00:58:52 2022] POST /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/action => generated 0 bytes in 255 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 00:58:52.659567 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-79cdb843-fc25-4c38-9492-711657900d90 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:52.660853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-79cdb843-fc25-4c38-9492-711657900d90 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:52.672268 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-79cdb843-fc25-4c38-9492-711657900d90 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:52.673165 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-79cdb843-fc25-4c38-9492-711657900d90 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:52.673897 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-79cdb843-fc25-4c38-9492-711657900d90 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:52.675204 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-79cdb843-fc25-4c38-9492-711657900d90 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:52.676222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-79cdb843-fc25-4c38-9492-711657900d90 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:52.676987 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-79cdb843-fc25-4c38-9492-711657900d90 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:52.747071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b554ac7c-daf3-46ad-ad82-6a6f0a646d36 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:52.757777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b554ac7c-daf3-46ad-ad82-6a6f0a646d36 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:52.759699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b554ac7c-daf3-46ad-ad82-6a6f0a646d36 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:52.761135 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b554ac7c-daf3-46ad-ad82-6a6f0a646d36 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1719 microversion: 2.20 time: 0.392888 Aug 28 00:58:52.761755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 257/509] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:52 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1719 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:52.893442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-79cdb843-fc25-4c38-9492-711657900d90 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:52.909529 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-79cdb843-fc25-4c38-9492-711657900d90 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1710 microversion: 2.1 time: 0.258202 Aug 28 00:58:52.910885 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 253/510] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:52 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1710 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:58:53.421211 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-acd17ce6-4c4c-49b6-aa7e-094580f78435 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:53.421821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-acd17ce6-4c4c-49b6-aa7e-094580f78435 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:53.429122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-acd17ce6-4c4c-49b6-aa7e-094580f78435 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:53.429760 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-acd17ce6-4c4c-49b6-aa7e-094580f78435 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:53.430360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-acd17ce6-4c4c-49b6-aa7e-094580f78435 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:53.431204 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-acd17ce6-4c4c-49b6-aa7e-094580f78435 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:53.431889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-acd17ce6-4c4c-49b6-aa7e-094580f78435 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:53.432398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-acd17ce6-4c4c-49b6-aa7e-094580f78435 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:53.580399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-acd17ce6-4c4c-49b6-aa7e-094580f78435 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:53.589014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-acd17ce6-4c4c-49b6-aa7e-094580f78435 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1318 microversion: 2.1 time: 0.171881 Aug 28 00:58:53.589805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 258/511] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:53 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1318 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:53.785570 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0f618170-b645-42f1-8c64-408a55e09aa5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:53.786084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0f618170-b645-42f1-8c64-408a55e09aa5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:53.793539 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f618170-b645-42f1-8c64-408a55e09aa5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:53.794043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f618170-b645-42f1-8c64-408a55e09aa5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:53.794457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f618170-b645-42f1-8c64-408a55e09aa5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:53.795158 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f618170-b645-42f1-8c64-408a55e09aa5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:53.795618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f618170-b645-42f1-8c64-408a55e09aa5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:53.796046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f618170-b645-42f1-8c64-408a55e09aa5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:53.944240 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c0c9b48a-3b79-49f9-948d-c0c71da27783 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:53.945456 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c0c9b48a-3b79-49f9-948d-c0c71da27783 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:53.953611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0c9b48a-3b79-49f9-948d-c0c71da27783 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:53.954127 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0c9b48a-3b79-49f9-948d-c0c71da27783 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:53.954592 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0c9b48a-3b79-49f9-948d-c0c71da27783 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:53.955339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0c9b48a-3b79-49f9-948d-c0c71da27783 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:53.955806 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0c9b48a-3b79-49f9-948d-c0c71da27783 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:53.956235 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0c9b48a-3b79-49f9-948d-c0c71da27783 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:53.965852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0f618170-b645-42f1-8c64-408a55e09aa5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:53.975576 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0f618170-b645-42f1-8c64-408a55e09aa5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:53.977415 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0f618170-b645-42f1-8c64-408a55e09aa5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:53.979044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0f618170-b645-42f1-8c64-408a55e09aa5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1719 microversion: 2.20 time: 0.197705 Aug 28 00:58:53.985039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 254/512] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:53 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1719 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:54.132311 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c0c9b48a-3b79-49f9-948d-c0c71da27783 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:54.141742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c0c9b48a-3b79-49f9-948d-c0c71da27783 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1710 microversion: 2.1 time: 0.202200 Aug 28 00:58:54.142316 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 259/513] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:53 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1710 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:58:54.442487 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-b18386fb-85e8-49d4-afd0-f34e20e0d933 req-7fe77662-ba6a-4ff9-9fbf-0371cd1967eb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2cbc0806-fe4b-4018-93bf-b4cf3c292bdc", "name": "network-vif-plugged", "status": "completed", "tag": "a816f8eb-89c2-45b9-b35d-cf9bca6f4a7c"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:58:54.460173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-b18386fb-85e8-49d4-afd0-f34e20e0d933 req-7fe77662-ba6a-4ff9-9fbf-0371cd1967eb service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:54.460811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-b18386fb-85e8-49d4-afd0-f34e20e0d933 req-7fe77662-ba6a-4ff9-9fbf-0371cd1967eb service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:54.461761 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-b18386fb-85e8-49d4-afd0-f34e20e0d933 req-7fe77662-ba6a-4ff9-9fbf-0371cd1967eb service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:54.487173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-b18386fb-85e8-49d4-afd0-f34e20e0d933 req-7fe77662-ba6a-4ff9-9fbf-0371cd1967eb service nova] Creating event network-vif-plugged:a816f8eb-89c2-45b9-b35d-cf9bca6f4a7c for instance 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc on hv-cinder-maste Aug 28 00:58:54.493007 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-b18386fb-85e8-49d4-afd0-f34e20e0d933 req-7fe77662-ba6a-4ff9-9fbf-0371cd1967eb service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056041 Aug 28 00:58:54.493786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 255/514] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:58:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:58:54.611843 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-84fe3bfa-490b-4d1f-96c5-4565b1604ae8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:54.612541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-84fe3bfa-490b-4d1f-96c5-4565b1604ae8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:54.619758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84fe3bfa-490b-4d1f-96c5-4565b1604ae8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:54.620379 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84fe3bfa-490b-4d1f-96c5-4565b1604ae8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:54.620902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84fe3bfa-490b-4d1f-96c5-4565b1604ae8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:54.621843 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84fe3bfa-490b-4d1f-96c5-4565b1604ae8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:54.622496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84fe3bfa-490b-4d1f-96c5-4565b1604ae8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:54.623113 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84fe3bfa-490b-4d1f-96c5-4565b1604ae8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:55.008769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ac734a43-756e-4a9b-8c79-1959267c5377 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:55.010194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ac734a43-756e-4a9b-8c79-1959267c5377 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:55.020629 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac734a43-756e-4a9b-8c79-1959267c5377 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:55.021750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac734a43-756e-4a9b-8c79-1959267c5377 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:55.022597 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac734a43-756e-4a9b-8c79-1959267c5377 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:55.023624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac734a43-756e-4a9b-8c79-1959267c5377 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:55.024191 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac734a43-756e-4a9b-8c79-1959267c5377 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:55.024909 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac734a43-756e-4a9b-8c79-1959267c5377 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:55.259475 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-84fe3bfa-490b-4d1f-96c5-4565b1604ae8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:55.268533 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-84fe3bfa-490b-4d1f-96c5-4565b1604ae8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1318 microversion: 2.1 time: 0.661241 Aug 28 00:58:55.269177 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 260/515] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:54 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1318 bytes in 663 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:55.274228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d29e90ea-8c0f-45b5-a396-3ecab9972ed1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:55.274837 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d29e90ea-8c0f-45b5-a396-3ecab9972ed1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:55.282469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d29e90ea-8c0f-45b5-a396-3ecab9972ed1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:55.282974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d29e90ea-8c0f-45b5-a396-3ecab9972ed1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:55.284159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d29e90ea-8c0f-45b5-a396-3ecab9972ed1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:55.284907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d29e90ea-8c0f-45b5-a396-3ecab9972ed1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:55.285421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d29e90ea-8c0f-45b5-a396-3ecab9972ed1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:55.285871 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d29e90ea-8c0f-45b5-a396-3ecab9972ed1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:55.303067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ac734a43-756e-4a9b-8c79-1959267c5377 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:55.312702 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ac734a43-756e-4a9b-8c79-1959267c5377 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:55.314279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ac734a43-756e-4a9b-8c79-1959267c5377 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:55.315524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ac734a43-756e-4a9b-8c79-1959267c5377 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1722 microversion: 2.20 time: 0.315453 Aug 28 00:58:55.316098 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 256/516] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:54 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1722 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:55.350226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b67c4f99-6dc8-473c-8a7b-b6a151913e5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "db8c499e-16a0-4d0c-b4f8-0f61f3a06a3c"}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:58:55.354575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b67c4f99-6dc8-473c-8a7b-b6a151913e5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:55.365554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b67c4f99-6dc8-473c-8a7b-b6a151913e5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:55.366968 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b67c4f99-6dc8-473c-8a7b-b6a151913e5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:55.367956 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b67c4f99-6dc8-473c-8a7b-b6a151913e5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:55.369644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b67c4f99-6dc8-473c-8a7b-b6a151913e5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:55.370980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b67c4f99-6dc8-473c-8a7b-b6a151913e5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:55.371879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b67c4f99-6dc8-473c-8a7b-b6a151913e5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:55.660013 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:58:55.700744 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d29e90ea-8c0f-45b5-a396-3ecab9972ed1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:55.715432 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d29e90ea-8c0f-45b5-a396-3ecab9972ed1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1710 microversion: 2.1 time: 0.444508 Aug 28 00:58:55.716085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 261/517] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:55 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1710 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:55.766923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b67c4f99-6dc8-473c-8a7b-b6a151913e5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "POST /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.423983 Aug 28 00:58:55.771397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 257/518] 10.222.0.42 () {62 vars in 1364 bytes} [Sun Aug 28 00:58:55 2022] POST /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 188 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 00:58:55.899002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bff552ff-b5e9-4f75-9477-e75da4b82cae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:58:55.899603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bff552ff-b5e9-4f75-9477-e75da4b82cae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:55.906260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bff552ff-b5e9-4f75-9477-e75da4b82cae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:55.906750 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bff552ff-b5e9-4f75-9477-e75da4b82cae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:55.907179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bff552ff-b5e9-4f75-9477-e75da4b82cae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:55.907900 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bff552ff-b5e9-4f75-9477-e75da4b82cae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:55.908349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bff552ff-b5e9-4f75-9477-e75da4b82cae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:55.908764 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bff552ff-b5e9-4f75-9477-e75da4b82cae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:55.957948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bff552ff-b5e9-4f75-9477-e75da4b82cae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120525) unshelve /opt/stack/nova/nova/compute/api.py:4591}} Aug 28 00:58:56.042376 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bff552ff-b5e9-4f75-9477-e75da4b82cae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "POST /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/action" status: 202 len: 0 microversion: 2.20 time: 0.146905 Aug 28 00:58:56.043233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 262/519] 10.222.0.42 () {62 vars in 1319 bytes} [Sun Aug 28 00:58:55 2022] POST /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/action => generated 0 bytes in 149 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Aug 28 00:58:56.052407 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-060e0988-368c-48b1-9424-7f1386cc639a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:56.053188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-060e0988-368c-48b1-9424-7f1386cc639a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:56.061157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-060e0988-368c-48b1-9424-7f1386cc639a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:56.061662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-060e0988-368c-48b1-9424-7f1386cc639a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:56.062287 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-060e0988-368c-48b1-9424-7f1386cc639a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:56.063211 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-060e0988-368c-48b1-9424-7f1386cc639a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:56.064041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-060e0988-368c-48b1-9424-7f1386cc639a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:56.064632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-060e0988-368c-48b1-9424-7f1386cc639a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:56.228733 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-060e0988-368c-48b1-9424-7f1386cc639a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:56.244451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-060e0988-368c-48b1-9424-7f1386cc639a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:56.246122 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-060e0988-368c-48b1-9424-7f1386cc639a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:56.247387 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-060e0988-368c-48b1-9424-7f1386cc639a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1748 microversion: 2.20 time: 0.199518 Aug 28 00:58:56.247961 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 258/520] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:56 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1748 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:56.300596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9a4a16c2-7df2-4279-a7e2-78cbc3f93daf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:56.301457 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9a4a16c2-7df2-4279-a7e2-78cbc3f93daf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:56.310915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a4a16c2-7df2-4279-a7e2-78cbc3f93daf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:56.311545 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a4a16c2-7df2-4279-a7e2-78cbc3f93daf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:56.312062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a4a16c2-7df2-4279-a7e2-78cbc3f93daf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:56.313526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a4a16c2-7df2-4279-a7e2-78cbc3f93daf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:56.314323 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a4a16c2-7df2-4279-a7e2-78cbc3f93daf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:56.315017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a4a16c2-7df2-4279-a7e2-78cbc3f93daf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:56.490940 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9a4a16c2-7df2-4279-a7e2-78cbc3f93daf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:56.503792 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9a4a16c2-7df2-4279-a7e2-78cbc3f93daf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1318 microversion: 2.1 time: 0.207093 Aug 28 00:58:56.504812 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 263/521] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:56 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1318 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:56.743417 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-87087aac-b2e4-49f4-9a49-da0046eb1e7a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:56.744762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-87087aac-b2e4-49f4-9a49-da0046eb1e7a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:56.756496 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-87087aac-b2e4-49f4-9a49-da0046eb1e7a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:56.758325 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-87087aac-b2e4-49f4-9a49-da0046eb1e7a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:56.759132 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-87087aac-b2e4-49f4-9a49-da0046eb1e7a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:56.761264 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-87087aac-b2e4-49f4-9a49-da0046eb1e7a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:56.762454 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-87087aac-b2e4-49f4-9a49-da0046eb1e7a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:56.763436 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-87087aac-b2e4-49f4-9a49-da0046eb1e7a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:57.152492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-87087aac-b2e4-49f4-9a49-da0046eb1e7a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:57.170716 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-87087aac-b2e4-49f4-9a49-da0046eb1e7a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1710 microversion: 2.1 time: 0.436324 Aug 28 00:58:57.171475 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 259/522] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:56 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1710 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 00:58:57.272740 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a9d32921-cfd5-4ab5-9daf-cb32c3ebc092 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:57.273409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a9d32921-cfd5-4ab5-9daf-cb32c3ebc092 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:57.280730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9d32921-cfd5-4ab5-9daf-cb32c3ebc092 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:57.281251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9d32921-cfd5-4ab5-9daf-cb32c3ebc092 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:57.281688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9d32921-cfd5-4ab5-9daf-cb32c3ebc092 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:57.282659 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9d32921-cfd5-4ab5-9daf-cb32c3ebc092 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:57.283179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9d32921-cfd5-4ab5-9daf-cb32c3ebc092 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:57.283603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9d32921-cfd5-4ab5-9daf-cb32c3ebc092 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:57.450300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a9d32921-cfd5-4ab5-9daf-cb32c3ebc092 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:57.453288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:58:57.462152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a9d32921-cfd5-4ab5-9daf-cb32c3ebc092 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:57.463600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a9d32921-cfd5-4ab5-9daf-cb32c3ebc092 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:57.464796 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a9d32921-cfd5-4ab5-9daf-cb32c3ebc092 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1748 microversion: 2.20 time: 0.196808 Aug 28 00:58:57.465434 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 264/523] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:57 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1748 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:57.537570 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-807bacb0-d334-46d2-9aa3-36e057d38083 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:57.538094 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-807bacb0-d334-46d2-9aa3-36e057d38083 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:57.545633 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-807bacb0-d334-46d2-9aa3-36e057d38083 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:57.546232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-807bacb0-d334-46d2-9aa3-36e057d38083 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:57.546708 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-807bacb0-d334-46d2-9aa3-36e057d38083 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:57.547502 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-807bacb0-d334-46d2-9aa3-36e057d38083 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:57.548037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-807bacb0-d334-46d2-9aa3-36e057d38083 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:57.548485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-807bacb0-d334-46d2-9aa3-36e057d38083 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:57.700928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-807bacb0-d334-46d2-9aa3-36e057d38083 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:57.713208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-807bacb0-d334-46d2-9aa3-36e057d38083 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1507 microversion: 2.1 time: 0.179397 Aug 28 00:58:57.714184 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 260/524] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:57 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1507 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:58.203112 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bfb4dc16-033c-45ad-b3e8-3a6541038a06 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:58.203986 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bfb4dc16-033c-45ad-b3e8-3a6541038a06 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:58.213655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bfb4dc16-033c-45ad-b3e8-3a6541038a06 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:58.214471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bfb4dc16-033c-45ad-b3e8-3a6541038a06 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:58.215031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bfb4dc16-033c-45ad-b3e8-3a6541038a06 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:58.215856 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bfb4dc16-033c-45ad-b3e8-3a6541038a06 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:58.216579 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bfb4dc16-033c-45ad-b3e8-3a6541038a06 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:58.217305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bfb4dc16-033c-45ad-b3e8-3a6541038a06 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:58.472090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bfb4dc16-033c-45ad-b3e8-3a6541038a06 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:58.481737 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bfb4dc16-033c-45ad-b3e8-3a6541038a06 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1710 microversion: 2.1 time: 0.283182 Aug 28 00:58:58.482332 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 265/525] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:58 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1710 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:58.492396 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9d55d377-b6cc-449b-91b8-3265a17845e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:58.493562 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9d55d377-b6cc-449b-91b8-3265a17845e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:58.506654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d55d377-b6cc-449b-91b8-3265a17845e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:58.507517 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d55d377-b6cc-449b-91b8-3265a17845e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:58.508302 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d55d377-b6cc-449b-91b8-3265a17845e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:58.509248 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d55d377-b6cc-449b-91b8-3265a17845e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:58.509862 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d55d377-b6cc-449b-91b8-3265a17845e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:58.510386 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d55d377-b6cc-449b-91b8-3265a17845e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:58.731824 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9d55d377-b6cc-449b-91b8-3265a17845e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:58.743425 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9d55d377-b6cc-449b-91b8-3265a17845e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:58.746379 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9d55d377-b6cc-449b-91b8-3265a17845e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:58.747977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9d55d377-b6cc-449b-91b8-3265a17845e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1750 microversion: 2.20 time: 0.263107 Aug 28 00:58:58.748819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 261/526] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:58 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1750 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:58:59.517381 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-429e736b-dfcd-4165-a39f-f45b5b64bd90 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:59.518320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-429e736b-dfcd-4165-a39f-f45b5b64bd90 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:59.525265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-429e736b-dfcd-4165-a39f-f45b5b64bd90 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:59.526560 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-429e736b-dfcd-4165-a39f-f45b5b64bd90 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:59.526560 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-429e736b-dfcd-4165-a39f-f45b5b64bd90 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:59.527006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-429e736b-dfcd-4165-a39f-f45b5b64bd90 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:59.527493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-429e736b-dfcd-4165-a39f-f45b5b64bd90 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:59.527942 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-429e736b-dfcd-4165-a39f-f45b5b64bd90 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:59.735870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-429e736b-dfcd-4165-a39f-f45b5b64bd90 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:59.746695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-429e736b-dfcd-4165-a39f-f45b5b64bd90 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1710 microversion: 2.1 time: 0.233443 Aug 28 00:58:59.747353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 266/527] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:58:59 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1710 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:58:59.783677 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-11593447-4ae0-4cb5-b721-12acba463422 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:58:59.784430 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-11593447-4ae0-4cb5-b721-12acba463422 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:58:59.797511 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-11593447-4ae0-4cb5-b721-12acba463422 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:59.798166 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-11593447-4ae0-4cb5-b721-12acba463422 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:59.798608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-11593447-4ae0-4cb5-b721-12acba463422 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:59.799912 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-11593447-4ae0-4cb5-b721-12acba463422 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:58:59.799912 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-11593447-4ae0-4cb5-b721-12acba463422 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:58:59.800210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-11593447-4ae0-4cb5-b721-12acba463422 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:58:59.959456 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-11593447-4ae0-4cb5-b721-12acba463422 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:59.970399 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-11593447-4ae0-4cb5-b721-12acba463422 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:59.971877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-11593447-4ae0-4cb5-b721-12acba463422 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:58:59.973290 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-11593447-4ae0-4cb5-b721-12acba463422 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1750 microversion: 2.20 time: 0.193841 Aug 28 00:58:59.980087 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 262/528] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:58:59 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1750 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:59:00.775528 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2a7abc1a-76d1-4ff3-a039-f49dec73ca1a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:00.776127 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2a7abc1a-76d1-4ff3-a039-f49dec73ca1a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:00.783931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a7abc1a-76d1-4ff3-a039-f49dec73ca1a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:00.784421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a7abc1a-76d1-4ff3-a039-f49dec73ca1a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:00.784852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a7abc1a-76d1-4ff3-a039-f49dec73ca1a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:00.785676 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a7abc1a-76d1-4ff3-a039-f49dec73ca1a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:00.786145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a7abc1a-76d1-4ff3-a039-f49dec73ca1a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:00.786570 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a7abc1a-76d1-4ff3-a039-f49dec73ca1a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:01.001173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-426a5a52-04ca-484b-87f8-7b0792f01b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:01.003443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-426a5a52-04ca-484b-87f8-7b0792f01b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:01.014727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-426a5a52-04ca-484b-87f8-7b0792f01b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:01.015458 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-426a5a52-04ca-484b-87f8-7b0792f01b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:01.016132 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-426a5a52-04ca-484b-87f8-7b0792f01b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:01.017428 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-426a5a52-04ca-484b-87f8-7b0792f01b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:01.018246 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-426a5a52-04ca-484b-87f8-7b0792f01b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:01.018987 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-426a5a52-04ca-484b-87f8-7b0792f01b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:01.056133 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2a7abc1a-76d1-4ff3-a039-f49dec73ca1a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:01.070722 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2a7abc1a-76d1-4ff3-a039-f49dec73ca1a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1714 microversion: 2.1 time: 0.298577 Aug 28 00:59:01.072343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 267/529] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 00:59:00 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1714 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 00:59:01.283497 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-426a5a52-04ca-484b-87f8-7b0792f01b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:01.298941 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-426a5a52-04ca-484b-87f8-7b0792f01b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:01.301957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-426a5a52-04ca-484b-87f8-7b0792f01b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:01.304201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-426a5a52-04ca-484b-87f8-7b0792f01b5b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.309851 Aug 28 00:59:01.305210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 263/530] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:00 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:59:02.342148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-87873b46-496e-41d2-bf66-dc985caf82a9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:02.344137 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-87873b46-496e-41d2-bf66-dc985caf82a9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:02.355928 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87873b46-496e-41d2-bf66-dc985caf82a9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:02.357187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87873b46-496e-41d2-bf66-dc985caf82a9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:02.357900 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87873b46-496e-41d2-bf66-dc985caf82a9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:02.359168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87873b46-496e-41d2-bf66-dc985caf82a9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:02.360315 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87873b46-496e-41d2-bf66-dc985caf82a9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:02.361097 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87873b46-496e-41d2-bf66-dc985caf82a9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:02.666272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-87873b46-496e-41d2-bf66-dc985caf82a9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:02.683503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-87873b46-496e-41d2-bf66-dc985caf82a9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:02.687168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-87873b46-496e-41d2-bf66-dc985caf82a9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:02.688909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-87873b46-496e-41d2-bf66-dc985caf82a9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.352522 Aug 28 00:59:02.689719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 268/531] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:02 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:03.714026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5ab063a9-4cdb-4986-87da-14ed80401e7a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:03.715210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5ab063a9-4cdb-4986-87da-14ed80401e7a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:03.722869 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ab063a9-4cdb-4986-87da-14ed80401e7a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:03.723372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ab063a9-4cdb-4986-87da-14ed80401e7a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:03.724088 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ab063a9-4cdb-4986-87da-14ed80401e7a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:03.727193 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ab063a9-4cdb-4986-87da-14ed80401e7a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:03.727770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ab063a9-4cdb-4986-87da-14ed80401e7a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:03.728137 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ab063a9-4cdb-4986-87da-14ed80401e7a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:03.898932 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5ab063a9-4cdb-4986-87da-14ed80401e7a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:03.910857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5ab063a9-4cdb-4986-87da-14ed80401e7a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:03.913474 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5ab063a9-4cdb-4986-87da-14ed80401e7a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:03.922041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5ab063a9-4cdb-4986-87da-14ed80401e7a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.212274 Aug 28 00:59:03.923820 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 264/532] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:03 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:04.968777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-97705b79-fcb4-4511-b0de-342bc26c5961 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:04.970876 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-97705b79-fcb4-4511-b0de-342bc26c5961 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:04.981668 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=89,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:59:04.986321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97705b79-fcb4-4511-b0de-342bc26c5961 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:04.987942 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97705b79-fcb4-4511-b0de-342bc26c5961 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:04.990193 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97705b79-fcb4-4511-b0de-342bc26c5961 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:04.991505 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97705b79-fcb4-4511-b0de-342bc26c5961 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:04.992352 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97705b79-fcb4-4511-b0de-342bc26c5961 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:04.993152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97705b79-fcb4-4511-b0de-342bc26c5961 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:05.186763 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-97705b79-fcb4-4511-b0de-342bc26c5961 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:05.203293 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-97705b79-fcb4-4511-b0de-342bc26c5961 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:05.205439 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-97705b79-fcb4-4511-b0de-342bc26c5961 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:05.207628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-97705b79-fcb4-4511-b0de-342bc26c5961 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.247306 Aug 28 00:59:05.208706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 269/533] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:04 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:06.254220 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f733d735-ca9e-4e69-942d-d730410f1cd4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:06.258055 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f733d735-ca9e-4e69-942d-d730410f1cd4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:06.274527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f733d735-ca9e-4e69-942d-d730410f1cd4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:06.275920 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f733d735-ca9e-4e69-942d-d730410f1cd4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:06.277251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f733d735-ca9e-4e69-942d-d730410f1cd4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:06.279029 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f733d735-ca9e-4e69-942d-d730410f1cd4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:06.280181 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f733d735-ca9e-4e69-942d-d730410f1cd4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:06.280991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f733d735-ca9e-4e69-942d-d730410f1cd4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:06.558968 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f733d735-ca9e-4e69-942d-d730410f1cd4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:06.569547 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f733d735-ca9e-4e69-942d-d730410f1cd4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:06.571620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f733d735-ca9e-4e69-942d-d730410f1cd4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:06.573075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f733d735-ca9e-4e69-942d-d730410f1cd4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.327668 Aug 28 00:59:06.573741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 265/534] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:06 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:07.600443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d80e521f-d3cc-4a69-ace8-7956a29041b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:07.601098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d80e521f-d3cc-4a69-ace8-7956a29041b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:07.608687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d80e521f-d3cc-4a69-ace8-7956a29041b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:07.609301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d80e521f-d3cc-4a69-ace8-7956a29041b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:07.609737 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d80e521f-d3cc-4a69-ace8-7956a29041b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:07.610463 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d80e521f-d3cc-4a69-ace8-7956a29041b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:07.610932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d80e521f-d3cc-4a69-ace8-7956a29041b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:07.611412 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d80e521f-d3cc-4a69-ace8-7956a29041b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:07.835072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d80e521f-d3cc-4a69-ace8-7956a29041b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:07.850982 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d80e521f-d3cc-4a69-ace8-7956a29041b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:07.852427 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d80e521f-d3cc-4a69-ace8-7956a29041b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:07.853800 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d80e521f-d3cc-4a69-ace8-7956a29041b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.257852 Aug 28 00:59:07.854600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 270/535] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:07 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:08.878862 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0d967d80-0a03-4652-980a-fbb1d01a96ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:08.879513 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0d967d80-0a03-4652-980a-fbb1d01a96ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:08.887346 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0d967d80-0a03-4652-980a-fbb1d01a96ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:08.888200 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0d967d80-0a03-4652-980a-fbb1d01a96ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:08.888794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0d967d80-0a03-4652-980a-fbb1d01a96ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:08.889705 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0d967d80-0a03-4652-980a-fbb1d01a96ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:08.890170 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0d967d80-0a03-4652-980a-fbb1d01a96ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:08.891090 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0d967d80-0a03-4652-980a-fbb1d01a96ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:09.124243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0d967d80-0a03-4652-980a-fbb1d01a96ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:09.140301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0d967d80-0a03-4652-980a-fbb1d01a96ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:09.143262 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0d967d80-0a03-4652-980a-fbb1d01a96ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:09.145474 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0d967d80-0a03-4652-980a-fbb1d01a96ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.270631 Aug 28 00:59:09.146499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 266/536] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:08 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:59:10.182504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c0b4e5af-f533-4c73-8000-628195bf97e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:10.183341 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c0b4e5af-f533-4c73-8000-628195bf97e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:10.190045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0b4e5af-f533-4c73-8000-628195bf97e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:10.190535 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0b4e5af-f533-4c73-8000-628195bf97e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:10.190969 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0b4e5af-f533-4c73-8000-628195bf97e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:10.191695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0b4e5af-f533-4c73-8000-628195bf97e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:10.192155 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0b4e5af-f533-4c73-8000-628195bf97e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:10.192578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0b4e5af-f533-4c73-8000-628195bf97e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:10.367898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c0b4e5af-f533-4c73-8000-628195bf97e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:10.384096 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c0b4e5af-f533-4c73-8000-628195bf97e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:10.387282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c0b4e5af-f533-4c73-8000-628195bf97e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:10.389768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c0b4e5af-f533-4c73-8000-628195bf97e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.211299 Aug 28 00:59:10.425375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 271/537] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:10 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:11.423953 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3b6a4cc5-7b31-46dc-9043-48db71c6c7a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:11.424689 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3b6a4cc5-7b31-46dc-9043-48db71c6c7a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:11.432504 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3b6a4cc5-7b31-46dc-9043-48db71c6c7a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:11.433235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3b6a4cc5-7b31-46dc-9043-48db71c6c7a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:11.433676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3b6a4cc5-7b31-46dc-9043-48db71c6c7a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:11.434673 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3b6a4cc5-7b31-46dc-9043-48db71c6c7a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:11.435182 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3b6a4cc5-7b31-46dc-9043-48db71c6c7a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:11.435622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3b6a4cc5-7b31-46dc-9043-48db71c6c7a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:11.632680 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3b6a4cc5-7b31-46dc-9043-48db71c6c7a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:11.642910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3b6a4cc5-7b31-46dc-9043-48db71c6c7a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:11.644375 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3b6a4cc5-7b31-46dc-9043-48db71c6c7a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:11.645643 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3b6a4cc5-7b31-46dc-9043-48db71c6c7a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.225860 Aug 28 00:59:11.647106 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 267/538] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:11 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:12.674537 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f3083583-aec4-4537-b3f1-aba295f000a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:12.675299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f3083583-aec4-4537-b3f1-aba295f000a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:12.683783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3083583-aec4-4537-b3f1-aba295f000a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:12.684511 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3083583-aec4-4537-b3f1-aba295f000a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:12.684997 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3083583-aec4-4537-b3f1-aba295f000a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:12.685809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3083583-aec4-4537-b3f1-aba295f000a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:12.686216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3083583-aec4-4537-b3f1-aba295f000a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:12.686644 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3083583-aec4-4537-b3f1-aba295f000a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:12.887169 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f3083583-aec4-4537-b3f1-aba295f000a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:12.896667 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f3083583-aec4-4537-b3f1-aba295f000a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:12.898263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f3083583-aec4-4537-b3f1-aba295f000a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:12.899731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f3083583-aec4-4537-b3f1-aba295f000a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.232604 Aug 28 00:59:12.900424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 272/539] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:12 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:13.935129 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6bb4f4a7-ef04-47d0-b307-23258851bafa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:13.937528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6bb4f4a7-ef04-47d0-b307-23258851bafa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:13.953489 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6bb4f4a7-ef04-47d0-b307-23258851bafa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:13.955154 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6bb4f4a7-ef04-47d0-b307-23258851bafa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:13.955840 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6bb4f4a7-ef04-47d0-b307-23258851bafa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:13.956985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6bb4f4a7-ef04-47d0-b307-23258851bafa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:13.957925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6bb4f4a7-ef04-47d0-b307-23258851bafa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:13.958975 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6bb4f4a7-ef04-47d0-b307-23258851bafa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:14.128482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6bb4f4a7-ef04-47d0-b307-23258851bafa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:14.143006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6bb4f4a7-ef04-47d0-b307-23258851bafa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:14.144576 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6bb4f4a7-ef04-47d0-b307-23258851bafa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:14.145881 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6bb4f4a7-ef04-47d0-b307-23258851bafa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.218946 Aug 28 00:59:14.146823 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 268/540] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:13 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:15.187341 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c251f647-13a0-4798-97fe-8ead555ef2cf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:15.188499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c251f647-13a0-4798-97fe-8ead555ef2cf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:15.200759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c251f647-13a0-4798-97fe-8ead555ef2cf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:15.201648 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c251f647-13a0-4798-97fe-8ead555ef2cf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:15.202425 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c251f647-13a0-4798-97fe-8ead555ef2cf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:15.203708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c251f647-13a0-4798-97fe-8ead555ef2cf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:15.204556 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c251f647-13a0-4798-97fe-8ead555ef2cf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:15.205383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c251f647-13a0-4798-97fe-8ead555ef2cf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:15.390389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c251f647-13a0-4798-97fe-8ead555ef2cf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:15.400176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c251f647-13a0-4798-97fe-8ead555ef2cf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:15.401732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c251f647-13a0-4798-97fe-8ead555ef2cf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:15.402976 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c251f647-13a0-4798-97fe-8ead555ef2cf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.222442 Aug 28 00:59:15.403574 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 273/541] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:15 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:16.427099 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e3b0c412-c8e5-4ee1-ad7a-8e96159384f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:16.427793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e3b0c412-c8e5-4ee1-ad7a-8e96159384f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:16.430583 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=56 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:59:16.436661 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3b0c412-c8e5-4ee1-ad7a-8e96159384f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:16.437184 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3b0c412-c8e5-4ee1-ad7a-8e96159384f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:16.437614 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3b0c412-c8e5-4ee1-ad7a-8e96159384f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:16.438322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3b0c412-c8e5-4ee1-ad7a-8e96159384f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:16.438766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3b0c412-c8e5-4ee1-ad7a-8e96159384f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:16.439181 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3b0c412-c8e5-4ee1-ad7a-8e96159384f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:16.610194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e3b0c412-c8e5-4ee1-ad7a-8e96159384f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:16.620478 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e3b0c412-c8e5-4ee1-ad7a-8e96159384f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:16.622095 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e3b0c412-c8e5-4ee1-ad7a-8e96159384f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:16.623601 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e3b0c412-c8e5-4ee1-ad7a-8e96159384f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.200461 Aug 28 00:59:16.624204 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 269/542] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:16 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:17.650284 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7e7acdec-0d6d-494f-9a0f-9b5fde0a3d8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:17.651254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7e7acdec-0d6d-494f-9a0f-9b5fde0a3d8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:17.658272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e7acdec-0d6d-494f-9a0f-9b5fde0a3d8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:17.658866 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e7acdec-0d6d-494f-9a0f-9b5fde0a3d8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:17.659406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e7acdec-0d6d-494f-9a0f-9b5fde0a3d8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:17.660245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e7acdec-0d6d-494f-9a0f-9b5fde0a3d8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:17.660698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e7acdec-0d6d-494f-9a0f-9b5fde0a3d8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:17.661157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e7acdec-0d6d-494f-9a0f-9b5fde0a3d8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:17.831553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7e7acdec-0d6d-494f-9a0f-9b5fde0a3d8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:17.842274 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7e7acdec-0d6d-494f-9a0f-9b5fde0a3d8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:17.843748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7e7acdec-0d6d-494f-9a0f-9b5fde0a3d8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:17.844943 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7e7acdec-0d6d-494f-9a0f-9b5fde0a3d8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.200098 Aug 28 00:59:17.845559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 274/543] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:17 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:18.869279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d482430a-e278-4ed3-af26-6ea57257aa31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:18.869926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d482430a-e278-4ed3-af26-6ea57257aa31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:18.877518 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d482430a-e278-4ed3-af26-6ea57257aa31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:18.878011 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d482430a-e278-4ed3-af26-6ea57257aa31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:18.878450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d482430a-e278-4ed3-af26-6ea57257aa31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:18.879180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d482430a-e278-4ed3-af26-6ea57257aa31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:18.879631 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d482430a-e278-4ed3-af26-6ea57257aa31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:18.880050 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d482430a-e278-4ed3-af26-6ea57257aa31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:19.029620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d482430a-e278-4ed3-af26-6ea57257aa31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:19.039092 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d482430a-e278-4ed3-af26-6ea57257aa31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:19.040572 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d482430a-e278-4ed3-af26-6ea57257aa31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:19.041919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d482430a-e278-4ed3-af26-6ea57257aa31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.177479 Aug 28 00:59:19.042495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 270/544] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:18 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:20.065638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9b4c8da9-36cb-49c9-996e-1684fee342b0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:20.066233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9b4c8da9-36cb-49c9-996e-1684fee342b0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:20.073774 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b4c8da9-36cb-49c9-996e-1684fee342b0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:20.074269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b4c8da9-36cb-49c9-996e-1684fee342b0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:20.074704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b4c8da9-36cb-49c9-996e-1684fee342b0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:20.075424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b4c8da9-36cb-49c9-996e-1684fee342b0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:20.075879 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b4c8da9-36cb-49c9-996e-1684fee342b0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:20.076288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b4c8da9-36cb-49c9-996e-1684fee342b0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:20.235677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9b4c8da9-36cb-49c9-996e-1684fee342b0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:20.252742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9b4c8da9-36cb-49c9-996e-1684fee342b0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:20.255671 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9b4c8da9-36cb-49c9-996e-1684fee342b0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:20.257798 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9b4c8da9-36cb-49c9-996e-1684fee342b0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.196005 Aug 28 00:59:20.258841 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 275/545] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:20 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:21.296561 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fb375065-1d55-432c-9831-d3cde7534cde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:21.297303 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fb375065-1d55-432c-9831-d3cde7534cde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:21.305503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fb375065-1d55-432c-9831-d3cde7534cde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:21.306062 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fb375065-1d55-432c-9831-d3cde7534cde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:21.306528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fb375065-1d55-432c-9831-d3cde7534cde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:21.307410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fb375065-1d55-432c-9831-d3cde7534cde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:21.307898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fb375065-1d55-432c-9831-d3cde7534cde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:21.308367 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fb375065-1d55-432c-9831-d3cde7534cde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:21.457306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fb375065-1d55-432c-9831-d3cde7534cde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:21.466844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fb375065-1d55-432c-9831-d3cde7534cde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:21.468309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fb375065-1d55-432c-9831-d3cde7534cde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:21.469700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fb375065-1d55-432c-9831-d3cde7534cde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.177277 Aug 28 00:59:21.470434 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 271/546] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:21 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:59:22.504852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-56b8c507-0505-4bce-b3d9-d4bd9559e7ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:22.505576 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-56b8c507-0505-4bce-b3d9-d4bd9559e7ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:22.512778 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56b8c507-0505-4bce-b3d9-d4bd9559e7ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:22.513444 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56b8c507-0505-4bce-b3d9-d4bd9559e7ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:22.513977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56b8c507-0505-4bce-b3d9-d4bd9559e7ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:22.514790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56b8c507-0505-4bce-b3d9-d4bd9559e7ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:22.515343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56b8c507-0505-4bce-b3d9-d4bd9559e7ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:22.515855 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56b8c507-0505-4bce-b3d9-d4bd9559e7ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:22.706386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-56b8c507-0505-4bce-b3d9-d4bd9559e7ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:22.716222 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-56b8c507-0505-4bce-b3d9-d4bd9559e7ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:22.717736 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-56b8c507-0505-4bce-b3d9-d4bd9559e7ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:22.718958 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-56b8c507-0505-4bce-b3d9-d4bd9559e7ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.218482 Aug 28 00:59:22.719535 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 276/547] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:22 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:23.743758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1da24340-2d2f-4e91-ac90-43039b51048f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:23.744883 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1da24340-2d2f-4e91-ac90-43039b51048f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:23.751965 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1da24340-2d2f-4e91-ac90-43039b51048f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:23.752456 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1da24340-2d2f-4e91-ac90-43039b51048f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:23.752924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1da24340-2d2f-4e91-ac90-43039b51048f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:23.753681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1da24340-2d2f-4e91-ac90-43039b51048f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:23.754138 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1da24340-2d2f-4e91-ac90-43039b51048f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:23.754569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1da24340-2d2f-4e91-ac90-43039b51048f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:23.910412 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1da24340-2d2f-4e91-ac90-43039b51048f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:23.926117 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1da24340-2d2f-4e91-ac90-43039b51048f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:23.928838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1da24340-2d2f-4e91-ac90-43039b51048f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:23.931042 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1da24340-2d2f-4e91-ac90-43039b51048f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.191511 Aug 28 00:59:23.932179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 272/548] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:23 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:24.968604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-df5a4f9d-a8d9-4b5f-9209-529218fc1235 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:24.969365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-df5a4f9d-a8d9-4b5f-9209-529218fc1235 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:24.976505 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df5a4f9d-a8d9-4b5f-9209-529218fc1235 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:24.977060 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df5a4f9d-a8d9-4b5f-9209-529218fc1235 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:24.977523 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df5a4f9d-a8d9-4b5f-9209-529218fc1235 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:24.978219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df5a4f9d-a8d9-4b5f-9209-529218fc1235 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:24.978784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df5a4f9d-a8d9-4b5f-9209-529218fc1235 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:24.979214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df5a4f9d-a8d9-4b5f-9209-529218fc1235 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:25.150008 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-df5a4f9d-a8d9-4b5f-9209-529218fc1235 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:25.160100 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-df5a4f9d-a8d9-4b5f-9209-529218fc1235 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:25.161614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-df5a4f9d-a8d9-4b5f-9209-529218fc1235 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:25.162817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-df5a4f9d-a8d9-4b5f-9209-529218fc1235 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.198532 Aug 28 00:59:25.163411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 277/549] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:24 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:26.188544 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2bd67aa4-545b-413d-97a9-62af026f2f8d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:26.189488 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2bd67aa4-545b-413d-97a9-62af026f2f8d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:26.197431 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2bd67aa4-545b-413d-97a9-62af026f2f8d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:26.197997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2bd67aa4-545b-413d-97a9-62af026f2f8d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:26.198491 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2bd67aa4-545b-413d-97a9-62af026f2f8d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:26.199279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2bd67aa4-545b-413d-97a9-62af026f2f8d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:26.199784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2bd67aa4-545b-413d-97a9-62af026f2f8d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:26.200266 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2bd67aa4-545b-413d-97a9-62af026f2f8d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:26.203415 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:UPDATE=3,nova_cell1:INSERT=1,nova_cell1:SELECT=94 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:59:26.370787 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2bd67aa4-545b-413d-97a9-62af026f2f8d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:26.386299 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2bd67aa4-545b-413d-97a9-62af026f2f8d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:26.389344 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2bd67aa4-545b-413d-97a9-62af026f2f8d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:26.391557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2bd67aa4-545b-413d-97a9-62af026f2f8d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.208220 Aug 28 00:59:26.392558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 273/550] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:26 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:27.429414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d3435e46-f59c-4abe-ac46-322c7fb09671 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:27.430201 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d3435e46-f59c-4abe-ac46-322c7fb09671 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:27.438087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d3435e46-f59c-4abe-ac46-322c7fb09671 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:27.438643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d3435e46-f59c-4abe-ac46-322c7fb09671 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:27.439122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d3435e46-f59c-4abe-ac46-322c7fb09671 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:27.439932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d3435e46-f59c-4abe-ac46-322c7fb09671 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:27.440444 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d3435e46-f59c-4abe-ac46-322c7fb09671 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:27.441579 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d3435e46-f59c-4abe-ac46-322c7fb09671 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:27.606256 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d3435e46-f59c-4abe-ac46-322c7fb09671 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:27.616606 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d3435e46-f59c-4abe-ac46-322c7fb09671 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:27.618266 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d3435e46-f59c-4abe-ac46-322c7fb09671 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:27.619600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d3435e46-f59c-4abe-ac46-322c7fb09671 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.194424 Aug 28 00:59:27.620208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 278/551] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:27 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:28.646269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-524e27d9-affa-4fe5-80dd-eebf863489df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:28.647304 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-524e27d9-affa-4fe5-80dd-eebf863489df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:28.655637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-524e27d9-affa-4fe5-80dd-eebf863489df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:28.656128 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-524e27d9-affa-4fe5-80dd-eebf863489df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:28.656565 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-524e27d9-affa-4fe5-80dd-eebf863489df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:28.657341 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-524e27d9-affa-4fe5-80dd-eebf863489df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:28.657799 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-524e27d9-affa-4fe5-80dd-eebf863489df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:28.658227 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-524e27d9-affa-4fe5-80dd-eebf863489df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:28.826264 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-524e27d9-affa-4fe5-80dd-eebf863489df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:28.836302 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-524e27d9-affa-4fe5-80dd-eebf863489df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:28.837895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-524e27d9-affa-4fe5-80dd-eebf863489df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:28.839140 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-524e27d9-affa-4fe5-80dd-eebf863489df tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.197480 Aug 28 00:59:28.839720 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 274/552] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:28 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:29.863597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-143440fe-5d83-42d5-8264-bed8f2e61e48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:29.864176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-143440fe-5d83-42d5-8264-bed8f2e61e48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:29.871551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-143440fe-5d83-42d5-8264-bed8f2e61e48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:29.872044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-143440fe-5d83-42d5-8264-bed8f2e61e48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:29.872505 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-143440fe-5d83-42d5-8264-bed8f2e61e48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:29.873255 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-143440fe-5d83-42d5-8264-bed8f2e61e48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:29.873704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-143440fe-5d83-42d5-8264-bed8f2e61e48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:29.874121 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-143440fe-5d83-42d5-8264-bed8f2e61e48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:29.877189 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=86 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:59:30.029368 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-143440fe-5d83-42d5-8264-bed8f2e61e48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:30.039189 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-143440fe-5d83-42d5-8264-bed8f2e61e48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:30.040620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-143440fe-5d83-42d5-8264-bed8f2e61e48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:30.041860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-143440fe-5d83-42d5-8264-bed8f2e61e48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.182495 Aug 28 00:59:30.042423 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 279/553] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:29 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:30.477423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3c57d40d-ae9a-4fa2-997c-20bfde3691c5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "7421938a-2d8f-4ae7-aac6-d08ec2a49354", "device": "/dev/sdb"}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:59:30.480647 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3c57d40d-ae9a-4fa2-997c-20bfde3691c5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:30.487927 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c57d40d-ae9a-4fa2-997c-20bfde3691c5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:30.488379 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c57d40d-ae9a-4fa2-997c-20bfde3691c5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:30.488809 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c57d40d-ae9a-4fa2-997c-20bfde3691c5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:30.489555 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c57d40d-ae9a-4fa2-997c-20bfde3691c5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:30.490002 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c57d40d-ae9a-4fa2-997c-20bfde3691c5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:30.490420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c57d40d-ae9a-4fa2-997c-20bfde3691c5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:30.835430 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3c57d40d-ae9a-4fa2-997c-20bfde3691c5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "POST /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.362271 Aug 28 00:59:30.836168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 275/554] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Aug 28 00:59:30 2022] POST /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc/os-volume_attachments => generated 194 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:59:31.083426 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f2d65493-03a3-4120-bfb9-60e541de1b01 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:31.084502 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f2d65493-03a3-4120-bfb9-60e541de1b01 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:31.094578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f2d65493-03a3-4120-bfb9-60e541de1b01 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:31.095572 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f2d65493-03a3-4120-bfb9-60e541de1b01 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:31.096038 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f2d65493-03a3-4120-bfb9-60e541de1b01 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:31.096942 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f2d65493-03a3-4120-bfb9-60e541de1b01 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:31.097466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f2d65493-03a3-4120-bfb9-60e541de1b01 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:31.097956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f2d65493-03a3-4120-bfb9-60e541de1b01 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:31.251924 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f2d65493-03a3-4120-bfb9-60e541de1b01 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:31.262530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f2d65493-03a3-4120-bfb9-60e541de1b01 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:31.264043 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f2d65493-03a3-4120-bfb9-60e541de1b01 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:31.265257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f2d65493-03a3-4120-bfb9-60e541de1b01 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.186968 Aug 28 00:59:31.265832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 280/555] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:31 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:31.324119 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f047f632-f1bd-46e6-aa00-095e489e66cc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c"}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:59:31.326710 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f047f632-f1bd-46e6-aa00-095e489e66cc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:31.334003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f047f632-f1bd-46e6-aa00-095e489e66cc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:31.334495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f047f632-f1bd-46e6-aa00-095e489e66cc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:31.334929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f047f632-f1bd-46e6-aa00-095e489e66cc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:31.335637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f047f632-f1bd-46e6-aa00-095e489e66cc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:31.336109 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f047f632-f1bd-46e6-aa00-095e489e66cc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:31.336535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f047f632-f1bd-46e6-aa00-095e489e66cc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:31.704087 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f047f632-f1bd-46e6-aa00-095e489e66cc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "POST /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.384826 Aug 28 00:59:31.705536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 276/556] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Aug 28 00:59:31 2022] POST /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 194 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:59:32.288158 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bd2507fa-b4f2-4c3d-abb8-b85d70aef0af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:32.289005 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bd2507fa-b4f2-4c3d-abb8-b85d70aef0af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:32.296950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bd2507fa-b4f2-4c3d-abb8-b85d70aef0af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:32.297537 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bd2507fa-b4f2-4c3d-abb8-b85d70aef0af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:32.298104 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bd2507fa-b4f2-4c3d-abb8-b85d70aef0af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:32.298977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bd2507fa-b4f2-4c3d-abb8-b85d70aef0af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:32.299489 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bd2507fa-b4f2-4c3d-abb8-b85d70aef0af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:32.300018 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bd2507fa-b4f2-4c3d-abb8-b85d70aef0af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:32.451995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bd2507fa-b4f2-4c3d-abb8-b85d70aef0af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:32.461584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bd2507fa-b4f2-4c3d-abb8-b85d70aef0af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:32.463029 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bd2507fa-b4f2-4c3d-abb8-b85d70aef0af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:32.464199 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bd2507fa-b4f2-4c3d-abb8-b85d70aef0af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.180120 Aug 28 00:59:32.464775 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 281/557] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:32 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:33.504358 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f13ad580-94d4-4bb6-9980-5e450b83d9e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:33.505663 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f13ad580-94d4-4bb6-9980-5e450b83d9e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:33.516507 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f13ad580-94d4-4bb6-9980-5e450b83d9e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:33.517372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f13ad580-94d4-4bb6-9980-5e450b83d9e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:33.518118 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f13ad580-94d4-4bb6-9980-5e450b83d9e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:33.519295 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f13ad580-94d4-4bb6-9980-5e450b83d9e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:33.520081 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f13ad580-94d4-4bb6-9980-5e450b83d9e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:33.520786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f13ad580-94d4-4bb6-9980-5e450b83d9e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:33.697839 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f13ad580-94d4-4bb6-9980-5e450b83d9e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:33.707346 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f13ad580-94d4-4bb6-9980-5e450b83d9e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:33.708834 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f13ad580-94d4-4bb6-9980-5e450b83d9e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:33.710077 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f13ad580-94d4-4bb6-9980-5e450b83d9e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.214745 Aug 28 00:59:33.710660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 277/558] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:33 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:34.738538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-416513f8-d969-42a9-bf1b-5bcdc3b1b8ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:34.739258 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-416513f8-d969-42a9-bf1b-5bcdc3b1b8ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:34.747060 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-416513f8-d969-42a9-bf1b-5bcdc3b1b8ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:34.747551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-416513f8-d969-42a9-bf1b-5bcdc3b1b8ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:34.748017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-416513f8-d969-42a9-bf1b-5bcdc3b1b8ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:34.748729 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-416513f8-d969-42a9-bf1b-5bcdc3b1b8ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:34.749237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-416513f8-d969-42a9-bf1b-5bcdc3b1b8ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:34.749754 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-416513f8-d969-42a9-bf1b-5bcdc3b1b8ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:34.918092 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-416513f8-d969-42a9-bf1b-5bcdc3b1b8ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:34.934763 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-416513f8-d969-42a9-bf1b-5bcdc3b1b8ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:34.936229 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-416513f8-d969-42a9-bf1b-5bcdc3b1b8ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:34.937482 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-416513f8-d969-42a9-bf1b-5bcdc3b1b8ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.203498 Aug 28 00:59:34.938315 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 282/559] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:34 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:35.972734 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e9b183e8-fabb-4726-bb88-6632b36a84e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:35.973412 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e9b183e8-fabb-4726-bb88-6632b36a84e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:35.980776 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e9b183e8-fabb-4726-bb88-6632b36a84e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:35.981288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e9b183e8-fabb-4726-bb88-6632b36a84e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:35.981718 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e9b183e8-fabb-4726-bb88-6632b36a84e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:35.982499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e9b183e8-fabb-4726-bb88-6632b36a84e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:35.983364 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e9b183e8-fabb-4726-bb88-6632b36a84e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:35.984280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e9b183e8-fabb-4726-bb88-6632b36a84e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:36.162811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e9b183e8-fabb-4726-bb88-6632b36a84e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:36.172404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e9b183e8-fabb-4726-bb88-6632b36a84e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:36.173969 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e9b183e8-fabb-4726-bb88-6632b36a84e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:36.175204 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e9b183e8-fabb-4726-bb88-6632b36a84e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.208420 Aug 28 00:59:36.175789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 278/560] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:35 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:37.208478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b60fbbec-f2ea-4cf2-b83c-ad19ea529e8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:37.209858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b60fbbec-f2ea-4cf2-b83c-ad19ea529e8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:37.212722 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=28 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 00:59:37.221655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b60fbbec-f2ea-4cf2-b83c-ad19ea529e8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:37.222356 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b60fbbec-f2ea-4cf2-b83c-ad19ea529e8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:37.223024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b60fbbec-f2ea-4cf2-b83c-ad19ea529e8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:37.223721 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-8db48267-b520-4b1f-a07b-944a25754881 req-6fbac071-b4dd-40a6-be2c-0da79447b2e6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dd61123b-7a96-4c9e-936c-5794a4fdd7df", "name": "network-vif-plugged", "status": "completed", "tag": "522d4a1b-32bc-4106-afb0-5995aecc53c0"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 00:59:37.224539 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b60fbbec-f2ea-4cf2-b83c-ad19ea529e8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:37.224800 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b60fbbec-f2ea-4cf2-b83c-ad19ea529e8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:37.225644 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b60fbbec-f2ea-4cf2-b83c-ad19ea529e8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:37.244952 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-8db48267-b520-4b1f-a07b-944a25754881 req-6fbac071-b4dd-40a6-be2c-0da79447b2e6 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:37.245934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-8db48267-b520-4b1f-a07b-944a25754881 req-6fbac071-b4dd-40a6-be2c-0da79447b2e6 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:37.246850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-8db48267-b520-4b1f-a07b-944a25754881 req-6fbac071-b4dd-40a6-be2c-0da79447b2e6 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:37.269385 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-8db48267-b520-4b1f-a07b-944a25754881 req-6fbac071-b4dd-40a6-be2c-0da79447b2e6 service nova] Creating event network-vif-plugged:522d4a1b-32bc-4106-afb0-5995aecc53c0 for instance dd61123b-7a96-4c9e-936c-5794a4fdd7df on hv-cinder-maste Aug 28 00:59:37.280031 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-8db48267-b520-4b1f-a07b-944a25754881 req-6fbac071-b4dd-40a6-be2c-0da79447b2e6 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063263 Aug 28 00:59:37.281056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 279/561] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 00:59:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 00:59:37.771027 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b60fbbec-f2ea-4cf2-b83c-ad19ea529e8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:37.782096 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b60fbbec-f2ea-4cf2-b83c-ad19ea529e8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:37.783852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b60fbbec-f2ea-4cf2-b83c-ad19ea529e8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:37.785297 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b60fbbec-f2ea-4cf2-b83c-ad19ea529e8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.581972 Aug 28 00:59:37.786004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 283/562] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:37 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 592 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 00:59:38.817175 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-abd37c48-0463-4832-a34d-7ac94d5b6bd0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:38.818499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-abd37c48-0463-4832-a34d-7ac94d5b6bd0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:38.834833 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-abd37c48-0463-4832-a34d-7ac94d5b6bd0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:38.835719 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-abd37c48-0463-4832-a34d-7ac94d5b6bd0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:38.836462 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-abd37c48-0463-4832-a34d-7ac94d5b6bd0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:38.837812 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-abd37c48-0463-4832-a34d-7ac94d5b6bd0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:38.838598 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-abd37c48-0463-4832-a34d-7ac94d5b6bd0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:38.839460 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-abd37c48-0463-4832-a34d-7ac94d5b6bd0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:39.045167 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-abd37c48-0463-4832-a34d-7ac94d5b6bd0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:39.055077 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-abd37c48-0463-4832-a34d-7ac94d5b6bd0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:39.056637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-abd37c48-0463-4832-a34d-7ac94d5b6bd0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:39.057956 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-abd37c48-0463-4832-a34d-7ac94d5b6bd0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.250202 Aug 28 00:59:39.058546 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 280/563] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:38 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:40.097166 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-219a6628-9cb7-4a2b-8b40-91c7b3c12001 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:40.097840 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-219a6628-9cb7-4a2b-8b40-91c7b3c12001 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:40.104767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-219a6628-9cb7-4a2b-8b40-91c7b3c12001 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:40.105301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-219a6628-9cb7-4a2b-8b40-91c7b3c12001 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:40.105735 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-219a6628-9cb7-4a2b-8b40-91c7b3c12001 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:40.106463 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-219a6628-9cb7-4a2b-8b40-91c7b3c12001 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:40.106917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-219a6628-9cb7-4a2b-8b40-91c7b3c12001 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:40.107342 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-219a6628-9cb7-4a2b-8b40-91c7b3c12001 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:40.271453 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-219a6628-9cb7-4a2b-8b40-91c7b3c12001 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:40.281034 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-219a6628-9cb7-4a2b-8b40-91c7b3c12001 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:40.282528 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-219a6628-9cb7-4a2b-8b40-91c7b3c12001 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:40.283718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-219a6628-9cb7-4a2b-8b40-91c7b3c12001 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.190591 Aug 28 00:59:40.284295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 284/564] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:40 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:41.312564 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fa2fd713-e439-45ef-99f6-3e2b9cb13cfe tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:41.313328 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fa2fd713-e439-45ef-99f6-3e2b9cb13cfe tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:41.322407 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fa2fd713-e439-45ef-99f6-3e2b9cb13cfe tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:41.323064 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fa2fd713-e439-45ef-99f6-3e2b9cb13cfe tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:41.323566 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fa2fd713-e439-45ef-99f6-3e2b9cb13cfe tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:41.324351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fa2fd713-e439-45ef-99f6-3e2b9cb13cfe tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:41.324849 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fa2fd713-e439-45ef-99f6-3e2b9cb13cfe tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:41.325471 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fa2fd713-e439-45ef-99f6-3e2b9cb13cfe tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:41.522307 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fa2fd713-e439-45ef-99f6-3e2b9cb13cfe tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:41.536123 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fa2fd713-e439-45ef-99f6-3e2b9cb13cfe tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:41.538034 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fa2fd713-e439-45ef-99f6-3e2b9cb13cfe tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:41.539618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fa2fd713-e439-45ef-99f6-3e2b9cb13cfe tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1806 microversion: 2.20 time: 0.231910 Aug 28 00:59:41.540400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 281/565] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:41 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1806 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 00:59:42.580474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-172d3308-8115-4c9f-aab2-67ac52036032 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 00:59:42.581441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-172d3308-8115-4c9f-aab2-67ac52036032 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 00:59:42.589497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-172d3308-8115-4c9f-aab2-67ac52036032 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:42.590094 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-172d3308-8115-4c9f-aab2-67ac52036032 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:42.590424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-172d3308-8115-4c9f-aab2-67ac52036032 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:42.591230 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-172d3308-8115-4c9f-aab2-67ac52036032 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 00:59:42.591616 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-172d3308-8115-4c9f-aab2-67ac52036032 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 00:59:42.592215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-172d3308-8115-4c9f-aab2-67ac52036032 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 00:59:42.818264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-172d3308-8115-4c9f-aab2-67ac52036032 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:42.828047 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-172d3308-8115-4c9f-aab2-67ac52036032 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:42.829643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-172d3308-8115-4c9f-aab2-67ac52036032 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 00:59:42.830828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-172d3308-8115-4c9f-aab2-67ac52036032 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1793 microversion: 2.20 time: 0.255002 Aug 28 00:59:42.831393 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 285/566] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 00:59:42 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1793 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:00:20.753657 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=25 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:00:20.755199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=59,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:00:20.872838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-39fe69c3-e8bd-403f-9171-f1b87e87eb24 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:20.873375 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-39fe69c3-e8bd-403f-9171-f1b87e87eb24 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:20.879738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-39fe69c3-e8bd-403f-9171-f1b87e87eb24 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:20.880227 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-39fe69c3-e8bd-403f-9171-f1b87e87eb24 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:20.880665 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-39fe69c3-e8bd-403f-9171-f1b87e87eb24 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:20.881414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-39fe69c3-e8bd-403f-9171-f1b87e87eb24 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:20.881867 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-39fe69c3-e8bd-403f-9171-f1b87e87eb24 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:20.882296 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-39fe69c3-e8bd-403f-9171-f1b87e87eb24 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:21.113599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.objects.instance [None req-39fe69c3-e8bd-403f-9171-f1b87e87eb24 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lazy-loading 'services' on Instance uuid e434e4f9-9d0e-46f2-bea3-395bae33fc92 {{(pid=120524) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:00:21.247542 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-39fe69c3-e8bd-403f-9171-f1b87e87eb24 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "DELETE /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments/2d7e8fa1-c167-4381-a38c-015d9ac4cf10" status: 202 len: 0 microversion: 2.1 time: 0.499916 Aug 28 01:00:21.248161 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 282/567] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Aug 28 01:00:20 2022] DELETE /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments/2d7e8fa1-c167-4381-a38c-015d9ac4cf10 => generated 0 bytes in 502 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 01:00:33.954837 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=5 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:00:33.955592 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=41 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:00:34.088698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c2683956-6e80-4012-834d-0ca41fe95031 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:34.089451 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c2683956-6e80-4012-834d-0ca41fe95031 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:34.098072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2683956-6e80-4012-834d-0ca41fe95031 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:34.098746 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2683956-6e80-4012-834d-0ca41fe95031 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:34.099244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2683956-6e80-4012-834d-0ca41fe95031 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:34.100113 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2683956-6e80-4012-834d-0ca41fe95031 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:34.100864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2683956-6e80-4012-834d-0ca41fe95031 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:34.101428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2683956-6e80-4012-834d-0ca41fe95031 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:34.140118 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c2683956-6e80-4012-834d-0ca41fe95031 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00/ips" status: 200 len: 150 microversion: 2.1 time: 0.191664 Aug 28 01:00:34.140833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 286/568] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Aug 28 01:00:33 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00/ips => generated 150 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:00:34.152654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=2 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:00:34.154454 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=9,nova_cell1:INSERT=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:00:34.162378 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-78442428-9869-4e99-a807-328237f77788 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "1dcdecb9-22d2-4941-9359-084d3178b65a"}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:00:34.166039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-78442428-9869-4e99-a807-328237f77788 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:34.175072 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78442428-9869-4e99-a807-328237f77788 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:34.175592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78442428-9869-4e99-a807-328237f77788 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:34.176019 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78442428-9869-4e99-a807-328237f77788 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:34.176724 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78442428-9869-4e99-a807-328237f77788 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:34.177202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78442428-9869-4e99-a807-328237f77788 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:34.177617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78442428-9869-4e99-a807-328237f77788 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:34.285563 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-78442428-9869-4e99-a807-328237f77788 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] HTTP exception thrown: Invalid volume: volume 1dcdecb9-22d2-4941-9359-084d3178b65a is already attached to instances: dff36aeb-f0c7-41f0-8ecc-9924adf8289c Aug 28 01:00:34.287293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-78442428-9869-4e99-a807-328237f77788 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Request method failure captured: Aug 28 01:00:34.287293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: POST /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00/os-volume_attachments HTTP/1.1 Aug 28 01:00:34.287293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:00:34.287293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:00:34.287293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:00:34.287293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 74 Aug 28 01:00:34.287293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:00:34.287293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:00:34.287293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:00:34.287293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:00:34.287293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCrzduxHoOEhvI8P-Mde9cL1oxkmc4fMm9FmGsHxLSNCBjncK-ctZ1RF74JsMq7QMGP1Qh6DECNOIg5DjWuRWqO-CpXkeFE6YBKwQT8SoQT23SwG4DtFzbguCYZajUyKFut9Dn943LOjL5V3-apkT_NK3VlDjeI9Rm1KYQSsNW0BOiQg Aug 28 01:00:34.287293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:00:34.287293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:00:34.287293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:00:34.287293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:00:34.287293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:00:34.287293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:00:34.287293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:00:34.287293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:00:34.287293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:00:34.287293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:00:34.289687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_95df81e14a8549fca0851234f4a00b89", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/95df81e14a8549fca0851234f4a00b89"}]}] Aug 28 01:00:34.289687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:00:34.289687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:00:34.289687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:00:34.289687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:00:34.289687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:00:34.289687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:00:34.289687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: d209174d0db04b3584556824c6d0a3d4 Aug 28 01:00:34.289687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:00:34.289687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Aug 28 01:00:34.289687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {"volumeAttachment": {"volumeId": "1dcdecb9-22d2-4941-9359-084d3178b65a"}} Aug 28 01:00:34.289687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:00:34.290960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Invalid volume: volume 1dcdecb9-22d2-4941-9359-084d3178b65a is already attached to instances: dff36aeb-f0c7-41f0-8ecc-9924adf8289c Aug 28 01:00:34.290960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'server_id': '27028458-f872-49aa-8d37-1a886d566c00', 'body': {'volumeAttachment': {'volumeId': '1dcdecb9-22d2-4941-9359-084d3178b65a'}}} Aug 28 01:00:34.290960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:00:34.290960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:00:34.290960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Aug 28 01:00:34.290960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Aug 28 01:00:34.290960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Aug 28 01:00:34.290960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:00:34.290960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Aug 28 01:00:34.290960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 28 01:00:34.290960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5128, in attach_volume Aug 28 01:00:34.290960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Aug 28 01:00:34.290960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4968, in _check_volume_already_attached Aug 28 01:00:34.290960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Aug 28 01:00:34.290960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 1dcdecb9-22d2-4941-9359-084d3178b65a is already attached to instances: dff36aeb-f0c7-41f0-8ecc-9924adf8289c Aug 28 01:00:34.290960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:00:34.290960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:00:34.290960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:00:34.290960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:00:34.293016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:00:34.293016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:00:34.293016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:00:34.293016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:00:34.293016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:00:34.293016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:00:34.293016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 28 01:00:34.293016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 28 01:00:34.293016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 28 01:00:34.293016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 28 01:00:34.293016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 28 01:00:34.293016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 28 01:00:34.293016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Aug 28 01:00:34.293016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 28 01:00:34.293016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 1dcdecb9-22d2-4941-9359-084d3178b65a is already attached to instances: dff36aeb-f0c7-41f0-8ecc-9924adf8289c Aug 28 01:00:34.293016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:00:34.293016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:00:34.293016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:00:34.293016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:00:34.293016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:00:34.295543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:00:34.295543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:00:34.295543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:00:34.295543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 1dcdecb9-22d2-4941-9359-084d3178b65a is already attached to instances: dff36aeb-f0c7-41f0-8ecc-9924adf8289c Aug 28 01:00:34.295543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:00:34.295543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-78442428-9869-4e99-a807-328237f77788 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Returning 400 to user: Invalid volume: volume 1dcdecb9-22d2-4941-9359-084d3178b65a is already attached to instances: dff36aeb-f0c7-41f0-8ecc-9924adf8289c {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:00:34.295543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-78442428-9869-4e99-a807-328237f77788 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "POST /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.138490 Aug 28 01:00:34.295543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 283/569] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Aug 28 01:00:34 2022] POST /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00/os-volume_attachments => generated 174 bytes in 140 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 28 01:00:34.300112 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ce333707-cc32-4d59-a384-ec18003f38ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:34.300630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ce333707-cc32-4d59-a384-ec18003f38ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:34.308203 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce333707-cc32-4d59-a384-ec18003f38ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:34.308755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce333707-cc32-4d59-a384-ec18003f38ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:34.309482 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce333707-cc32-4d59-a384-ec18003f38ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:34.310734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce333707-cc32-4d59-a384-ec18003f38ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:34.311423 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce333707-cc32-4d59-a384-ec18003f38ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:34.312071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce333707-cc32-4d59-a384-ec18003f38ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:34.343209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ce333707-cc32-4d59-a384-ec18003f38ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Going to try to terminate instance {{(pid=120525) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 28 01:00:34.435719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ce333707-cc32-4d59-a384-ec18003f38ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "DELETE /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 204 len: 0 microversion: 2.1 time: 0.140429 Aug 28 01:00:34.436505 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 287/570] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 01:00:34 2022] DELETE /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Aug 28 01:00:34.445742 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7976b642-68ad-41c7-a50c-d3859daf64a4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:34.446365 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7976b642-68ad-41c7-a50c-d3859daf64a4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:34.453248 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7976b642-68ad-41c7-a50c-d3859daf64a4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:34.453882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7976b642-68ad-41c7-a50c-d3859daf64a4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:34.454410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7976b642-68ad-41c7-a50c-d3859daf64a4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:34.455258 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7976b642-68ad-41c7-a50c-d3859daf64a4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:34.455917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7976b642-68ad-41c7-a50c-d3859daf64a4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:34.456632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7976b642-68ad-41c7-a50c-d3859daf64a4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:34.611523 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7976b642-68ad-41c7-a50c-d3859daf64a4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:34.620178 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7976b642-68ad-41c7-a50c-d3859daf64a4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1681 microversion: 2.1 time: 0.178737 Aug 28 01:00:34.620836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 284/571] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:34 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1681 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:00:35.643450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c7210778-15a4-49a2-9020-f447b6532011 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:35.644141 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c7210778-15a4-49a2-9020-f447b6532011 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:35.652075 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7210778-15a4-49a2-9020-f447b6532011 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:35.652651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7210778-15a4-49a2-9020-f447b6532011 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:35.653135 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7210778-15a4-49a2-9020-f447b6532011 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:35.654028 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7210778-15a4-49a2-9020-f447b6532011 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:35.654485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7210778-15a4-49a2-9020-f447b6532011 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:35.655043 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7210778-15a4-49a2-9020-f447b6532011 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:35.812475 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c7210778-15a4-49a2-9020-f447b6532011 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:35.823488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c7210778-15a4-49a2-9020-f447b6532011 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1681 microversion: 2.1 time: 0.184279 Aug 28 01:00:35.824086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 288/572] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:35 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1681 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:00:36.845873 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-90f031e7-cb96-4da2-a414-ecc5fde1cdd6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:36.846492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-90f031e7-cb96-4da2-a414-ecc5fde1cdd6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:36.855218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90f031e7-cb96-4da2-a414-ecc5fde1cdd6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:36.855758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90f031e7-cb96-4da2-a414-ecc5fde1cdd6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:36.856225 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90f031e7-cb96-4da2-a414-ecc5fde1cdd6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:36.857222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90f031e7-cb96-4da2-a414-ecc5fde1cdd6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:36.857748 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90f031e7-cb96-4da2-a414-ecc5fde1cdd6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:36.858289 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90f031e7-cb96-4da2-a414-ecc5fde1cdd6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:37.010248 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-90f031e7-cb96-4da2-a414-ecc5fde1cdd6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:37.019136 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-90f031e7-cb96-4da2-a414-ecc5fde1cdd6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1681 microversion: 2.1 time: 0.177354 Aug 28 01:00:37.019726 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 285/573] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:36 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1681 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:00:38.053173 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-25bab730-05c5-4c8f-b093-85dd0364af69 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:38.054242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-25bab730-05c5-4c8f-b093-85dd0364af69 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:38.061655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-25bab730-05c5-4c8f-b093-85dd0364af69 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:38.062143 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-25bab730-05c5-4c8f-b093-85dd0364af69 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:38.062574 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-25bab730-05c5-4c8f-b093-85dd0364af69 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:38.063276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-25bab730-05c5-4c8f-b093-85dd0364af69 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:38.063729 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-25bab730-05c5-4c8f-b093-85dd0364af69 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:38.064146 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-25bab730-05c5-4c8f-b093-85dd0364af69 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:38.220245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-25bab730-05c5-4c8f-b093-85dd0364af69 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:38.228547 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-25bab730-05c5-4c8f-b093-85dd0364af69 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1681 microversion: 2.1 time: 0.181361 Aug 28 01:00:38.229156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 289/574] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:38 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1681 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:00:39.255242 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8cfe000f-6e41-45bd-b684-e51af82b261c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:39.255908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8cfe000f-6e41-45bd-b684-e51af82b261c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:39.263627 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8cfe000f-6e41-45bd-b684-e51af82b261c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:39.264174 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8cfe000f-6e41-45bd-b684-e51af82b261c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:39.264748 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8cfe000f-6e41-45bd-b684-e51af82b261c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:39.265556 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8cfe000f-6e41-45bd-b684-e51af82b261c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:39.266047 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8cfe000f-6e41-45bd-b684-e51af82b261c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:39.266541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8cfe000f-6e41-45bd-b684-e51af82b261c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:39.420226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8cfe000f-6e41-45bd-b684-e51af82b261c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:39.429549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8cfe000f-6e41-45bd-b684-e51af82b261c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1681 microversion: 2.1 time: 0.178112 Aug 28 01:00:39.430398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 286/575] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:39 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1681 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:00:40.465081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7e8f01ab-b03f-4cb9-8cfc-f025b4d7a0b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:40.467855 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7e8f01ab-b03f-4cb9-8cfc-f025b4d7a0b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:40.474180 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e8f01ab-b03f-4cb9-8cfc-f025b4d7a0b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:40.474686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e8f01ab-b03f-4cb9-8cfc-f025b4d7a0b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:40.475376 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e8f01ab-b03f-4cb9-8cfc-f025b4d7a0b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:40.476227 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e8f01ab-b03f-4cb9-8cfc-f025b4d7a0b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:40.476902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e8f01ab-b03f-4cb9-8cfc-f025b4d7a0b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:40.477343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e8f01ab-b03f-4cb9-8cfc-f025b4d7a0b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:40.657907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7e8f01ab-b03f-4cb9-8cfc-f025b4d7a0b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:40.667424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7e8f01ab-b03f-4cb9-8cfc-f025b4d7a0b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1681 microversion: 2.1 time: 0.206926 Aug 28 01:00:40.668221 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 290/576] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:40 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1681 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:00:41.691287 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7558559b-22c4-4551-a72a-bc748c7cdad1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:41.693809 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7558559b-22c4-4551-a72a-bc748c7cdad1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:41.703107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7558559b-22c4-4551-a72a-bc748c7cdad1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:41.703900 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7558559b-22c4-4551-a72a-bc748c7cdad1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:41.704533 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7558559b-22c4-4551-a72a-bc748c7cdad1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:41.705449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7558559b-22c4-4551-a72a-bc748c7cdad1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:41.706064 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7558559b-22c4-4551-a72a-bc748c7cdad1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:41.706641 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7558559b-22c4-4551-a72a-bc748c7cdad1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:41.849635 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-86cf8e1b-32f1-4d6c-8214-6f776cdc8298 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:00:41.854165 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-86cf8e1b-32f1-4d6c-8214-6f776cdc8298 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:41.865846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86cf8e1b-32f1-4d6c-8214-6f776cdc8298 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:41.866728 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86cf8e1b-32f1-4d6c-8214-6f776cdc8298 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:41.867498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86cf8e1b-32f1-4d6c-8214-6f776cdc8298 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:41.868757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86cf8e1b-32f1-4d6c-8214-6f776cdc8298 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:41.869602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86cf8e1b-32f1-4d6c-8214-6f776cdc8298 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:41.870361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86cf8e1b-32f1-4d6c-8214-6f776cdc8298 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:41.913897 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.objects.instance [None req-86cf8e1b-32f1-4d6c-8214-6f776cdc8298 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lazy-loading 'flavor' on Instance uuid 03969ac5-934e-4201-8087-0927eae2bbd2 {{(pid=120525) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:00:42.128680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-86cf8e1b-32f1-4d6c-8214-6f776cdc8298 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "POST /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/action" status: 200 len: 32 microversion: 2.1 time: 0.286597 Aug 28 01:00:42.129322 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 291/577] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Aug 28 01:00:41 2022] POST /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/action => generated 32 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 28 01:00:42.142080 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-de33669a-b359-4a10-a299-1697b8277378 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:42.144424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-de33669a-b359-4a10-a299-1697b8277378 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:42.154034 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de33669a-b359-4a10-a299-1697b8277378 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:42.154712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de33669a-b359-4a10-a299-1697b8277378 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:42.155401 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de33669a-b359-4a10-a299-1697b8277378 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:42.156651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de33669a-b359-4a10-a299-1697b8277378 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:42.157507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de33669a-b359-4a10-a299-1697b8277378 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:42.158985 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de33669a-b359-4a10-a299-1697b8277378 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:42.451085 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7558559b-22c4-4551-a72a-bc748c7cdad1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:42.459913 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7558559b-22c4-4551-a72a-bc748c7cdad1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1681 microversion: 2.1 time: 0.772626 Aug 28 01:00:42.460825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 287/578] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:41 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1681 bytes in 774 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:00:43.098362 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-de33669a-b359-4a10-a299-1697b8277378 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:43.107646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-de33669a-b359-4a10-a299-1697b8277378 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.971425 Aug 28 01:00:43.108208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 292/579] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:42 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 973 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:00:43.483968 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-eebfb694-d642-49e8-a694-5fe958549d9e req-f8960ad5-649b-458e-ad58-3c626e8a7e5f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "27028458-f872-49aa-8d37-1a886d566c00", "name": "network-vif-unplugged", "status": "completed", "tag": "59c8cffa-e3e1-4a46-979a-94e0428e33d6"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:00:43.488650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-86c523d5-e3f9-4eea-9e4f-e38c1b1b208f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:43.490823 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-86c523d5-e3f9-4eea-9e4f-e38c1b1b208f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:43.495975 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-eebfb694-d642-49e8-a694-5fe958549d9e req-f8960ad5-649b-458e-ad58-3c626e8a7e5f service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:43.496601 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-eebfb694-d642-49e8-a694-5fe958549d9e req-f8960ad5-649b-458e-ad58-3c626e8a7e5f service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:43.497239 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-eebfb694-d642-49e8-a694-5fe958549d9e req-f8960ad5-649b-458e-ad58-3c626e8a7e5f service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:43.503685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86c523d5-e3f9-4eea-9e4f-e38c1b1b208f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:43.504191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86c523d5-e3f9-4eea-9e4f-e38c1b1b208f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:43.504895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86c523d5-e3f9-4eea-9e4f-e38c1b1b208f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:43.506455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86c523d5-e3f9-4eea-9e4f-e38c1b1b208f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:43.507233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86c523d5-e3f9-4eea-9e4f-e38c1b1b208f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:43.507791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86c523d5-e3f9-4eea-9e4f-e38c1b1b208f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:43.509829 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-eebfb694-d642-49e8-a694-5fe958549d9e req-f8960ad5-649b-458e-ad58-3c626e8a7e5f service nova] Creating event network-vif-unplugged:59c8cffa-e3e1-4a46-979a-94e0428e33d6 for instance 27028458-f872-49aa-8d37-1a886d566c00 on hv-cinder-maste Aug 28 01:00:43.515678 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-eebfb694-d642-49e8-a694-5fe958549d9e req-f8960ad5-649b-458e-ad58-3c626e8a7e5f service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.149602 Aug 28 01:00:43.516391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 288/580] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:00:43 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:00:43.938345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-86c523d5-e3f9-4eea-9e4f-e38c1b1b208f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:43.950954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-86c523d5-e3f9-4eea-9e4f-e38c1b1b208f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1681 microversion: 2.1 time: 0.471244 Aug 28 01:00:43.952010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 293/581] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:43 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1681 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:00:44.130547 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c24b725f-e2de-4c65-b2af-86f604dea654 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:44.131448 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c24b725f-e2de-4c65-b2af-86f604dea654 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:44.141758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c24b725f-e2de-4c65-b2af-86f604dea654 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:44.142976 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c24b725f-e2de-4c65-b2af-86f604dea654 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:44.143607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c24b725f-e2de-4c65-b2af-86f604dea654 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:44.145013 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c24b725f-e2de-4c65-b2af-86f604dea654 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:44.145817 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c24b725f-e2de-4c65-b2af-86f604dea654 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:44.146551 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c24b725f-e2de-4c65-b2af-86f604dea654 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:44.625515 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c24b725f-e2de-4c65-b2af-86f604dea654 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:44.637336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c24b725f-e2de-4c65-b2af-86f604dea654 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.511028 Aug 28 01:00:44.639323 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 289/582] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:44 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 514 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:00:44.798081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-73bad55f-51cd-48aa-9902-425a9c9f99ed req-7493d141-37b2-4d86-a50c-f83115adb515 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "27028458-f872-49aa-8d37-1a886d566c00", "name": "network-vif-deleted", "tag": "59c8cffa-e3e1-4a46-979a-94e0428e33d6"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:00:44.809070 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-73bad55f-51cd-48aa-9902-425a9c9f99ed req-7493d141-37b2-4d86-a50c-f83115adb515 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:44.809655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-73bad55f-51cd-48aa-9902-425a9c9f99ed req-7493d141-37b2-4d86-a50c-f83115adb515 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:44.810338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-73bad55f-51cd-48aa-9902-425a9c9f99ed req-7493d141-37b2-4d86-a50c-f83115adb515 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:44.823630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-73bad55f-51cd-48aa-9902-425a9c9f99ed req-7493d141-37b2-4d86-a50c-f83115adb515 service nova] Creating event network-vif-deleted:59c8cffa-e3e1-4a46-979a-94e0428e33d6 for instance 27028458-f872-49aa-8d37-1a886d566c00 on hv-cinder-maste Aug 28 01:00:44.829360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-73bad55f-51cd-48aa-9902-425a9c9f99ed req-7493d141-37b2-4d86-a50c-f83115adb515 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.173184 Aug 28 01:00:44.830743 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 294/583] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:00:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:00:44.988076 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5a8f7290-9248-4b61-8a6c-db6ecb73d671 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:44.989114 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5a8f7290-9248-4b61-8a6c-db6ecb73d671 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:44.996204 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a8f7290-9248-4b61-8a6c-db6ecb73d671 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:44.997055 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a8f7290-9248-4b61-8a6c-db6ecb73d671 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:44.997689 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a8f7290-9248-4b61-8a6c-db6ecb73d671 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:44.998794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a8f7290-9248-4b61-8a6c-db6ecb73d671 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:44.999453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a8f7290-9248-4b61-8a6c-db6ecb73d671 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:45.000052 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a8f7290-9248-4b61-8a6c-db6ecb73d671 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:45.092894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5a8f7290-9248-4b61-8a6c-db6ecb73d671 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:45.105715 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5a8f7290-9248-4b61-8a6c-db6ecb73d671 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1326 microversion: 2.1 time: 0.121801 Aug 28 01:00:45.107898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 290/584] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:44 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1326 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:00:45.675940 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-147551c5-aac6-4aaf-9c1c-52f44cda99ac tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:45.679266 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-147551c5-aac6-4aaf-9c1c-52f44cda99ac tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:45.692170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-147551c5-aac6-4aaf-9c1c-52f44cda99ac tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:45.693675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-147551c5-aac6-4aaf-9c1c-52f44cda99ac tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:45.695457 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-147551c5-aac6-4aaf-9c1c-52f44cda99ac tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:45.697207 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-147551c5-aac6-4aaf-9c1c-52f44cda99ac tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:45.698615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-147551c5-aac6-4aaf-9c1c-52f44cda99ac tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:45.699453 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-147551c5-aac6-4aaf-9c1c-52f44cda99ac tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:45.985312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-147551c5-aac6-4aaf-9c1c-52f44cda99ac tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:46.000486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-147551c5-aac6-4aaf-9c1c-52f44cda99ac tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.333555 Aug 28 01:00:46.001171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 295/585] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:45 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:00:46.148821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e95da720-305b-47ae-b02b-b2d7ab1843a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:46.150080 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e95da720-305b-47ae-b02b-b2d7ab1843a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:46.163551 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e95da720-305b-47ae-b02b-b2d7ab1843a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:46.164210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e95da720-305b-47ae-b02b-b2d7ab1843a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:46.164923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e95da720-305b-47ae-b02b-b2d7ab1843a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:46.166179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e95da720-305b-47ae-b02b-b2d7ab1843a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:46.166971 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e95da720-305b-47ae-b02b-b2d7ab1843a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:46.167635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e95da720-305b-47ae-b02b-b2d7ab1843a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:46.216386 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e95da720-305b-47ae-b02b-b2d7ab1843a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:46.216875 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e95da720-305b-47ae-b02b-b2d7ab1843a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:46.217357 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e95da720-305b-47ae-b02b-b2d7ab1843a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:46.217810 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.objects.instance [None req-e95da720-305b-47ae-b02b-b2d7ab1843a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lazy-loading 'fault' on Instance uuid 27028458-f872-49aa-8d37-1a886d566c00 {{(pid=120524) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:00:46.265517 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e95da720-305b-47ae-b02b-b2d7ab1843a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:46.275026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e95da720-305b-47ae-b02b-b2d7ab1843a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 200 len: 1331 microversion: 2.1 time: 0.134791 Aug 28 01:00:46.276585 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 291/586] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:46 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 1331 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:00:47.036762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7a6f7eb7-7310-4a84-a104-a758a10f7eee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:47.037662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7a6f7eb7-7310-4a84-a104-a758a10f7eee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:47.044552 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7a6f7eb7-7310-4a84-a104-a758a10f7eee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:47.045079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7a6f7eb7-7310-4a84-a104-a758a10f7eee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:47.045498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7a6f7eb7-7310-4a84-a104-a758a10f7eee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:47.046237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7a6f7eb7-7310-4a84-a104-a758a10f7eee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:47.046688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7a6f7eb7-7310-4a84-a104-a758a10f7eee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:47.047239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7a6f7eb7-7310-4a84-a104-a758a10f7eee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:47.201701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7a6f7eb7-7310-4a84-a104-a758a10f7eee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:47.210825 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7a6f7eb7-7310-4a84-a104-a758a10f7eee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.178551 Aug 28 01:00:47.211799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 296/587] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:47 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:00:47.307783 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-65931e23-3dc3-45be-90fe-774e218abc47 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:47.308670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-65931e23-3dc3-45be-90fe-774e218abc47 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:47.315235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-65931e23-3dc3-45be-90fe-774e218abc47 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:47.315733 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-65931e23-3dc3-45be-90fe-774e218abc47 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:47.316171 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-65931e23-3dc3-45be-90fe-774e218abc47 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:47.316900 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-65931e23-3dc3-45be-90fe-774e218abc47 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:47.317404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-65931e23-3dc3-45be-90fe-774e218abc47 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:47.317826 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-65931e23-3dc3-45be-90fe-774e218abc47 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:47.333096 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-65931e23-3dc3-45be-90fe-774e218abc47 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] HTTP exception thrown: Instance 27028458-f872-49aa-8d37-1a886d566c00 could not be found. Aug 28 01:00:47.334985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-65931e23-3dc3-45be-90fe-774e218abc47 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Request method failure captured: Aug 28 01:00:47.334985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 HTTP/1.1 Aug 28 01:00:47.334985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:00:47.334985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:00:47.334985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:00:47.334985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:00:47.334985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:00:47.334985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:00:47.334985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:00:47.334985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:00:47.334985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCrzduxHoOEhvI8P-Mde9cL1oxkmc4fMm9FmGsHxLSNCBjncK-ctZ1RF74JsMq7QMGP1Qh6DECNOIg5DjWuRWqO-CpXkeFE6YBKwQT8SoQT23SwG4DtFzbguCYZajUyKFut9Dn943LOjL5V3-apkT_NK3VlDjeI9Rm1KYQSsNW0BOiQg Aug 28 01:00:47.334985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:00:47.334985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:00:47.334985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:00:47.334985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:00:47.334985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:00:47.334985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:00:47.334985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:00:47.334985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:00:47.334985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:00:47.334985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:00:47.337297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_95df81e14a8549fca0851234f4a00b89", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/95df81e14a8549fca0851234f4a00b89"}]}] Aug 28 01:00:47.337297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:00:47.337297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:00:47.337297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:00:47.337297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:00:47.337297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:00:47.337297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:00:47.337297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: d209174d0db04b3584556824c6d0a3d4 Aug 28 01:00:47.337297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:00:47.337297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:00:47.337297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Instance 27028458-f872-49aa-8d37-1a886d566c00 could not be found. Aug 28 01:00:47.337297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'id': '27028458-f872-49aa-8d37-1a886d566c00'} Aug 28 01:00:47.338571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:00:47.338571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:00:47.338571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:00:47.338571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:00:47.338571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:00:47.338571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:00:47.338571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:00:47.338571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:00:47.338571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:00:47.338571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:00:47.338571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 27028458-f872-49aa-8d37-1a886d566c00 could not be found. Aug 28 01:00:47.338571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:00:47.338571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:00:47.338571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:00:47.338571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:00:47.338571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:00:47.338571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:00:47.338571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:00:47.338571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:00:47.338571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:00:47.341001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:00:47.341001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 01:00:47.341001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 01:00:47.341001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:00:47.341001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:00:47.341001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:00:47.341001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:00:47.341001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 27028458-f872-49aa-8d37-1a886d566c00 could not be found. Aug 28 01:00:47.341001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:00:47.341001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:00:47.341001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:00:47.341001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:00:47.341001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:00:47.341001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:00:47.341001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:00:47.341001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:00:47.341001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 27028458-f872-49aa-8d37-1a886d566c00 could not be found. Aug 28 01:00:47.341001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:00:47.343006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-65931e23-3dc3-45be-90fe-774e218abc47 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Returning 404 to user: Instance 27028458-f872-49aa-8d37-1a886d566c00 could not be found. {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:00:47.343006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-65931e23-3dc3-45be-90fe-774e218abc47 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 404 len: 111 microversion: 2.1 time: 0.033275 Aug 28 01:00:47.343006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 292/588] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:47 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 28 01:00:47.348517 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d413fc5e-1f74-408b-856c-85aa8fb8533f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '.version_select at 0x7f0e6e8a1a60>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:47.373084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d413fc5e-1f74-408b-856c-85aa8fb8533f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-669625462" status: 202 len: 0 microversion: 2.1 time: 0.028452 Aug 28 01:00:47.373784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 297/589] 10.222.0.42 () {58 vars in 1218 bytes} [Sun Aug 28 01:00:47 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-669625462 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 01:00:48.234433 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-35ab6987-2494-4f9d-8428-0fb2d06aaba0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:48.235681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-35ab6987-2494-4f9d-8428-0fb2d06aaba0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:48.245416 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-35ab6987-2494-4f9d-8428-0fb2d06aaba0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:48.246232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-35ab6987-2494-4f9d-8428-0fb2d06aaba0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:48.246991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-35ab6987-2494-4f9d-8428-0fb2d06aaba0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:48.248143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-35ab6987-2494-4f9d-8428-0fb2d06aaba0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:48.248908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-35ab6987-2494-4f9d-8428-0fb2d06aaba0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:48.249801 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-35ab6987-2494-4f9d-8428-0fb2d06aaba0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:48.274519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8375d83b-3d6e-4a4b-b423-bad97d09dc3d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:48.275406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8375d83b-3d6e-4a4b-b423-bad97d09dc3d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:48.287363 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8375d83b-3d6e-4a4b-b423-bad97d09dc3d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:48.288293 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8375d83b-3d6e-4a4b-b423-bad97d09dc3d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:48.289088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8375d83b-3d6e-4a4b-b423-bad97d09dc3d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:48.290448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8375d83b-3d6e-4a4b-b423-bad97d09dc3d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:48.291249 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8375d83b-3d6e-4a4b-b423-bad97d09dc3d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:48.292021 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8375d83b-3d6e-4a4b-b423-bad97d09dc3d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:48.395780 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-35ab6987-2494-4f9d-8428-0fb2d06aaba0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:48.405048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-35ab6987-2494-4f9d-8428-0fb2d06aaba0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.175079 Aug 28 01:00:48.405793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 293/590] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:48 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:00:48.499806 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.objects.instance [None req-8375d83b-3d6e-4a4b-b423-bad97d09dc3d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lazy-loading 'services' on Instance uuid dff36aeb-f0c7-41f0-8ecc-9924adf8289c {{(pid=120525) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:00:48.623224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8375d83b-3d6e-4a4b-b423-bad97d09dc3d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "DELETE /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments/1dcdecb9-22d2-4941-9359-084d3178b65a" status: 202 len: 0 microversion: 2.1 time: 0.357028 Aug 28 01:00:48.623893 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 298/591] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Aug 28 01:00:48 2022] DELETE /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments/1dcdecb9-22d2-4941-9359-084d3178b65a => generated 0 bytes in 359 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 28 01:00:48.635616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cdcfd025-ea64-4630-8c0e-46d6f789e652 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:48.639986 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cdcfd025-ea64-4630-8c0e-46d6f789e652 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:48.651101 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cdcfd025-ea64-4630-8c0e-46d6f789e652 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:48.651822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cdcfd025-ea64-4630-8c0e-46d6f789e652 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:48.652599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cdcfd025-ea64-4630-8c0e-46d6f789e652 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:48.653939 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cdcfd025-ea64-4630-8c0e-46d6f789e652 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:48.654755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cdcfd025-ea64-4630-8c0e-46d6f789e652 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:48.655605 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cdcfd025-ea64-4630-8c0e-46d6f789e652 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:48.696695 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cdcfd025-ea64-4630-8c0e-46d6f789e652 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067744 Aug 28 01:00:48.697446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 294/592] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:00:48 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:00:49.428219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-04e9e750-9286-436b-91f3-6473b3746f0b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:49.428951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-04e9e750-9286-436b-91f3-6473b3746f0b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:49.436033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-04e9e750-9286-436b-91f3-6473b3746f0b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:49.436551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-04e9e750-9286-436b-91f3-6473b3746f0b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:49.437014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-04e9e750-9286-436b-91f3-6473b3746f0b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:49.437739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-04e9e750-9286-436b-91f3-6473b3746f0b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:49.438197 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-04e9e750-9286-436b-91f3-6473b3746f0b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:49.438594 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-04e9e750-9286-436b-91f3-6473b3746f0b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:49.594833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-04e9e750-9286-436b-91f3-6473b3746f0b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:49.603862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-04e9e750-9286-436b-91f3-6473b3746f0b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.180473 Aug 28 01:00:49.604464 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 299/593] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:49 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:00:49.714257 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f47c27d6-9374-4285-bf2b-d7681f8d487e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:49.717404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f47c27d6-9374-4285-bf2b-d7681f8d487e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:49.726307 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f47c27d6-9374-4285-bf2b-d7681f8d487e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:49.726795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f47c27d6-9374-4285-bf2b-d7681f8d487e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:49.727214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f47c27d6-9374-4285-bf2b-d7681f8d487e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:49.727936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f47c27d6-9374-4285-bf2b-d7681f8d487e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:49.728382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f47c27d6-9374-4285-bf2b-d7681f8d487e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:49.728848 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f47c27d6-9374-4285-bf2b-d7681f8d487e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:49.763609 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f47c27d6-9374-4285-bf2b-d7681f8d487e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056313 Aug 28 01:00:49.764226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 295/594] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:00:49 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:00:50.217071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-de7ce0de-c589-4145-a323-956c09f6af95 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:50.217695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-de7ce0de-c589-4145-a323-956c09f6af95 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:50.224397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de7ce0de-c589-4145-a323-956c09f6af95 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:50.224880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de7ce0de-c589-4145-a323-956c09f6af95 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:50.225373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de7ce0de-c589-4145-a323-956c09f6af95 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:50.226078 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de7ce0de-c589-4145-a323-956c09f6af95 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:50.226518 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de7ce0de-c589-4145-a323-956c09f6af95 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:50.226938 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de7ce0de-c589-4145-a323-956c09f6af95 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:50.231522 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:00:50.261596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-de7ce0de-c589-4145-a323-956c09f6af95 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments/db8c499e-16a0-4d0c-b4f8-0f61f3a06a3c" status: 200 len: 194 microversion: 2.20 time: 0.165587 Aug 28 01:00:50.262210 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 300/595] 10.222.0.42 () {60 vars in 1454 bytes} [Sun Aug 28 01:00:50 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments/db8c499e-16a0-4d0c-b4f8-0f61f3a06a3c => generated 194 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:00:50.379559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-44adcf62-fb4e-4640-91e7-c174d8263dc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:50.380521 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-44adcf62-fb4e-4640-91e7-c174d8263dc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:50.391045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-44adcf62-fb4e-4640-91e7-c174d8263dc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:50.391881 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-44adcf62-fb4e-4640-91e7-c174d8263dc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:50.392632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-44adcf62-fb4e-4640-91e7-c174d8263dc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:50.393885 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-44adcf62-fb4e-4640-91e7-c174d8263dc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:50.394704 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-44adcf62-fb4e-4640-91e7-c174d8263dc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:50.395439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-44adcf62-fb4e-4640-91e7-c174d8263dc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:50.530189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.objects.instance [None req-44adcf62-fb4e-4640-91e7-c174d8263dc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lazy-loading 'services' on Instance uuid dd61123b-7a96-4c9e-936c-5794a4fdd7df {{(pid=120524) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:00:50.624929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-44adcf62-fb4e-4640-91e7-c174d8263dc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "DELETE /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments/db8c499e-16a0-4d0c-b4f8-0f61f3a06a3c" status: 202 len: 0 microversion: 2.20 time: 0.252791 Aug 28 01:00:50.625952 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 296/596] 10.222.0.42 () {60 vars in 1457 bytes} [Sun Aug 28 01:00:50 2022] DELETE /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments/db8c499e-16a0-4d0c-b4f8-0f61f3a06a3c => generated 0 bytes in 256 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Aug 28 01:00:50.629261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e2f72c0e-0fab-4698-854f-00561e82f64d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:50.630390 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e2f72c0e-0fab-4698-854f-00561e82f64d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:50.640125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9a3f6f57-faf6-460c-ae38-8e0000787985 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:50.641321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2f72c0e-0fab-4698-854f-00561e82f64d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:50.642118 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2f72c0e-0fab-4698-854f-00561e82f64d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:50.642851 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2f72c0e-0fab-4698-854f-00561e82f64d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:50.644010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2f72c0e-0fab-4698-854f-00561e82f64d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:50.644150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9a3f6f57-faf6-460c-ae38-8e0000787985 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:50.645018 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2f72c0e-0fab-4698-854f-00561e82f64d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:50.645717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2f72c0e-0fab-4698-854f-00561e82f64d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:50.654180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9a3f6f57-faf6-460c-ae38-8e0000787985 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:50.654676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9a3f6f57-faf6-460c-ae38-8e0000787985 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:50.655189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9a3f6f57-faf6-460c-ae38-8e0000787985 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:50.655828 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9a3f6f57-faf6-460c-ae38-8e0000787985 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:50.656284 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9a3f6f57-faf6-460c-ae38-8e0000787985 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:50.656703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9a3f6f57-faf6-460c-ae38-8e0000787985 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:50.711275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9a3f6f57-faf6-460c-ae38-8e0000787985 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.078565 Aug 28 01:00:50.712560 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 297/597] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:00:50 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:00:50.777466 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f10e5252-0988-4e91-86f4-ed74e122f264 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:50.780547 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f10e5252-0988-4e91-86f4-ed74e122f264 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:50.788075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f10e5252-0988-4e91-86f4-ed74e122f264 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:50.788576 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f10e5252-0988-4e91-86f4-ed74e122f264 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:50.789038 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f10e5252-0988-4e91-86f4-ed74e122f264 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:50.789778 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f10e5252-0988-4e91-86f4-ed74e122f264 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:50.790258 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f10e5252-0988-4e91-86f4-ed74e122f264 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:50.790687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f10e5252-0988-4e91-86f4-ed74e122f264 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:50.809488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e2f72c0e-0fab-4698-854f-00561e82f64d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:50.818768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e2f72c0e-0fab-4698-854f-00561e82f64d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.195047 Aug 28 01:00:50.819297 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 301/598] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:50 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:00:50.825595 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f10e5252-0988-4e91-86f4-ed74e122f264 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052003 Aug 28 01:00:50.826187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 298/599] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:00:50 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:00:51.728895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ab0ab658-df2f-4057-bbf1-751cbf397fe0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:51.731882 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ab0ab658-df2f-4057-bbf1-751cbf397fe0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:51.740695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab0ab658-df2f-4057-bbf1-751cbf397fe0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:51.741276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab0ab658-df2f-4057-bbf1-751cbf397fe0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:51.741716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab0ab658-df2f-4057-bbf1-751cbf397fe0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:51.742456 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab0ab658-df2f-4057-bbf1-751cbf397fe0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:51.742911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab0ab658-df2f-4057-bbf1-751cbf397fe0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:51.743336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab0ab658-df2f-4057-bbf1-751cbf397fe0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:51.780022 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ab0ab658-df2f-4057-bbf1-751cbf397fe0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055488 Aug 28 01:00:51.780859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 302/600] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:00:51 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:00:51.839653 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fbf73334-e789-41cf-aa42-c868bc672744 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:51.843074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fbf73334-e789-41cf-aa42-c868bc672744 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:51.854347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fbf73334-e789-41cf-aa42-c868bc672744 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:51.854680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-86c74033-f1a9-4abd-b108-c8f138fd6e4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:51.855098 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fbf73334-e789-41cf-aa42-c868bc672744 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:51.855568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-86c74033-f1a9-4abd-b108-c8f138fd6e4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:51.855803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fbf73334-e789-41cf-aa42-c868bc672744 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:51.856955 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fbf73334-e789-41cf-aa42-c868bc672744 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:51.857724 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fbf73334-e789-41cf-aa42-c868bc672744 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:51.858401 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fbf73334-e789-41cf-aa42-c868bc672744 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:51.865534 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86c74033-f1a9-4abd-b108-c8f138fd6e4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:51.866260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86c74033-f1a9-4abd-b108-c8f138fd6e4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:51.866974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86c74033-f1a9-4abd-b108-c8f138fd6e4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:51.868392 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86c74033-f1a9-4abd-b108-c8f138fd6e4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:51.869334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86c74033-f1a9-4abd-b108-c8f138fd6e4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:51.870019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86c74033-f1a9-4abd-b108-c8f138fd6e4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:51.894743 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:00:51.901704 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fbf73334-e789-41cf-aa42-c868bc672744 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.066487 Aug 28 01:00:51.902580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 299/601] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:00:51 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:00:52.037623 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-86c74033-f1a9-4abd-b108-c8f138fd6e4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:52.046742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-86c74033-f1a9-4abd-b108-c8f138fd6e4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.199447 Aug 28 01:00:52.047373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 303/602] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:51 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:00:52.793392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4b16870d-5d3b-420c-b27b-37aa84e4d861 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:52.795480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4b16870d-5d3b-420c-b27b-37aa84e4d861 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:52.803193 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b16870d-5d3b-420c-b27b-37aa84e4d861 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:52.803688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b16870d-5d3b-420c-b27b-37aa84e4d861 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:52.804106 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b16870d-5d3b-420c-b27b-37aa84e4d861 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:52.804863 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b16870d-5d3b-420c-b27b-37aa84e4d861 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:52.805363 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b16870d-5d3b-420c-b27b-37aa84e4d861 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:52.805794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b16870d-5d3b-420c-b27b-37aa84e4d861 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:52.840764 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4b16870d-5d3b-420c-b27b-37aa84e4d861 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051631 Aug 28 01:00:52.841611 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 300/603] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:00:52 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:00:52.917570 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c9c90cbb-ad90-4ac5-b24c-be5a3a9da0cc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:52.919833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c9c90cbb-ad90-4ac5-b24c-be5a3a9da0cc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:52.927213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c9c90cbb-ad90-4ac5-b24c-be5a3a9da0cc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:52.927708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c9c90cbb-ad90-4ac5-b24c-be5a3a9da0cc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:52.928142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c9c90cbb-ad90-4ac5-b24c-be5a3a9da0cc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:52.928834 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c9c90cbb-ad90-4ac5-b24c-be5a3a9da0cc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:52.929321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c9c90cbb-ad90-4ac5-b24c-be5a3a9da0cc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:52.929764 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c9c90cbb-ad90-4ac5-b24c-be5a3a9da0cc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:52.964247 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c9c90cbb-ad90-4ac5-b24c-be5a3a9da0cc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050583 Aug 28 01:00:52.964862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 304/604] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:00:52 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:00:53.068939 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-86f293b2-24dc-458b-8868-779a44021841 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:53.069858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-86f293b2-24dc-458b-8868-779a44021841 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:53.077081 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86f293b2-24dc-458b-8868-779a44021841 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:53.077604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86f293b2-24dc-458b-8868-779a44021841 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:53.078032 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86f293b2-24dc-458b-8868-779a44021841 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:53.078743 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86f293b2-24dc-458b-8868-779a44021841 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:53.079188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86f293b2-24dc-458b-8868-779a44021841 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:53.079592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86f293b2-24dc-458b-8868-779a44021841 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:53.235172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-86f293b2-24dc-458b-8868-779a44021841 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:53.250956 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-86f293b2-24dc-458b-8868-779a44021841 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.185743 Aug 28 01:00:53.251991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 301/605] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:53 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:00:53.857092 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1372b9c9-28b4-47b4-a78b-558b4bfa85e0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:53.859285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1372b9c9-28b4-47b4-a78b-558b4bfa85e0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:53.866176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1372b9c9-28b4-47b4-a78b-558b4bfa85e0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:53.866666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1372b9c9-28b4-47b4-a78b-558b4bfa85e0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:53.867071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1372b9c9-28b4-47b4-a78b-558b4bfa85e0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:53.867806 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1372b9c9-28b4-47b4-a78b-558b4bfa85e0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:53.868251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1372b9c9-28b4-47b4-a78b-558b4bfa85e0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:53.868667 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1372b9c9-28b4-47b4-a78b-558b4bfa85e0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:53.903426 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1372b9c9-28b4-47b4-a78b-558b4bfa85e0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050489 Aug 28 01:00:53.903997 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 305/606] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:00:53 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:00:53.979022 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a91d1281-b097-4ebd-8a36-2b3a57d58a76 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:53.982294 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a91d1281-b097-4ebd-8a36-2b3a57d58a76 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:53.990662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a91d1281-b097-4ebd-8a36-2b3a57d58a76 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:53.991263 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a91d1281-b097-4ebd-8a36-2b3a57d58a76 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:53.991878 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a91d1281-b097-4ebd-8a36-2b3a57d58a76 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:53.992794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a91d1281-b097-4ebd-8a36-2b3a57d58a76 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:53.993358 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a91d1281-b097-4ebd-8a36-2b3a57d58a76 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:53.993864 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a91d1281-b097-4ebd-8a36-2b3a57d58a76 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:54.029818 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a91d1281-b097-4ebd-8a36-2b3a57d58a76 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055397 Aug 28 01:00:54.030450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 302/607] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:00:53 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:00:54.288536 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ab675b8e-9f6e-4606-beff-39fb52f211b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:54.289224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ab675b8e-9f6e-4606-beff-39fb52f211b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:54.297795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab675b8e-9f6e-4606-beff-39fb52f211b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:54.298563 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab675b8e-9f6e-4606-beff-39fb52f211b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:54.299083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab675b8e-9f6e-4606-beff-39fb52f211b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:54.299880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab675b8e-9f6e-4606-beff-39fb52f211b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:54.300344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab675b8e-9f6e-4606-beff-39fb52f211b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:54.300772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab675b8e-9f6e-4606-beff-39fb52f211b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:54.470792 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ab675b8e-9f6e-4606-beff-39fb52f211b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:54.480642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ab675b8e-9f6e-4606-beff-39fb52f211b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.196727 Aug 28 01:00:54.481312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 306/608] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:54 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:00:54.916736 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-402c3a41-d341-41b2-acce-16bd84d738ca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:54.919278 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-402c3a41-d341-41b2-acce-16bd84d738ca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:54.928260 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-402c3a41-d341-41b2-acce-16bd84d738ca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:54.928762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-402c3a41-d341-41b2-acce-16bd84d738ca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:54.929222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-402c3a41-d341-41b2-acce-16bd84d738ca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:54.929947 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-402c3a41-d341-41b2-acce-16bd84d738ca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:54.930426 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-402c3a41-d341-41b2-acce-16bd84d738ca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:54.930844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-402c3a41-d341-41b2-acce-16bd84d738ca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:54.965950 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-402c3a41-d341-41b2-acce-16bd84d738ca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053288 Aug 28 01:00:54.966521 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 303/609] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:00:54 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:00:55.043975 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b494d8ae-e2c4-463d-b952-55ca48deb9c4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:55.046146 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b494d8ae-e2c4-463d-b952-55ca48deb9c4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:55.054264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b494d8ae-e2c4-463d-b952-55ca48deb9c4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:55.054762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b494d8ae-e2c4-463d-b952-55ca48deb9c4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:55.055199 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b494d8ae-e2c4-463d-b952-55ca48deb9c4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:55.055913 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b494d8ae-e2c4-463d-b952-55ca48deb9c4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:55.056366 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b494d8ae-e2c4-463d-b952-55ca48deb9c4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:55.056792 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b494d8ae-e2c4-463d-b952-55ca48deb9c4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:55.092414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b494d8ae-e2c4-463d-b952-55ca48deb9c4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052892 Aug 28 01:00:55.093052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 307/610] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:00:55 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:00:55.503333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a91a742b-2460-4e1d-ac1a-9a58418cb531 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:55.504082 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a91a742b-2460-4e1d-ac1a-9a58418cb531 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:55.511705 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a91a742b-2460-4e1d-ac1a-9a58418cb531 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:55.512246 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a91a742b-2460-4e1d-ac1a-9a58418cb531 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:55.512760 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a91a742b-2460-4e1d-ac1a-9a58418cb531 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:55.513627 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a91a742b-2460-4e1d-ac1a-9a58418cb531 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:55.514231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a91a742b-2460-4e1d-ac1a-9a58418cb531 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:55.514720 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a91a742b-2460-4e1d-ac1a-9a58418cb531 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:55.695624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a91a742b-2460-4e1d-ac1a-9a58418cb531 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:55.705619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a91a742b-2460-4e1d-ac1a-9a58418cb531 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.206160 Aug 28 01:00:55.706464 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 304/611] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:55 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:00:55.979523 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b16a8b0a-7ea0-497e-bfef-425fffd5fa65 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:55.982064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b16a8b0a-7ea0-497e-bfef-425fffd5fa65 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:55.989438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b16a8b0a-7ea0-497e-bfef-425fffd5fa65 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:55.989932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b16a8b0a-7ea0-497e-bfef-425fffd5fa65 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:55.990370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b16a8b0a-7ea0-497e-bfef-425fffd5fa65 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:55.991109 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b16a8b0a-7ea0-497e-bfef-425fffd5fa65 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:55.991580 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b16a8b0a-7ea0-497e-bfef-425fffd5fa65 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:55.991987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b16a8b0a-7ea0-497e-bfef-425fffd5fa65 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:56.035174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b16a8b0a-7ea0-497e-bfef-425fffd5fa65 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.059632 Aug 28 01:00:56.036013 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 308/612] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:00:55 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 28 01:00:56.106483 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3baa4409-cf2a-43e1-a809-fe9d2d6afdee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:56.108868 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3baa4409-cf2a-43e1-a809-fe9d2d6afdee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:56.118315 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3baa4409-cf2a-43e1-a809-fe9d2d6afdee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:56.119081 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3baa4409-cf2a-43e1-a809-fe9d2d6afdee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:56.119765 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3baa4409-cf2a-43e1-a809-fe9d2d6afdee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:56.120832 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3baa4409-cf2a-43e1-a809-fe9d2d6afdee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:56.121559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3baa4409-cf2a-43e1-a809-fe9d2d6afdee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:56.122187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3baa4409-cf2a-43e1-a809-fe9d2d6afdee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:56.157965 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3baa4409-cf2a-43e1-a809-fe9d2d6afdee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055730 Aug 28 01:00:56.158739 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 305/613] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:00:56 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:00:56.730581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e0969915-aecf-419f-b32f-8114313e5bd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:56.731216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e0969915-aecf-419f-b32f-8114313e5bd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:56.737834 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0969915-aecf-419f-b32f-8114313e5bd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:56.738427 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0969915-aecf-419f-b32f-8114313e5bd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:56.738853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0969915-aecf-419f-b32f-8114313e5bd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:56.739553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0969915-aecf-419f-b32f-8114313e5bd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:56.739983 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0969915-aecf-419f-b32f-8114313e5bd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:56.740473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0969915-aecf-419f-b32f-8114313e5bd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:56.894796 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e0969915-aecf-419f-b32f-8114313e5bd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:56.904477 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e0969915-aecf-419f-b32f-8114313e5bd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.179605 Aug 28 01:00:56.905099 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 309/614] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:56 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:00:57.048162 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8e24987c-b635-4235-8564-02f3daea6f71 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:57.050283 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8e24987c-b635-4235-8564-02f3daea6f71 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:57.057254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e24987c-b635-4235-8564-02f3daea6f71 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:57.057736 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e24987c-b635-4235-8564-02f3daea6f71 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:57.058166 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e24987c-b635-4235-8564-02f3daea6f71 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:57.058965 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e24987c-b635-4235-8564-02f3daea6f71 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:57.059405 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e24987c-b635-4235-8564-02f3daea6f71 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:57.059816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e24987c-b635-4235-8564-02f3daea6f71 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:57.093481 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8e24987c-b635-4235-8564-02f3daea6f71 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049391 Aug 28 01:00:57.094080 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 306/615] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:00:57 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:00:57.170640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ac0e16a8-b89f-4a98-91c8-02c579afa7b8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:57.172783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ac0e16a8-b89f-4a98-91c8-02c579afa7b8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:57.180036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac0e16a8-b89f-4a98-91c8-02c579afa7b8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:57.180517 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac0e16a8-b89f-4a98-91c8-02c579afa7b8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:57.180953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac0e16a8-b89f-4a98-91c8-02c579afa7b8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:57.181793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac0e16a8-b89f-4a98-91c8-02c579afa7b8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:57.182248 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac0e16a8-b89f-4a98-91c8-02c579afa7b8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:57.182672 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac0e16a8-b89f-4a98-91c8-02c579afa7b8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:57.215079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ac0e16a8-b89f-4a98-91c8-02c579afa7b8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048392 Aug 28 01:00:57.215661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 310/616] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:00:57 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:00:57.930776 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-477cb649-3692-49cf-ac99-d8fffe83be44 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:57.931825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-477cb649-3692-49cf-ac99-d8fffe83be44 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:57.942402 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-477cb649-3692-49cf-ac99-d8fffe83be44 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:57.943255 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-477cb649-3692-49cf-ac99-d8fffe83be44 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:57.944022 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-477cb649-3692-49cf-ac99-d8fffe83be44 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:57.945483 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-477cb649-3692-49cf-ac99-d8fffe83be44 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:57.946166 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-477cb649-3692-49cf-ac99-d8fffe83be44 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:57.946818 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-477cb649-3692-49cf-ac99-d8fffe83be44 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:58.106513 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5e3548d4-a2db-4bb3-85ff-b4ca0ff6c629 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:58.108685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5e3548d4-a2db-4bb3-85ff-b4ca0ff6c629 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:58.115284 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e3548d4-a2db-4bb3-85ff-b4ca0ff6c629 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:58.116051 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e3548d4-a2db-4bb3-85ff-b4ca0ff6c629 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:58.116747 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e3548d4-a2db-4bb3-85ff-b4ca0ff6c629 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:58.117813 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e3548d4-a2db-4bb3-85ff-b4ca0ff6c629 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:58.118434 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e3548d4-a2db-4bb3-85ff-b4ca0ff6c629 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:58.119073 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e3548d4-a2db-4bb3-85ff-b4ca0ff6c629 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:58.132117 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-477cb649-3692-49cf-ac99-d8fffe83be44 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:58.143309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-477cb649-3692-49cf-ac99-d8fffe83be44 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.219605 Aug 28 01:00:58.143906 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 307/617] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:57 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:00:58.154468 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5e3548d4-a2db-4bb3-85ff-b4ca0ff6c629 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051978 Aug 28 01:00:58.155066 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 311/618] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:00:58 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:00:58.228279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7085987c-790f-47de-83f0-176994022a57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:58.230458 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7085987c-790f-47de-83f0-176994022a57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:58.237683 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7085987c-790f-47de-83f0-176994022a57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:58.238311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7085987c-790f-47de-83f0-176994022a57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:58.238834 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7085987c-790f-47de-83f0-176994022a57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:58.239637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7085987c-790f-47de-83f0-176994022a57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:58.240215 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7085987c-790f-47de-83f0-176994022a57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:58.240724 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7085987c-790f-47de-83f0-176994022a57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:58.276137 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7085987c-790f-47de-83f0-176994022a57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051877 Aug 28 01:00:58.276751 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 308/619] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:00:58 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:00:59.165779 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-005172f7-8262-4e5d-b9a6-771abc817038 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:59.166630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-005172f7-8262-4e5d-b9a6-771abc817038 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:59.169267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-af5948b7-c982-4eb8-9a46-26ae38363ea9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:59.172553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-af5948b7-c982-4eb8-9a46-26ae38363ea9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:59.173788 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-005172f7-8262-4e5d-b9a6-771abc817038 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:59.174229 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-005172f7-8262-4e5d-b9a6-771abc817038 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:59.174642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-005172f7-8262-4e5d-b9a6-771abc817038 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:59.175389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-005172f7-8262-4e5d-b9a6-771abc817038 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:59.175815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-005172f7-8262-4e5d-b9a6-771abc817038 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:59.176280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-005172f7-8262-4e5d-b9a6-771abc817038 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:59.182543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af5948b7-c982-4eb8-9a46-26ae38363ea9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:59.183305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af5948b7-c982-4eb8-9a46-26ae38363ea9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:59.183903 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af5948b7-c982-4eb8-9a46-26ae38363ea9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:59.184871 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af5948b7-c982-4eb8-9a46-26ae38363ea9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:59.185454 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af5948b7-c982-4eb8-9a46-26ae38363ea9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:59.185898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af5948b7-c982-4eb8-9a46-26ae38363ea9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:59.222745 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-af5948b7-c982-4eb8-9a46-26ae38363ea9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.059345 Aug 28 01:00:59.223317 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 309/620] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:00:59 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 28 01:00:59.288334 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-30c055db-05b6-4d5c-9529-279aa86d148f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:00:59.290692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-30c055db-05b6-4d5c-9529-279aa86d148f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:00:59.297143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-30c055db-05b6-4d5c-9529-279aa86d148f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:59.297641 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-30c055db-05b6-4d5c-9529-279aa86d148f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:59.298076 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-30c055db-05b6-4d5c-9529-279aa86d148f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:59.298799 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-30c055db-05b6-4d5c-9529-279aa86d148f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:00:59.299257 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-30c055db-05b6-4d5c-9529-279aa86d148f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:00:59.299679 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-30c055db-05b6-4d5c-9529-279aa86d148f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:00:59.337212 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-30c055db-05b6-4d5c-9529-279aa86d148f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052963 Aug 28 01:00:59.338791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 310/621] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:00:59 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:00:59.340408 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-005172f7-8262-4e5d-b9a6-771abc817038 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:00:59.349460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-005172f7-8262-4e5d-b9a6-771abc817038 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.187434 Aug 28 01:00:59.350053 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 312/622] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:00:59 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:00.237625 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-691d03fc-c3c6-4ad6-bd2f-2c6b5cb8d481 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:00.240032 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-691d03fc-c3c6-4ad6-bd2f-2c6b5cb8d481 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:00.247503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-691d03fc-c3c6-4ad6-bd2f-2c6b5cb8d481 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:00.248031 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-691d03fc-c3c6-4ad6-bd2f-2c6b5cb8d481 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:00.248464 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-691d03fc-c3c6-4ad6-bd2f-2c6b5cb8d481 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:00.249229 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-691d03fc-c3c6-4ad6-bd2f-2c6b5cb8d481 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:00.249681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-691d03fc-c3c6-4ad6-bd2f-2c6b5cb8d481 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:00.250107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-691d03fc-c3c6-4ad6-bd2f-2c6b5cb8d481 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:00.285726 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-691d03fc-c3c6-4ad6-bd2f-2c6b5cb8d481 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052243 Aug 28 01:01:00.286341 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 311/623] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:00 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:00.353623 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e87928b4-6118-4651-a75c-36e527362526 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:00.355826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e87928b4-6118-4651-a75c-36e527362526 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:00.363431 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e87928b4-6118-4651-a75c-36e527362526 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:00.363920 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e87928b4-6118-4651-a75c-36e527362526 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:00.364352 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e87928b4-6118-4651-a75c-36e527362526 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:00.365110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e87928b4-6118-4651-a75c-36e527362526 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:00.365596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e87928b4-6118-4651-a75c-36e527362526 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:00.366010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e87928b4-6118-4651-a75c-36e527362526 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:00.372128 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-eb13d43c-ea20-4f7a-8631-df2ae8fbb8d8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:00.372840 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-eb13d43c-ea20-4f7a-8631-df2ae8fbb8d8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:00.379077 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb13d43c-ea20-4f7a-8631-df2ae8fbb8d8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:00.379572 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb13d43c-ea20-4f7a-8631-df2ae8fbb8d8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:00.380005 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb13d43c-ea20-4f7a-8631-df2ae8fbb8d8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:00.380825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb13d43c-ea20-4f7a-8631-df2ae8fbb8d8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:00.381312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb13d43c-ea20-4f7a-8631-df2ae8fbb8d8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:00.381739 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb13d43c-ea20-4f7a-8631-df2ae8fbb8d8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:00.403845 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e87928b4-6118-4651-a75c-36e527362526 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054332 Aug 28 01:01:00.404694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 313/624] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:00 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:00.525514 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-eb13d43c-ea20-4f7a-8631-df2ae8fbb8d8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:00.541015 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-eb13d43c-ea20-4f7a-8631-df2ae8fbb8d8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.172344 Aug 28 01:01:00.542087 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 312/625] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:00 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:01.301159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8f1cbd72-fba7-4a6d-90b2-0d38f39dcef6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:01.303720 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8f1cbd72-fba7-4a6d-90b2-0d38f39dcef6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:01.310908 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f1cbd72-fba7-4a6d-90b2-0d38f39dcef6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:01.311407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f1cbd72-fba7-4a6d-90b2-0d38f39dcef6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:01.311847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f1cbd72-fba7-4a6d-90b2-0d38f39dcef6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:01.312572 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f1cbd72-fba7-4a6d-90b2-0d38f39dcef6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:01.313064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f1cbd72-fba7-4a6d-90b2-0d38f39dcef6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:01.313495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f1cbd72-fba7-4a6d-90b2-0d38f39dcef6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:01.353438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8f1cbd72-fba7-4a6d-90b2-0d38f39dcef6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.057442 Aug 28 01:01:01.354048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 314/626] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:01 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:01.419807 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8beb5fa9-1843-4f18-be2a-2a48f44f7228 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:01.422404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8beb5fa9-1843-4f18-be2a-2a48f44f7228 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:01.430110 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8beb5fa9-1843-4f18-be2a-2a48f44f7228 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:01.430602 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8beb5fa9-1843-4f18-be2a-2a48f44f7228 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:01.431030 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8beb5fa9-1843-4f18-be2a-2a48f44f7228 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:01.431742 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8beb5fa9-1843-4f18-be2a-2a48f44f7228 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:01.432190 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8beb5fa9-1843-4f18-be2a-2a48f44f7228 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:01.432605 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8beb5fa9-1843-4f18-be2a-2a48f44f7228 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:01.471790 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8beb5fa9-1843-4f18-be2a-2a48f44f7228 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059062 Aug 28 01:01:01.472355 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 313/627] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:01 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:01.564566 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f85eac69-16b5-4b82-a690-f9a5247775db tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:01.565085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f85eac69-16b5-4b82-a690-f9a5247775db tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:01.572557 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f85eac69-16b5-4b82-a690-f9a5247775db tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:01.573062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f85eac69-16b5-4b82-a690-f9a5247775db tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:01.573539 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f85eac69-16b5-4b82-a690-f9a5247775db tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:01.574235 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f85eac69-16b5-4b82-a690-f9a5247775db tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:01.574716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f85eac69-16b5-4b82-a690-f9a5247775db tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:01.575143 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f85eac69-16b5-4b82-a690-f9a5247775db tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:01.765977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f85eac69-16b5-4b82-a690-f9a5247775db tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:01.775166 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f85eac69-16b5-4b82-a690-f9a5247775db tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.214601 Aug 28 01:01:01.775815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 315/628] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:01 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:02.366931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-83a5bcd1-3f93-472a-85ce-2a042572f48b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:02.369082 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-83a5bcd1-3f93-472a-85ce-2a042572f48b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:02.375816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-83a5bcd1-3f93-472a-85ce-2a042572f48b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:02.376306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-83a5bcd1-3f93-472a-85ce-2a042572f48b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:02.376752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-83a5bcd1-3f93-472a-85ce-2a042572f48b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:02.377521 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-83a5bcd1-3f93-472a-85ce-2a042572f48b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:02.377968 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-83a5bcd1-3f93-472a-85ce-2a042572f48b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:02.378399 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-83a5bcd1-3f93-472a-85ce-2a042572f48b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:02.421794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-83a5bcd1-3f93-472a-85ce-2a042572f48b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.059027 Aug 28 01:01:02.422802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 314/629] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:02 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:02.484259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2dc7a2ba-2c15-4788-84c9-55cf358f129f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:02.486513 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2dc7a2ba-2c15-4788-84c9-55cf358f129f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:02.493296 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2dc7a2ba-2c15-4788-84c9-55cf358f129f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:02.493783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2dc7a2ba-2c15-4788-84c9-55cf358f129f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:02.494193 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2dc7a2ba-2c15-4788-84c9-55cf358f129f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:02.494934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2dc7a2ba-2c15-4788-84c9-55cf358f129f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:02.495350 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2dc7a2ba-2c15-4788-84c9-55cf358f129f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:02.495806 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2dc7a2ba-2c15-4788-84c9-55cf358f129f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:02.525661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:01:02.531358 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2dc7a2ba-2c15-4788-84c9-55cf358f129f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051092 Aug 28 01:01:02.531951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 316/630] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:02 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:02.806067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0e8fce15-6b3f-4d84-b382-c7f4c15a836c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:02.806840 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0e8fce15-6b3f-4d84-b382-c7f4c15a836c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:02.815175 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e8fce15-6b3f-4d84-b382-c7f4c15a836c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:02.815726 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e8fce15-6b3f-4d84-b382-c7f4c15a836c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:02.816203 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e8fce15-6b3f-4d84-b382-c7f4c15a836c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:02.817010 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e8fce15-6b3f-4d84-b382-c7f4c15a836c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:02.817516 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e8fce15-6b3f-4d84-b382-c7f4c15a836c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:02.818089 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e8fce15-6b3f-4d84-b382-c7f4c15a836c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:02.972318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0e8fce15-6b3f-4d84-b382-c7f4c15a836c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:02.982091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0e8fce15-6b3f-4d84-b382-c7f4c15a836c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.180418 Aug 28 01:01:02.982691 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 315/631] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:02 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:03.436488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-000427c0-831a-4621-8661-05e4f1cfa767 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:03.438779 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-000427c0-831a-4621-8661-05e4f1cfa767 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:03.446948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-000427c0-831a-4621-8661-05e4f1cfa767 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:03.447447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-000427c0-831a-4621-8661-05e4f1cfa767 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:03.447888 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-000427c0-831a-4621-8661-05e4f1cfa767 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:03.448614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-000427c0-831a-4621-8661-05e4f1cfa767 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:03.449101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-000427c0-831a-4621-8661-05e4f1cfa767 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:03.449530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-000427c0-831a-4621-8661-05e4f1cfa767 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:03.485737 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-000427c0-831a-4621-8661-05e4f1cfa767 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053862 Aug 28 01:01:03.486801 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 317/632] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:03 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:03.544713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-65dfb6f4-1fda-4e8e-bda9-75e66fcdc44f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:03.547330 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-65dfb6f4-1fda-4e8e-bda9-75e66fcdc44f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:03.554674 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-65dfb6f4-1fda-4e8e-bda9-75e66fcdc44f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:03.555318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-65dfb6f4-1fda-4e8e-bda9-75e66fcdc44f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:03.555894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-65dfb6f4-1fda-4e8e-bda9-75e66fcdc44f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:03.556767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-65dfb6f4-1fda-4e8e-bda9-75e66fcdc44f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:03.557348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-65dfb6f4-1fda-4e8e-bda9-75e66fcdc44f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:03.557948 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-65dfb6f4-1fda-4e8e-bda9-75e66fcdc44f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:03.577281 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:01:03.593399 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-65dfb6f4-1fda-4e8e-bda9-75e66fcdc44f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053022 Aug 28 01:01:03.593986 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 316/633] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:03 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:04.004241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-95056cf0-9a10-41d0-af11-157151e9e005 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:04.004833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-95056cf0-9a10-41d0-af11-157151e9e005 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:04.007590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=71,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:01:04.016153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-95056cf0-9a10-41d0-af11-157151e9e005 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:04.016745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-95056cf0-9a10-41d0-af11-157151e9e005 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:04.017236 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-95056cf0-9a10-41d0-af11-157151e9e005 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:04.018212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-95056cf0-9a10-41d0-af11-157151e9e005 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:04.018831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-95056cf0-9a10-41d0-af11-157151e9e005 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:04.019298 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-95056cf0-9a10-41d0-af11-157151e9e005 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:04.201309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-95056cf0-9a10-41d0-af11-157151e9e005 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:04.210694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-95056cf0-9a10-41d0-af11-157151e9e005 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1715 microversion: 2.1 time: 0.210531 Aug 28 01:01:04.211403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 318/634] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:03 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1715 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:04.286041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-eda47ee5-41d2-422d-b95b-c87026af8657 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:04.286673 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-eda47ee5-41d2-422d-b95b-c87026af8657 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:04.289647 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=71 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:01:04.295982 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eda47ee5-41d2-422d-b95b-c87026af8657 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:04.296478 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eda47ee5-41d2-422d-b95b-c87026af8657 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:04.297036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eda47ee5-41d2-422d-b95b-c87026af8657 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:04.297767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eda47ee5-41d2-422d-b95b-c87026af8657 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:04.298216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eda47ee5-41d2-422d-b95b-c87026af8657 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:04.298648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eda47ee5-41d2-422d-b95b-c87026af8657 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:04.389904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-eda47ee5-41d2-422d-b95b-c87026af8657 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] HTTP exception thrown: Cannot 'detach_volume' instance 03969ac5-934e-4201-8087-0927eae2bbd2 while it is in vm_state rescued Aug 28 01:01:04.391618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-eda47ee5-41d2-422d-b95b-c87026af8657 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Request method failure captured: Aug 28 01:01:04.391618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c HTTP/1.1 Aug 28 01:01:04.391618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:01:04.391618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:01:04.391618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:01:04.391618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:01:04.391618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:01:04.391618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:01:04.391618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:01:04.391618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:01:04.391618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCrzWcuiPK_0ynCrragXxXd_fRw7lHCmbNVMAqr4NhyopxGAZIiQrt0m3y3NawnXLWqyOVM4YbBc-yLRrJ4cXICU0MsT01bsu6VzxaCxGd6JRCChv8LlCV7kFwwyy7Ffwd9JpYuFLguuT-xhEcVlfMIZxgSYB22XTBbtvAeHM_AtK1Xg Aug 28 01:01:04.391618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:01:04.391618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:01:04.391618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:01:04.391618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:01:04.391618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:01:04.391618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:01:04.391618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 01:01:04.391618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:01:04.391618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:01:04.391618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:01:04.394526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_6f3acee4af62483ab1f2cbf3a47b09e1", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/6f3acee4af62483ab1f2cbf3a47b09e1"}]}] Aug 28 01:01:04.394526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:01:04.394526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 01:01:04.394526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:01:04.394526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 01:01:04.394526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:01:04.394526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:01:04.394526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: 1ab2fcb3759045c7a076e5c8222a421d Aug 28 01:01:04.394526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 01:01:04.394526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:01:04.394526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Cannot 'detach_volume' instance 03969ac5-934e-4201-8087-0927eae2bbd2 while it is in vm_state rescued Aug 28 01:01:04.395896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'server_id': '03969ac5-934e-4201-8087-0927eae2bbd2', 'id': '6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c'} Aug 28 01:01:04.395896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:01:04.395896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:01:04.395896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 28 01:01:04.395896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 28 01:01:04.395896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Aug 28 01:01:04.395896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:01:04.395896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Aug 28 01:01:04.395896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Aug 28 01:01:04.395896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 03969ac5-934e-4201-8087-0927eae2bbd2 in vm_state rescued. Cannot detach_volume while the instance is in this state. Aug 28 01:01:04.395896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:01:04.395896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:01:04.395896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:01:04.395896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:01:04.395896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:01:04.395896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:01:04.395896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:01:04.395896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:01:04.395896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:01:04.395896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:01:04.398331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Aug 28 01:01:04.398331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Aug 28 01:01:04.398331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Aug 28 01:01:04.398331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Aug 28 01:01:04.398331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 03969ac5-934e-4201-8087-0927eae2bbd2 while it is in vm_state rescued Aug 28 01:01:04.398331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:01:04.398331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:01:04.398331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:01:04.398331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:01:04.398331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:01:04.398331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:01:04.398331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:01:04.398331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:01:04.398331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 03969ac5-934e-4201-8087-0927eae2bbd2 while it is in vm_state rescued Aug 28 01:01:04.398331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:01:04.398331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-eda47ee5-41d2-422d-b95b-c87026af8657 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Returning 409 to user: Cannot 'detach_volume' instance 03969ac5-934e-4201-8087-0927eae2bbd2 while it is in vm_state rescued {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:01:04.400594 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-eda47ee5-41d2-422d-b95b-c87026af8657 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c" status: 409 len: 152 microversion: 2.1 time: 0.112319 Aug 28 01:01:04.400594 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 317/635] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Aug 28 01:01:04 2022] DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c => generated 152 bytes in 114 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Aug 28 01:01:04.409041 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-234e9c06-afde-405f-86c0-0ec0e1ffd0a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:01:04.409998 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-234e9c06-afde-405f-86c0-0ec0e1ffd0a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:04.421274 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-234e9c06-afde-405f-86c0-0ec0e1ffd0a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:04.421995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-234e9c06-afde-405f-86c0-0ec0e1ffd0a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:04.422766 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-234e9c06-afde-405f-86c0-0ec0e1ffd0a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:04.424037 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-234e9c06-afde-405f-86c0-0ec0e1ffd0a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:04.424735 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-234e9c06-afde-405f-86c0-0ec0e1ffd0a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:04.425538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-234e9c06-afde-405f-86c0-0ec0e1ffd0a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:04.504334 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a0146907-215f-44c7-a729-6af45e84965a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:04.508522 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a0146907-215f-44c7-a729-6af45e84965a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:04.522110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-234e9c06-afde-405f-86c0-0ec0e1ffd0a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "POST /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/action" status: 202 len: 0 microversion: 2.1 time: 0.120951 Aug 28 01:01:04.524207 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 319/636] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Aug 28 01:01:04 2022] POST /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/action => generated 0 bytes in 124 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 01:01:04.527526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a0146907-215f-44c7-a729-6af45e84965a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:04.528403 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a0146907-215f-44c7-a729-6af45e84965a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:04.529599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a0146907-215f-44c7-a729-6af45e84965a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:04.530712 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a0146907-215f-44c7-a729-6af45e84965a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:04.531906 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a0146907-215f-44c7-a729-6af45e84965a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:04.532466 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a0146907-215f-44c7-a729-6af45e84965a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:04.535799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0fc6c764-11af-4a4e-9b1d-f6a872244e09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:04.536506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0fc6c764-11af-4a4e-9b1d-f6a872244e09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:04.543934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0fc6c764-11af-4a4e-9b1d-f6a872244e09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:04.544584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0fc6c764-11af-4a4e-9b1d-f6a872244e09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:04.545305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0fc6c764-11af-4a4e-9b1d-f6a872244e09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:04.546176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0fc6c764-11af-4a4e-9b1d-f6a872244e09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:04.546568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0fc6c764-11af-4a4e-9b1d-f6a872244e09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:04.546996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0fc6c764-11af-4a4e-9b1d-f6a872244e09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:04.596289 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a0146907-215f-44c7-a729-6af45e84965a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.100585 Aug 28 01:01:04.601991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 318/637] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:04 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:04.610995 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a809d250-ca5d-42d8-8855-e4c330150fba tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:04.616649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a809d250-ca5d-42d8-8855-e4c330150fba tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:04.625634 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a809d250-ca5d-42d8-8855-e4c330150fba tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:04.626515 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a809d250-ca5d-42d8-8855-e4c330150fba tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:04.627290 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a809d250-ca5d-42d8-8855-e4c330150fba tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:04.628563 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a809d250-ca5d-42d8-8855-e4c330150fba tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:04.629382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a809d250-ca5d-42d8-8855-e4c330150fba tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:04.630177 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a809d250-ca5d-42d8-8855-e4c330150fba tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:04.690406 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a809d250-ca5d-42d8-8855-e4c330150fba tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.086491 Aug 28 01:01:04.691473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 319/638] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:04 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:04.902119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0fc6c764-11af-4a4e-9b1d-f6a872244e09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:04.912472 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0fc6c764-11af-4a4e-9b1d-f6a872244e09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1723 microversion: 2.1 time: 0.381895 Aug 28 01:01:04.929011 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 320/639] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:04 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1723 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:05.617310 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b1a0f22f-1dda-481e-9902-27485e059b9d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:05.621166 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b1a0f22f-1dda-481e-9902-27485e059b9d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:05.635305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b1a0f22f-1dda-481e-9902-27485e059b9d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:05.636592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b1a0f22f-1dda-481e-9902-27485e059b9d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:05.637654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b1a0f22f-1dda-481e-9902-27485e059b9d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:05.639297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b1a0f22f-1dda-481e-9902-27485e059b9d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:05.639774 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b1a0f22f-1dda-481e-9902-27485e059b9d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:05.640506 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b1a0f22f-1dda-481e-9902-27485e059b9d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:05.699540 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b1a0f22f-1dda-481e-9902-27485e059b9d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.088402 Aug 28 01:01:05.700938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 320/640] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:05 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:05.710659 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b5cde638-8281-477b-84c3-6e5e91b6da82 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:05.713711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b5cde638-8281-477b-84c3-6e5e91b6da82 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:05.722386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5cde638-8281-477b-84c3-6e5e91b6da82 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:05.722879 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5cde638-8281-477b-84c3-6e5e91b6da82 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:05.723319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5cde638-8281-477b-84c3-6e5e91b6da82 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:05.724185 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5cde638-8281-477b-84c3-6e5e91b6da82 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:05.724576 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5cde638-8281-477b-84c3-6e5e91b6da82 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:05.725028 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5cde638-8281-477b-84c3-6e5e91b6da82 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:05.762336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b5cde638-8281-477b-84c3-6e5e91b6da82 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055626 Aug 28 01:01:05.763171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 321/641] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:05 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:05.949194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a3f90de4-9128-4756-9b0d-5fc7ad1e0394 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:05.950100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a3f90de4-9128-4756-9b0d-5fc7ad1e0394 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:05.957440 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3f90de4-9128-4756-9b0d-5fc7ad1e0394 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:05.958139 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3f90de4-9128-4756-9b0d-5fc7ad1e0394 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:05.958670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3f90de4-9128-4756-9b0d-5fc7ad1e0394 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:05.959544 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3f90de4-9128-4756-9b0d-5fc7ad1e0394 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:05.960133 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3f90de4-9128-4756-9b0d-5fc7ad1e0394 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:05.960639 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3f90de4-9128-4756-9b0d-5fc7ad1e0394 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:06.148877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a3f90de4-9128-4756-9b0d-5fc7ad1e0394 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:06.158825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a3f90de4-9128-4756-9b0d-5fc7ad1e0394 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1723 microversion: 2.1 time: 0.214468 Aug 28 01:01:06.159467 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 321/642] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:05 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1723 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:06.714009 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fd4d4125-75f4-4df1-b209-20f59c1dc1db tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:06.716123 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fd4d4125-75f4-4df1-b209-20f59c1dc1db tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:06.723399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd4d4125-75f4-4df1-b209-20f59c1dc1db tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:06.723877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd4d4125-75f4-4df1-b209-20f59c1dc1db tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:06.724320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd4d4125-75f4-4df1-b209-20f59c1dc1db tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:06.725083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd4d4125-75f4-4df1-b209-20f59c1dc1db tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:06.725537 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd4d4125-75f4-4df1-b209-20f59c1dc1db tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:06.725957 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd4d4125-75f4-4df1-b209-20f59c1dc1db tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:06.767160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fd4d4125-75f4-4df1-b209-20f59c1dc1db tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.057355 Aug 28 01:01:06.769033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 322/643] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:06 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:06.774957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6dfa44eb-07b0-451c-9449-780395a71042 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:06.777201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6dfa44eb-07b0-451c-9449-780395a71042 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:06.784755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6dfa44eb-07b0-451c-9449-780395a71042 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:06.785278 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6dfa44eb-07b0-451c-9449-780395a71042 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:06.785713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6dfa44eb-07b0-451c-9449-780395a71042 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:06.786428 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6dfa44eb-07b0-451c-9449-780395a71042 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:06.786882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6dfa44eb-07b0-451c-9449-780395a71042 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:06.787289 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6dfa44eb-07b0-451c-9449-780395a71042 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:06.821588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6dfa44eb-07b0-451c-9449-780395a71042 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050766 Aug 28 01:01:06.822177 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 322/644] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:06 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:07.181108 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f382dd2d-6a8b-444b-a06a-8589f9eeb355 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:07.181731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f382dd2d-6a8b-444b-a06a-8589f9eeb355 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:07.188736 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f382dd2d-6a8b-444b-a06a-8589f9eeb355 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:07.189276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f382dd2d-6a8b-444b-a06a-8589f9eeb355 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:07.189726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f382dd2d-6a8b-444b-a06a-8589f9eeb355 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:07.190490 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f382dd2d-6a8b-444b-a06a-8589f9eeb355 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:07.190953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f382dd2d-6a8b-444b-a06a-8589f9eeb355 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:07.191380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f382dd2d-6a8b-444b-a06a-8589f9eeb355 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:07.340863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f382dd2d-6a8b-444b-a06a-8589f9eeb355 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:07.352496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f382dd2d-6a8b-444b-a06a-8589f9eeb355 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1723 microversion: 2.1 time: 0.175386 Aug 28 01:01:07.353326 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 323/645] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:07 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1723 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:07.784692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-48f5530d-1140-442e-8213-93d9d7eb23a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:07.787327 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-48f5530d-1140-442e-8213-93d9d7eb23a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:07.794808 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48f5530d-1140-442e-8213-93d9d7eb23a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:07.795298 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48f5530d-1140-442e-8213-93d9d7eb23a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:07.795721 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48f5530d-1140-442e-8213-93d9d7eb23a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:07.796428 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48f5530d-1140-442e-8213-93d9d7eb23a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:07.796874 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48f5530d-1140-442e-8213-93d9d7eb23a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:07.797329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48f5530d-1140-442e-8213-93d9d7eb23a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:07.833694 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-48f5530d-1140-442e-8213-93d9d7eb23a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053452 Aug 28 01:01:07.834456 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 323/646] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:07 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:07.839060 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-079d83cd-2bc5-40a8-9c6c-fb900884ded2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:07.842857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-079d83cd-2bc5-40a8-9c6c-fb900884ded2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:07.854732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-079d83cd-2bc5-40a8-9c6c-fb900884ded2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:07.855580 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-079d83cd-2bc5-40a8-9c6c-fb900884ded2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:07.856332 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-079d83cd-2bc5-40a8-9c6c-fb900884ded2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:07.857645 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-079d83cd-2bc5-40a8-9c6c-fb900884ded2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:07.858458 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-079d83cd-2bc5-40a8-9c6c-fb900884ded2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:07.859192 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-079d83cd-2bc5-40a8-9c6c-fb900884ded2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:07.896530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-079d83cd-2bc5-40a8-9c6c-fb900884ded2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064770 Aug 28 01:01:07.897143 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 324/647] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:07 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:01:08.375283 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f9c23229-a42e-43fa-b72f-67fb374ce264 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:08.375802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f9c23229-a42e-43fa-b72f-67fb374ce264 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:08.382608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9c23229-a42e-43fa-b72f-67fb374ce264 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:08.383104 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9c23229-a42e-43fa-b72f-67fb374ce264 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:08.383545 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9c23229-a42e-43fa-b72f-67fb374ce264 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:08.384267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9c23229-a42e-43fa-b72f-67fb374ce264 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:08.384718 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9c23229-a42e-43fa-b72f-67fb374ce264 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:08.385182 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9c23229-a42e-43fa-b72f-67fb374ce264 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:08.539277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f9c23229-a42e-43fa-b72f-67fb374ce264 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:08.552580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f9c23229-a42e-43fa-b72f-67fb374ce264 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1723 microversion: 2.1 time: 0.181162 Aug 28 01:01:08.553618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 324/648] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:08 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1723 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:08.848428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e1f96304-178f-4977-993b-a5d3c969163f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:08.850965 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e1f96304-178f-4977-993b-a5d3c969163f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:08.858650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1f96304-178f-4977-993b-a5d3c969163f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:08.859191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1f96304-178f-4977-993b-a5d3c969163f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:08.859717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1f96304-178f-4977-993b-a5d3c969163f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:08.860432 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1f96304-178f-4977-993b-a5d3c969163f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:08.860882 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1f96304-178f-4977-993b-a5d3c969163f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:08.861407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1f96304-178f-4977-993b-a5d3c969163f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:08.897634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e1f96304-178f-4977-993b-a5d3c969163f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053785 Aug 28 01:01:08.898126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 325/649] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:08 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:08.912938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-01b4ab90-d46f-4a6a-86fb-a4e8bcc57c8f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:08.915214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-01b4ab90-d46f-4a6a-86fb-a4e8bcc57c8f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:08.922703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-01b4ab90-d46f-4a6a-86fb-a4e8bcc57c8f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:08.923253 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-01b4ab90-d46f-4a6a-86fb-a4e8bcc57c8f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:08.923719 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-01b4ab90-d46f-4a6a-86fb-a4e8bcc57c8f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:08.924478 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-01b4ab90-d46f-4a6a-86fb-a4e8bcc57c8f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:08.924993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-01b4ab90-d46f-4a6a-86fb-a4e8bcc57c8f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:08.925594 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-01b4ab90-d46f-4a6a-86fb-a4e8bcc57c8f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:08.962840 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-01b4ab90-d46f-4a6a-86fb-a4e8bcc57c8f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053940 Aug 28 01:01:08.963402 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 325/650] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:08 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:09.575584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-673b5a62-3e3f-429c-8255-ba0c842ae8ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:09.576214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-673b5a62-3e3f-429c-8255-ba0c842ae8ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:09.582805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-673b5a62-3e3f-429c-8255-ba0c842ae8ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:09.583417 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-673b5a62-3e3f-429c-8255-ba0c842ae8ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:09.583940 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-673b5a62-3e3f-429c-8255-ba0c842ae8ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:09.584714 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-673b5a62-3e3f-429c-8255-ba0c842ae8ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:09.585243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-673b5a62-3e3f-429c-8255-ba0c842ae8ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:09.585721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-673b5a62-3e3f-429c-8255-ba0c842ae8ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:09.760034 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-673b5a62-3e3f-429c-8255-ba0c842ae8ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:09.772150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-673b5a62-3e3f-429c-8255-ba0c842ae8ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1729 microversion: 2.1 time: 0.200094 Aug 28 01:01:09.773379 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 326/651] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:09 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1729 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:09.780193 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7678d73a-944c-4da7-9ec1-2690d366c12a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:09.780859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7678d73a-944c-4da7-9ec1-2690d366c12a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:09.788692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7678d73a-944c-4da7-9ec1-2690d366c12a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:09.789208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7678d73a-944c-4da7-9ec1-2690d366c12a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:09.789702 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7678d73a-944c-4da7-9ec1-2690d366c12a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:09.790439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7678d73a-944c-4da7-9ec1-2690d366c12a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:09.790896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7678d73a-944c-4da7-9ec1-2690d366c12a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:09.791318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7678d73a-944c-4da7-9ec1-2690d366c12a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:09.808744 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-445dbf82-8eb3-4aa7-b015-9d3536a9ed12 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:09.809408 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-445dbf82-8eb3-4aa7-b015-9d3536a9ed12 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:09.816077 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-445dbf82-8eb3-4aa7-b015-9d3536a9ed12 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:09.816566 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-445dbf82-8eb3-4aa7-b015-9d3536a9ed12 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:09.817019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-445dbf82-8eb3-4aa7-b015-9d3536a9ed12 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:09.817730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-445dbf82-8eb3-4aa7-b015-9d3536a9ed12 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:09.818217 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-445dbf82-8eb3-4aa7-b015-9d3536a9ed12 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:09.818632 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-445dbf82-8eb3-4aa7-b015-9d3536a9ed12 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:09.829329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7678d73a-944c-4da7-9ec1-2690d366c12a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Going to try to terminate instance {{(pid=120524) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 28 01:01:09.902563 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-445dbf82-8eb3-4aa7-b015-9d3536a9ed12 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.098305 Aug 28 01:01:09.905887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 327/652] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Aug 28 01:01:09 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-interface => generated 285 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:01:09.915493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8b08192c-fcf5-48c6-aaa9-a536f7065d08 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:09.918860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8b08192c-fcf5-48c6-aaa9-a536f7065d08 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:09.927603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b08192c-fcf5-48c6-aaa9-a536f7065d08 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:09.928493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b08192c-fcf5-48c6-aaa9-a536f7065d08 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:09.929256 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b08192c-fcf5-48c6-aaa9-a536f7065d08 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:09.930386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b08192c-fcf5-48c6-aaa9-a536f7065d08 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:09.930953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b08192c-fcf5-48c6-aaa9-a536f7065d08 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:09.931608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b08192c-fcf5-48c6-aaa9-a536f7065d08 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:09.986655 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7678d73a-944c-4da7-9ec1-2690d366c12a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "DELETE /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 204 len: 0 microversion: 2.1 time: 0.210316 Aug 28 01:01:09.987828 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 326/653] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 01:01:09 2022] DELETE /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 28 01:01:09.993022 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8b08192c-fcf5-48c6-aaa9-a536f7065d08 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.085158 Aug 28 01:01:09.993942 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 328/654] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:09 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:09.996821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8e6ccb31-7762-49d0-ab6f-8937140df939 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:10.001347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8e6ccb31-7762-49d0-ab6f-8937140df939 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:10.008707 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-70e71176-1d56-4389-a2c2-a2702bb7d2d7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:10.009686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-70e71176-1d56-4389-a2c2-a2702bb7d2d7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:10.011385 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e6ccb31-7762-49d0-ab6f-8937140df939 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:10.012316 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e6ccb31-7762-49d0-ab6f-8937140df939 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:10.012828 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e6ccb31-7762-49d0-ab6f-8937140df939 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:10.014080 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e6ccb31-7762-49d0-ab6f-8937140df939 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:10.014862 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e6ccb31-7762-49d0-ab6f-8937140df939 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:10.015689 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e6ccb31-7762-49d0-ab6f-8937140df939 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:10.015951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70e71176-1d56-4389-a2c2-a2702bb7d2d7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:10.016445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70e71176-1d56-4389-a2c2-a2702bb7d2d7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:10.016909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70e71176-1d56-4389-a2c2-a2702bb7d2d7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:10.017643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70e71176-1d56-4389-a2c2-a2702bb7d2d7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:10.018088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70e71176-1d56-4389-a2c2-a2702bb7d2d7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:10.018509 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70e71176-1d56-4389-a2c2-a2702bb7d2d7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:10.070003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8e6ccb31-7762-49d0-ab6f-8937140df939 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.079609 Aug 28 01:01:10.071586 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 327/655] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:09 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:10.275114 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-70e71176-1d56-4389-a2c2-a2702bb7d2d7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:10.286198 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-70e71176-1d56-4389-a2c2-a2702bb7d2d7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1576 microversion: 2.1 time: 0.283944 Aug 28 01:01:10.287175 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 329/656] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:10 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1576 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:11.008886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a6cb2cab-a00b-4dcf-acd1-58898ee81375 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:11.011570 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a6cb2cab-a00b-4dcf-acd1-58898ee81375 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:11.020168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a6cb2cab-a00b-4dcf-acd1-58898ee81375 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:11.020839 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a6cb2cab-a00b-4dcf-acd1-58898ee81375 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:11.021440 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a6cb2cab-a00b-4dcf-acd1-58898ee81375 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:11.022779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a6cb2cab-a00b-4dcf-acd1-58898ee81375 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:11.023414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a6cb2cab-a00b-4dcf-acd1-58898ee81375 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:11.023983 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a6cb2cab-a00b-4dcf-acd1-58898ee81375 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:11.076410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a6cb2cab-a00b-4dcf-acd1-58898ee81375 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.072246 Aug 28 01:01:11.077489 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 328/657] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:11 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:11.089808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e297c993-807b-47e1-b916-3ae2c1372938 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:11.092626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e297c993-807b-47e1-b916-3ae2c1372938 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:11.100955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e297c993-807b-47e1-b916-3ae2c1372938 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:11.101748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e297c993-807b-47e1-b916-3ae2c1372938 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:11.102267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e297c993-807b-47e1-b916-3ae2c1372938 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:11.103062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e297c993-807b-47e1-b916-3ae2c1372938 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:11.103622 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e297c993-807b-47e1-b916-3ae2c1372938 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:11.104566 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e297c993-807b-47e1-b916-3ae2c1372938 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:11.140530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e297c993-807b-47e1-b916-3ae2c1372938 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055362 Aug 28 01:01:11.141156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 330/658] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:11 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:11.316394 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-421980f5-8390-4901-9151-b72f3dd82e8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:11.318125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-421980f5-8390-4901-9151-b72f3dd82e8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:11.328455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-6731c5a0-3e13-4823-a22e-fef5acb84c7b req-e95a9409-0a3a-4dcb-a114-21c8b386dc1b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "03969ac5-934e-4201-8087-0927eae2bbd2", "tag": "b9bc50e7-87a3-480a-8c62-87fe75411e15"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:01:11.330674 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-421980f5-8390-4901-9151-b72f3dd82e8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:11.331970 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-421980f5-8390-4901-9151-b72f3dd82e8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:11.333126 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-421980f5-8390-4901-9151-b72f3dd82e8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:11.334902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-421980f5-8390-4901-9151-b72f3dd82e8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:11.336148 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-421980f5-8390-4901-9151-b72f3dd82e8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:11.337329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-421980f5-8390-4901-9151-b72f3dd82e8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:11.355346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-6731c5a0-3e13-4823-a22e-fef5acb84c7b req-e95a9409-0a3a-4dcb-a114-21c8b386dc1b service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:11.357080 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-6731c5a0-3e13-4823-a22e-fef5acb84c7b req-e95a9409-0a3a-4dcb-a114-21c8b386dc1b service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:11.357827 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-6731c5a0-3e13-4823-a22e-fef5acb84c7b req-e95a9409-0a3a-4dcb-a114-21c8b386dc1b service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:11.382182 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-6731c5a0-3e13-4823-a22e-fef5acb84c7b req-e95a9409-0a3a-4dcb-a114-21c8b386dc1b service nova] Creating event network-changed:b9bc50e7-87a3-480a-8c62-87fe75411e15 for instance 03969ac5-934e-4201-8087-0927eae2bbd2 on hv-cinder-maste Aug 28 01:01:11.391544 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-6731c5a0-3e13-4823-a22e-fef5acb84c7b req-e95a9409-0a3a-4dcb-a114-21c8b386dc1b service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070439 Aug 28 01:01:11.392623 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 331/659] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:01:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:11.542545 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-421980f5-8390-4901-9151-b72f3dd82e8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:11.555533 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-421980f5-8390-4901-9151-b72f3dd82e8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1576 microversion: 2.1 time: 0.246940 Aug 28 01:01:11.556403 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 329/660] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:11 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1576 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:12.094937 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ea91a847-5a05-495e-a11f-3c587fabf0f5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:12.099166 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ea91a847-5a05-495e-a11f-3c587fabf0f5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:12.110022 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea91a847-5a05-495e-a11f-3c587fabf0f5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:12.110859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea91a847-5a05-495e-a11f-3c587fabf0f5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:12.111604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea91a847-5a05-495e-a11f-3c587fabf0f5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:12.112850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea91a847-5a05-495e-a11f-3c587fabf0f5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:12.113746 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea91a847-5a05-495e-a11f-3c587fabf0f5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:12.114509 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea91a847-5a05-495e-a11f-3c587fabf0f5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:12.156172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-23a75c62-198e-4e25-8c82-8d48f84b53b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:12.157076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ea91a847-5a05-495e-a11f-3c587fabf0f5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.068272 Aug 28 01:01:12.159931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-23a75c62-198e-4e25-8c82-8d48f84b53b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:12.161498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 332/661] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:12 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 28 01:01:12.170771 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23a75c62-198e-4e25-8c82-8d48f84b53b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:12.171525 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23a75c62-198e-4e25-8c82-8d48f84b53b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:12.172311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23a75c62-198e-4e25-8c82-8d48f84b53b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:12.173935 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23a75c62-198e-4e25-8c82-8d48f84b53b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:12.174781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23a75c62-198e-4e25-8c82-8d48f84b53b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:12.175838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23a75c62-198e-4e25-8c82-8d48f84b53b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:12.244650 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-23a75c62-198e-4e25-8c82-8d48f84b53b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.094105 Aug 28 01:01:12.249051 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 330/662] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:12 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:01:12.583768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-82e59c48-5c96-47ca-bedc-c51bb76700d2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:12.584803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-82e59c48-5c96-47ca-bedc-c51bb76700d2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:12.595326 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-82e59c48-5c96-47ca-bedc-c51bb76700d2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:12.596196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-82e59c48-5c96-47ca-bedc-c51bb76700d2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:12.596935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-82e59c48-5c96-47ca-bedc-c51bb76700d2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:12.598191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-82e59c48-5c96-47ca-bedc-c51bb76700d2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:12.598952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-82e59c48-5c96-47ca-bedc-c51bb76700d2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:12.599828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-82e59c48-5c96-47ca-bedc-c51bb76700d2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:12.636874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:01:12.954961 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-82e59c48-5c96-47ca-bedc-c51bb76700d2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:12.967039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-82e59c48-5c96-47ca-bedc-c51bb76700d2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1576 microversion: 2.1 time: 0.389754 Aug 28 01:01:12.977341 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 333/663] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:12 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1576 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:13.170053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1799b7eb-b258-464e-8e35-074346e7ac99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:13.172561 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1799b7eb-b258-464e-8e35-074346e7ac99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:13.179828 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1799b7eb-b258-464e-8e35-074346e7ac99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:13.180324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1799b7eb-b258-464e-8e35-074346e7ac99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:13.180763 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1799b7eb-b258-464e-8e35-074346e7ac99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:13.181527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1799b7eb-b258-464e-8e35-074346e7ac99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:13.181983 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1799b7eb-b258-464e-8e35-074346e7ac99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:13.182402 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1799b7eb-b258-464e-8e35-074346e7ac99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:13.202407 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:01:13.220352 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1799b7eb-b258-464e-8e35-074346e7ac99 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054850 Aug 28 01:01:13.224988 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 331/664] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:13 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:13.259820 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2d69a9c9-bc06-44e5-9560-3fe25f2fd279 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:13.262277 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2d69a9c9-bc06-44e5-9560-3fe25f2fd279 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:13.271187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d69a9c9-bc06-44e5-9560-3fe25f2fd279 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:13.272030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d69a9c9-bc06-44e5-9560-3fe25f2fd279 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:13.272834 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d69a9c9-bc06-44e5-9560-3fe25f2fd279 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:13.274114 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d69a9c9-bc06-44e5-9560-3fe25f2fd279 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:13.275010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d69a9c9-bc06-44e5-9560-3fe25f2fd279 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:13.275757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d69a9c9-bc06-44e5-9560-3fe25f2fd279 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:13.332780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2d69a9c9-bc06-44e5-9560-3fe25f2fd279 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.077162 Aug 28 01:01:13.333581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 334/665] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:13 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:13.433000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-442278c8-f2ac-4f6e-8c8a-516fa5ed07db req-f33a85e9-e0ec-47cb-b538-5012ae0e8d6e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "03969ac5-934e-4201-8087-0927eae2bbd2", "tag": "b9bc50e7-87a3-480a-8c62-87fe75411e15"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:01:13.446186 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-442278c8-f2ac-4f6e-8c8a-516fa5ed07db req-f33a85e9-e0ec-47cb-b538-5012ae0e8d6e service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:13.446660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-442278c8-f2ac-4f6e-8c8a-516fa5ed07db req-f33a85e9-e0ec-47cb-b538-5012ae0e8d6e service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:13.447087 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-442278c8-f2ac-4f6e-8c8a-516fa5ed07db req-f33a85e9-e0ec-47cb-b538-5012ae0e8d6e service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:13.459884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-442278c8-f2ac-4f6e-8c8a-516fa5ed07db req-f33a85e9-e0ec-47cb-b538-5012ae0e8d6e service nova] Creating event network-changed:b9bc50e7-87a3-480a-8c62-87fe75411e15 for instance 03969ac5-934e-4201-8087-0927eae2bbd2 on hv-cinder-maste Aug 28 01:01:13.465410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-442278c8-f2ac-4f6e-8c8a-516fa5ed07db req-f33a85e9-e0ec-47cb-b538-5012ae0e8d6e service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039032 Aug 28 01:01:13.466105 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 332/666] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:01:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:14.007668 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fd174d03-ef2a-4df2-b115-0f1d7e06088d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:14.009359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fd174d03-ef2a-4df2-b115-0f1d7e06088d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:14.021510 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd174d03-ef2a-4df2-b115-0f1d7e06088d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:14.022148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd174d03-ef2a-4df2-b115-0f1d7e06088d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:14.022713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd174d03-ef2a-4df2-b115-0f1d7e06088d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:14.023890 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd174d03-ef2a-4df2-b115-0f1d7e06088d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:14.025114 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd174d03-ef2a-4df2-b115-0f1d7e06088d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:14.025466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd174d03-ef2a-4df2-b115-0f1d7e06088d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:14.134228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fd174d03-ef2a-4df2-b115-0f1d7e06088d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:14.147058 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fd174d03-ef2a-4df2-b115-0f1d7e06088d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.147571 Aug 28 01:01:14.148248 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 335/667] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:13 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:01:14.237488 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0be48253-1362-49e4-9157-79a4daa54d66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:14.240841 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0be48253-1362-49e4-9157-79a4daa54d66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:14.247818 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0be48253-1362-49e4-9157-79a4daa54d66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:14.248320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0be48253-1362-49e4-9157-79a4daa54d66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:14.248782 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0be48253-1362-49e4-9157-79a4daa54d66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:14.250410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0be48253-1362-49e4-9157-79a4daa54d66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:14.250986 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0be48253-1362-49e4-9157-79a4daa54d66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:14.251609 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0be48253-1362-49e4-9157-79a4daa54d66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:14.290901 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0be48253-1362-49e4-9157-79a4daa54d66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.060809 Aug 28 01:01:14.291770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 333/668] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:14 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 28 01:01:14.349002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-63da2df3-4826-4090-8b3c-bf7487331e12 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:14.352286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-63da2df3-4826-4090-8b3c-bf7487331e12 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:14.359715 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63da2df3-4826-4090-8b3c-bf7487331e12 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:14.360213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63da2df3-4826-4090-8b3c-bf7487331e12 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:14.360687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63da2df3-4826-4090-8b3c-bf7487331e12 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:14.361596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63da2df3-4826-4090-8b3c-bf7487331e12 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:14.362045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63da2df3-4826-4090-8b3c-bf7487331e12 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:14.362470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63da2df3-4826-4090-8b3c-bf7487331e12 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:14.424322 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-63da2df3-4826-4090-8b3c-bf7487331e12 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.078946 Aug 28 01:01:14.426069 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 336/669] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:14 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:15.172186 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4b31fa20-b47e-448f-8f47-d0c569517a66 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:15.172782 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4b31fa20-b47e-448f-8f47-d0c569517a66 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:15.179535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b31fa20-b47e-448f-8f47-d0c569517a66 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:15.180029 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b31fa20-b47e-448f-8f47-d0c569517a66 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:15.180455 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b31fa20-b47e-448f-8f47-d0c569517a66 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:15.181230 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b31fa20-b47e-448f-8f47-d0c569517a66 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:15.181682 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b31fa20-b47e-448f-8f47-d0c569517a66 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:15.182099 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b31fa20-b47e-448f-8f47-d0c569517a66 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:15.250195 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4b31fa20-b47e-448f-8f47-d0c569517a66 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:15.259061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4b31fa20-b47e-448f-8f47-d0c569517a66 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.090774 Aug 28 01:01:15.259873 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 334/670] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:15 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:15.304380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b9d97460-01c3-4617-9f53-53f03a53199e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:15.306719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b9d97460-01c3-4617-9f53-53f03a53199e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:15.313730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9d97460-01c3-4617-9f53-53f03a53199e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:15.314305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9d97460-01c3-4617-9f53-53f03a53199e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:15.314859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9d97460-01c3-4617-9f53-53f03a53199e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:15.315654 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9d97460-01c3-4617-9f53-53f03a53199e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:15.316149 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9d97460-01c3-4617-9f53-53f03a53199e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:15.316637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9d97460-01c3-4617-9f53-53f03a53199e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:15.350661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b9d97460-01c3-4617-9f53-53f03a53199e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050413 Aug 28 01:01:15.351552 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 337/671] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:15 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:15.441291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-603578f2-89b0-4b06-8153-b2a84f516840 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:15.443724 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-603578f2-89b0-4b06-8153-b2a84f516840 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:15.450537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-603578f2-89b0-4b06-8153-b2a84f516840 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:15.451030 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-603578f2-89b0-4b06-8153-b2a84f516840 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:15.451457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-603578f2-89b0-4b06-8153-b2a84f516840 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:15.452174 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-603578f2-89b0-4b06-8153-b2a84f516840 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:15.452629 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-603578f2-89b0-4b06-8153-b2a84f516840 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:15.453077 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-603578f2-89b0-4b06-8153-b2a84f516840 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:15.486610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-603578f2-89b0-4b06-8153-b2a84f516840 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049500 Aug 28 01:01:15.487201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 335/672] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:15 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:15.505388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-e7d1ef79-f083-45e1-8fed-82c81305ccf2 req-34e1e11e-d8ea-4e79-85a2-abac1ca6f547 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1cc5e59c-500c-421e-920c-c08b8ae3a206", "name": "network-vif-deleted", "tag": "de0f5772-f2b1-42a3-84e4-e24d81b36597"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:01:15.515888 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-e7d1ef79-f083-45e1-8fed-82c81305ccf2 req-34e1e11e-d8ea-4e79-85a2-abac1ca6f547 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:15.516363 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-e7d1ef79-f083-45e1-8fed-82c81305ccf2 req-34e1e11e-d8ea-4e79-85a2-abac1ca6f547 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:15.516793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-e7d1ef79-f083-45e1-8fed-82c81305ccf2 req-34e1e11e-d8ea-4e79-85a2-abac1ca6f547 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:15.529944 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-e7d1ef79-f083-45e1-8fed-82c81305ccf2 req-34e1e11e-d8ea-4e79-85a2-abac1ca6f547 service nova] Creating event network-vif-deleted:de0f5772-f2b1-42a3-84e4-e24d81b36597 for instance 1cc5e59c-500c-421e-920c-c08b8ae3a206 on hv-cinder-maste Aug 28 01:01:15.535626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-e7d1ef79-f083-45e1-8fed-82c81305ccf2 req-34e1e11e-d8ea-4e79-85a2-abac1ca6f547 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034025 Aug 28 01:01:15.536187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 338/673] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:01:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:16.281739 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2e203e63-d4c2-4780-a950-6a98b708b6d0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:16.282543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2e203e63-d4c2-4780-a950-6a98b708b6d0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:16.290465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e203e63-d4c2-4780-a950-6a98b708b6d0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:16.290964 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e203e63-d4c2-4780-a950-6a98b708b6d0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:16.291406 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e203e63-d4c2-4780-a950-6a98b708b6d0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:16.292203 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e203e63-d4c2-4780-a950-6a98b708b6d0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:16.292660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e203e63-d4c2-4780-a950-6a98b708b6d0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:16.293168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e203e63-d4c2-4780-a950-6a98b708b6d0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:16.364695 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2e203e63-d4c2-4780-a950-6a98b708b6d0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:16.368563 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cbad7cfe-f690-4e9c-a5e5-d325a456d4fe tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:16.371122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cbad7cfe-f690-4e9c-a5e5-d325a456d4fe tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:16.378488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbad7cfe-f690-4e9c-a5e5-d325a456d4fe tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:16.378962 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2e203e63-d4c2-4780-a950-6a98b708b6d0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.101722 Aug 28 01:01:16.379126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbad7cfe-f690-4e9c-a5e5-d325a456d4fe tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:16.379409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbad7cfe-f690-4e9c-a5e5-d325a456d4fe tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:16.379980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 336/674] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:16 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:16.380172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbad7cfe-f690-4e9c-a5e5-d325a456d4fe tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:16.380629 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbad7cfe-f690-4e9c-a5e5-d325a456d4fe tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:16.381093 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbad7cfe-f690-4e9c-a5e5-d325a456d4fe tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:16.415746 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cbad7cfe-f690-4e9c-a5e5-d325a456d4fe tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.056175 Aug 28 01:01:16.416390 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 339/675] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:16 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 28 01:01:16.500028 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bcf80bc1-ba54-4d1e-b0ba-89e6c9f88703 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:16.502649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bcf80bc1-ba54-4d1e-b0ba-89e6c9f88703 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:16.509694 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcf80bc1-ba54-4d1e-b0ba-89e6c9f88703 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:16.510261 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcf80bc1-ba54-4d1e-b0ba-89e6c9f88703 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:16.510698 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcf80bc1-ba54-4d1e-b0ba-89e6c9f88703 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:16.511436 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcf80bc1-ba54-4d1e-b0ba-89e6c9f88703 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:16.511923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcf80bc1-ba54-4d1e-b0ba-89e6c9f88703 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:16.512419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcf80bc1-ba54-4d1e-b0ba-89e6c9f88703 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:16.549197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bcf80bc1-ba54-4d1e-b0ba-89e6c9f88703 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053371 Aug 28 01:01:16.550091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 337/676] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:16 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:17.403339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fecbfcda-6704-4ca7-80f7-41a73fa363a7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:17.404018 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fecbfcda-6704-4ca7-80f7-41a73fa363a7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:17.412077 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fecbfcda-6704-4ca7-80f7-41a73fa363a7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:17.412575 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fecbfcda-6704-4ca7-80f7-41a73fa363a7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:17.413052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fecbfcda-6704-4ca7-80f7-41a73fa363a7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:17.413777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fecbfcda-6704-4ca7-80f7-41a73fa363a7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:17.414234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fecbfcda-6704-4ca7-80f7-41a73fa363a7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:17.414662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fecbfcda-6704-4ca7-80f7-41a73fa363a7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:17.428602 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f2938378-1431-4dca-a5dd-6653f7604df1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:17.431746 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f2938378-1431-4dca-a5dd-6653f7604df1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:17.438411 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2938378-1431-4dca-a5dd-6653f7604df1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:17.438921 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2938378-1431-4dca-a5dd-6653f7604df1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:17.439355 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2938378-1431-4dca-a5dd-6653f7604df1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:17.440077 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2938378-1431-4dca-a5dd-6653f7604df1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:17.440530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2938378-1431-4dca-a5dd-6653f7604df1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:17.440950 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2938378-1431-4dca-a5dd-6653f7604df1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:17.475835 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f2938378-1431-4dca-a5dd-6653f7604df1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051416 Aug 28 01:01:17.476443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 338/677] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:17 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 28 01:01:17.494331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fecbfcda-6704-4ca7-80f7-41a73fa363a7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:17.503934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fecbfcda-6704-4ca7-80f7-41a73fa363a7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.104698 Aug 28 01:01:17.504694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 340/678] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:17 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:17.562709 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-21c5f7e8-6b92-4be2-bcbc-7dbea685bbf2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:17.564946 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-21c5f7e8-6b92-4be2-bcbc-7dbea685bbf2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:17.572220 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-21c5f7e8-6b92-4be2-bcbc-7dbea685bbf2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:17.572713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-21c5f7e8-6b92-4be2-bcbc-7dbea685bbf2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:17.573357 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-21c5f7e8-6b92-4be2-bcbc-7dbea685bbf2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:17.574060 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-21c5f7e8-6b92-4be2-bcbc-7dbea685bbf2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:17.574502 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-21c5f7e8-6b92-4be2-bcbc-7dbea685bbf2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:17.574912 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-21c5f7e8-6b92-4be2-bcbc-7dbea685bbf2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:17.607644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-21c5f7e8-6b92-4be2-bcbc-7dbea685bbf2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048891 Aug 28 01:01:17.608229 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 339/679] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:17 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:18.490735 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ed80056f-d19e-4eeb-92e1-ba99b57adb45 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:18.492996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ed80056f-d19e-4eeb-92e1-ba99b57adb45 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:18.500338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed80056f-d19e-4eeb-92e1-ba99b57adb45 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:18.500990 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed80056f-d19e-4eeb-92e1-ba99b57adb45 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:18.501477 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed80056f-d19e-4eeb-92e1-ba99b57adb45 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:18.502282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed80056f-d19e-4eeb-92e1-ba99b57adb45 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:18.502780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed80056f-d19e-4eeb-92e1-ba99b57adb45 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:18.503360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed80056f-d19e-4eeb-92e1-ba99b57adb45 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:18.525960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1ef47bbe-ca40-4cec-a22e-343021c0112f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:18.527144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1ef47bbe-ca40-4cec-a22e-343021c0112f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:18.535799 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1ef47bbe-ca40-4cec-a22e-343021c0112f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:18.536354 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1ef47bbe-ca40-4cec-a22e-343021c0112f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:18.537043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1ef47bbe-ca40-4cec-a22e-343021c0112f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:18.537917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1ef47bbe-ca40-4cec-a22e-343021c0112f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:18.538620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1ef47bbe-ca40-4cec-a22e-343021c0112f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:18.539191 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1ef47bbe-ca40-4cec-a22e-343021c0112f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:18.540381 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ed80056f-d19e-4eeb-92e1-ba99b57adb45 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053810 Aug 28 01:01:18.541325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 341/680] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:18 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:18.609251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1ef47bbe-ca40-4cec-a22e-343021c0112f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:18.620403 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1ef47bbe-ca40-4cec-a22e-343021c0112f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.098640 Aug 28 01:01:18.620840 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7c142940-4a64-478a-a75f-5d9e04abfb5b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:18.621226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 340/681] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:18 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:01:18.623631 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7c142940-4a64-478a-a75f-5d9e04abfb5b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:18.630946 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c142940-4a64-478a-a75f-5d9e04abfb5b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:18.631446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c142940-4a64-478a-a75f-5d9e04abfb5b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:18.631883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c142940-4a64-478a-a75f-5d9e04abfb5b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:18.632598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c142940-4a64-478a-a75f-5d9e04abfb5b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:18.633253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c142940-4a64-478a-a75f-5d9e04abfb5b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:18.633748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c142940-4a64-478a-a75f-5d9e04abfb5b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:18.669382 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7c142940-4a64-478a-a75f-5d9e04abfb5b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052005 Aug 28 01:01:18.670022 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 342/682] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:18 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:19.555410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6fa605ff-9555-41f8-8e71-0b150b9c6b9c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:19.557638 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6fa605ff-9555-41f8-8e71-0b150b9c6b9c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:19.565120 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6fa605ff-9555-41f8-8e71-0b150b9c6b9c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:19.565741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6fa605ff-9555-41f8-8e71-0b150b9c6b9c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:19.566276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6fa605ff-9555-41f8-8e71-0b150b9c6b9c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:19.567102 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6fa605ff-9555-41f8-8e71-0b150b9c6b9c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:19.567680 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6fa605ff-9555-41f8-8e71-0b150b9c6b9c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:19.568261 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6fa605ff-9555-41f8-8e71-0b150b9c6b9c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:19.610761 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6fa605ff-9555-41f8-8e71-0b150b9c6b9c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.059229 Aug 28 01:01:19.611642 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 341/683] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:19 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:19.644706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8de70917-8182-4d99-a973-a397b45a80fc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:19.645398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8de70917-8182-4d99-a973-a397b45a80fc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:19.653351 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8de70917-8182-4d99-a973-a397b45a80fc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:19.654155 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8de70917-8182-4d99-a973-a397b45a80fc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:19.654824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8de70917-8182-4d99-a973-a397b45a80fc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:19.655604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8de70917-8182-4d99-a973-a397b45a80fc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:19.656113 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8de70917-8182-4d99-a973-a397b45a80fc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:19.656585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8de70917-8182-4d99-a973-a397b45a80fc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:19.683404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-48615ed0-c0df-4e9a-8432-190a1f500f2c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:19.686055 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-48615ed0-c0df-4e9a-8432-190a1f500f2c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:19.693434 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48615ed0-c0df-4e9a-8432-190a1f500f2c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:19.694527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48615ed0-c0df-4e9a-8432-190a1f500f2c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:19.695067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48615ed0-c0df-4e9a-8432-190a1f500f2c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:19.695896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48615ed0-c0df-4e9a-8432-190a1f500f2c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:19.696450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48615ed0-c0df-4e9a-8432-190a1f500f2c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:19.697006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48615ed0-c0df-4e9a-8432-190a1f500f2c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:19.726337 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8de70917-8182-4d99-a973-a397b45a80fc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:19.733999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-48615ed0-c0df-4e9a-8432-190a1f500f2c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054978 Aug 28 01:01:19.734795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 342/684] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:19 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:19.739288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8de70917-8182-4d99-a973-a397b45a80fc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.099766 Aug 28 01:01:19.740842 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 343/685] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:19 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:20.625018 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7ef568ef-01a9-47c7-83f0-d1659efe0b6a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:20.627228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7ef568ef-01a9-47c7-83f0-d1659efe0b6a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:20.634519 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ef568ef-01a9-47c7-83f0-d1659efe0b6a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:20.635030 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ef568ef-01a9-47c7-83f0-d1659efe0b6a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:20.635489 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ef568ef-01a9-47c7-83f0-d1659efe0b6a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:20.636222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ef568ef-01a9-47c7-83f0-d1659efe0b6a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:20.636673 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ef568ef-01a9-47c7-83f0-d1659efe0b6a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:20.637167 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ef568ef-01a9-47c7-83f0-d1659efe0b6a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:20.671459 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7ef568ef-01a9-47c7-83f0-d1659efe0b6a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050351 Aug 28 01:01:20.672016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 343/686] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:20 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:20.747666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f3aca93a-12fb-4709-8f8b-c66f9b8279c3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:20.750095 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f3aca93a-12fb-4709-8f8b-c66f9b8279c3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:20.757417 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3aca93a-12fb-4709-8f8b-c66f9b8279c3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:20.758007 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3aca93a-12fb-4709-8f8b-c66f9b8279c3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:20.758415 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3aca93a-12fb-4709-8f8b-c66f9b8279c3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:20.759131 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3aca93a-12fb-4709-8f8b-c66f9b8279c3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:20.759581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3aca93a-12fb-4709-8f8b-c66f9b8279c3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:20.760000 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3aca93a-12fb-4709-8f8b-c66f9b8279c3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:20.762958 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7cd376da-c3c9-4745-9f46-f84fafa7ae8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:20.763667 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7cd376da-c3c9-4745-9f46-f84fafa7ae8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:20.770918 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7cd376da-c3c9-4745-9f46-f84fafa7ae8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:20.771428 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7cd376da-c3c9-4745-9f46-f84fafa7ae8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:20.771861 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7cd376da-c3c9-4745-9f46-f84fafa7ae8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:20.772726 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7cd376da-c3c9-4745-9f46-f84fafa7ae8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:20.773332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7cd376da-c3c9-4745-9f46-f84fafa7ae8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:20.773758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7cd376da-c3c9-4745-9f46-f84fafa7ae8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:20.804787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f3aca93a-12fb-4709-8f8b-c66f9b8279c3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061007 Aug 28 01:01:20.805454 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 344/687] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:20 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:20.853707 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7cd376da-c3c9-4745-9f46-f84fafa7ae8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:20.862202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7cd376da-c3c9-4745-9f46-f84fafa7ae8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.103918 Aug 28 01:01:20.862967 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 344/688] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:20 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:01:21.686401 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bc40a7b3-4ec2-449c-b747-faa96a42b5f7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:21.689398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bc40a7b3-4ec2-449c-b747-faa96a42b5f7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:21.698104 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc40a7b3-4ec2-449c-b747-faa96a42b5f7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:21.698789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc40a7b3-4ec2-449c-b747-faa96a42b5f7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:21.699276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc40a7b3-4ec2-449c-b747-faa96a42b5f7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:21.700208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc40a7b3-4ec2-449c-b747-faa96a42b5f7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:21.700926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc40a7b3-4ec2-449c-b747-faa96a42b5f7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:21.701495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc40a7b3-4ec2-449c-b747-faa96a42b5f7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:21.738700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bc40a7b3-4ec2-449c-b747-faa96a42b5f7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.057383 Aug 28 01:01:21.739488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 345/689] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:21 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:21.817795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0f6de057-769f-40d6-b863-735a04b8ceff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:21.820420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0f6de057-769f-40d6-b863-735a04b8ceff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:21.827570 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f6de057-769f-40d6-b863-735a04b8ceff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:21.828188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f6de057-769f-40d6-b863-735a04b8ceff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:21.828650 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f6de057-769f-40d6-b863-735a04b8ceff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:21.829414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f6de057-769f-40d6-b863-735a04b8ceff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:21.829853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f6de057-769f-40d6-b863-735a04b8ceff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:21.830362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f6de057-769f-40d6-b863-735a04b8ceff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:21.864398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0f6de057-769f-40d6-b863-735a04b8ceff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050534 Aug 28 01:01:21.865176 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 345/690] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:21 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:21.884720 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-df6211dd-3126-4beb-9b38-2d403cf0cef2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:21.885450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-df6211dd-3126-4beb-9b38-2d403cf0cef2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:21.893010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df6211dd-3126-4beb-9b38-2d403cf0cef2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:21.893531 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df6211dd-3126-4beb-9b38-2d403cf0cef2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:21.893961 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df6211dd-3126-4beb-9b38-2d403cf0cef2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:21.894681 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df6211dd-3126-4beb-9b38-2d403cf0cef2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:21.895133 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df6211dd-3126-4beb-9b38-2d403cf0cef2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:21.895551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df6211dd-3126-4beb-9b38-2d403cf0cef2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:21.967890 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-df6211dd-3126-4beb-9b38-2d403cf0cef2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:21.977230 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-df6211dd-3126-4beb-9b38-2d403cf0cef2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.096857 Aug 28 01:01:21.978039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 346/691] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:21 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:22.753335 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-46be5b30-5447-4cce-86ca-bedf906ddbb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:22.755659 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-46be5b30-5447-4cce-86ca-bedf906ddbb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:22.762675 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-46be5b30-5447-4cce-86ca-bedf906ddbb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:22.763177 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-46be5b30-5447-4cce-86ca-bedf906ddbb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:22.763602 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-46be5b30-5447-4cce-86ca-bedf906ddbb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:22.764333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-46be5b30-5447-4cce-86ca-bedf906ddbb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:22.764791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-46be5b30-5447-4cce-86ca-bedf906ddbb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:22.765246 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-46be5b30-5447-4cce-86ca-bedf906ddbb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:22.799375 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-46be5b30-5447-4cce-86ca-bedf906ddbb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050280 Aug 28 01:01:22.799969 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 346/692] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:22 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:22.877804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b723a021-47a2-44ef-b4b7-bf167e0ea8e7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:22.880066 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b723a021-47a2-44ef-b4b7-bf167e0ea8e7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:22.886927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b723a021-47a2-44ef-b4b7-bf167e0ea8e7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:22.887696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b723a021-47a2-44ef-b4b7-bf167e0ea8e7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:22.887816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b723a021-47a2-44ef-b4b7-bf167e0ea8e7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:22.888558 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b723a021-47a2-44ef-b4b7-bf167e0ea8e7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:22.889047 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b723a021-47a2-44ef-b4b7-bf167e0ea8e7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:22.889475 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b723a021-47a2-44ef-b4b7-bf167e0ea8e7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:22.925599 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b723a021-47a2-44ef-b4b7-bf167e0ea8e7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051799 Aug 28 01:01:22.926183 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 347/693] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:22 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:22.999943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-21b7796f-b02d-459c-8414-a7545edf1f3e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:23.000673 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-21b7796f-b02d-459c-8414-a7545edf1f3e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:23.008981 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-21b7796f-b02d-459c-8414-a7545edf1f3e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:23.009512 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-21b7796f-b02d-459c-8414-a7545edf1f3e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:23.009943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-21b7796f-b02d-459c-8414-a7545edf1f3e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:23.010683 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-21b7796f-b02d-459c-8414-a7545edf1f3e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:23.011150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-21b7796f-b02d-459c-8414-a7545edf1f3e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:23.011571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-21b7796f-b02d-459c-8414-a7545edf1f3e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:23.083893 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-21b7796f-b02d-459c-8414-a7545edf1f3e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:23.098086 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-21b7796f-b02d-459c-8414-a7545edf1f3e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.101936 Aug 28 01:01:23.099001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 347/694] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:22 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:23.812442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ff731179-dd72-42b9-a2f9-7626477273c1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:23.814931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ff731179-dd72-42b9-a2f9-7626477273c1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:23.822140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ff731179-dd72-42b9-a2f9-7626477273c1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:23.822637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ff731179-dd72-42b9-a2f9-7626477273c1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:23.823064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ff731179-dd72-42b9-a2f9-7626477273c1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:23.823782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ff731179-dd72-42b9-a2f9-7626477273c1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:23.824257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ff731179-dd72-42b9-a2f9-7626477273c1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:23.824682 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ff731179-dd72-42b9-a2f9-7626477273c1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:23.859115 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ff731179-dd72-42b9-a2f9-7626477273c1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050682 Aug 28 01:01:23.859762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 348/695] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:23 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:23.938915 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-62e06fea-2a41-443a-b408-29f419dacc73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:23.941128 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-62e06fea-2a41-443a-b408-29f419dacc73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:23.949003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-62e06fea-2a41-443a-b408-29f419dacc73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:23.949482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-62e06fea-2a41-443a-b408-29f419dacc73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:23.949919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-62e06fea-2a41-443a-b408-29f419dacc73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:23.950638 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-62e06fea-2a41-443a-b408-29f419dacc73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:23.951090 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-62e06fea-2a41-443a-b408-29f419dacc73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:23.951498 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-62e06fea-2a41-443a-b408-29f419dacc73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:23.982172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:01:23.987832 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-62e06fea-2a41-443a-b408-29f419dacc73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053254 Aug 28 01:01:23.988414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 348/696] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:23 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:24.122983 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-77455442-3250-4650-a72e-bd6c41d9f4dc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:24.124074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-77455442-3250-4650-a72e-bd6c41d9f4dc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:24.132253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-77455442-3250-4650-a72e-bd6c41d9f4dc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:24.132840 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-77455442-3250-4650-a72e-bd6c41d9f4dc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:24.133365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-77455442-3250-4650-a72e-bd6c41d9f4dc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:24.134203 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-77455442-3250-4650-a72e-bd6c41d9f4dc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:24.134841 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-77455442-3250-4650-a72e-bd6c41d9f4dc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:24.135349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-77455442-3250-4650-a72e-bd6c41d9f4dc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:24.211777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-77455442-3250-4650-a72e-bd6c41d9f4dc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:24.220841 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-77455442-3250-4650-a72e-bd6c41d9f4dc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.105161 Aug 28 01:01:24.221553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 349/697] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:24 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:01:24.872677 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-de381caf-6094-45f7-8691-38830b84c73b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:24.874918 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-de381caf-6094-45f7-8691-38830b84c73b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:24.882143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de381caf-6094-45f7-8691-38830b84c73b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:24.882818 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de381caf-6094-45f7-8691-38830b84c73b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:24.883280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de381caf-6094-45f7-8691-38830b84c73b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:24.884128 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de381caf-6094-45f7-8691-38830b84c73b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:24.884724 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de381caf-6094-45f7-8691-38830b84c73b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:24.885252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de381caf-6094-45f7-8691-38830b84c73b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:24.922476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-de381caf-6094-45f7-8691-38830b84c73b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054009 Aug 28 01:01:24.923115 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 349/698] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:24 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:25.001113 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9e8c9717-9172-46e0-9fda-2219eb573eab tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:25.003535 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9e8c9717-9172-46e0-9fda-2219eb573eab tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:25.010263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9e8c9717-9172-46e0-9fda-2219eb573eab tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:25.010755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9e8c9717-9172-46e0-9fda-2219eb573eab tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:25.011220 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9e8c9717-9172-46e0-9fda-2219eb573eab tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:25.011948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9e8c9717-9172-46e0-9fda-2219eb573eab tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:25.012404 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9e8c9717-9172-46e0-9fda-2219eb573eab tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:25.012830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9e8c9717-9172-46e0-9fda-2219eb573eab tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:25.043759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:01:25.049880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9e8c9717-9172-46e0-9fda-2219eb573eab tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052943 Aug 28 01:01:25.050488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 350/699] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:24 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:25.243473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b3275587-433c-44ed-9495-54415753ae95 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:25.244172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b3275587-433c-44ed-9495-54415753ae95 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:25.251954 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3275587-433c-44ed-9495-54415753ae95 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:25.252472 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3275587-433c-44ed-9495-54415753ae95 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:25.252907 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3275587-433c-44ed-9495-54415753ae95 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:25.253695 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3275587-433c-44ed-9495-54415753ae95 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:25.254153 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3275587-433c-44ed-9495-54415753ae95 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:25.254578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3275587-433c-44ed-9495-54415753ae95 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:25.326984 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b3275587-433c-44ed-9495-54415753ae95 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:25.335894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b3275587-433c-44ed-9495-54415753ae95 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.096792 Aug 28 01:01:25.336491 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 350/700] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:25 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:25.936229 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5001a6b3-df02-4db9-8f7c-a0eaccf9d297 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:25.938582 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5001a6b3-df02-4db9-8f7c-a0eaccf9d297 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:25.947083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5001a6b3-df02-4db9-8f7c-a0eaccf9d297 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:25.947662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5001a6b3-df02-4db9-8f7c-a0eaccf9d297 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:25.948099 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5001a6b3-df02-4db9-8f7c-a0eaccf9d297 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:25.948829 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5001a6b3-df02-4db9-8f7c-a0eaccf9d297 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:25.949319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5001a6b3-df02-4db9-8f7c-a0eaccf9d297 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:25.949745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5001a6b3-df02-4db9-8f7c-a0eaccf9d297 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:25.987278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5001a6b3-df02-4db9-8f7c-a0eaccf9d297 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055567 Aug 28 01:01:25.987996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 351/701] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:25 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:26.066190 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-00d626e4-a124-437c-9162-fcf72c7ed74f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:26.068411 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-00d626e4-a124-437c-9162-fcf72c7ed74f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:26.075625 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00d626e4-a124-437c-9162-fcf72c7ed74f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:26.076109 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00d626e4-a124-437c-9162-fcf72c7ed74f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:26.076549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00d626e4-a124-437c-9162-fcf72c7ed74f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:26.077333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00d626e4-a124-437c-9162-fcf72c7ed74f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:26.077789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00d626e4-a124-437c-9162-fcf72c7ed74f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:26.078215 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00d626e4-a124-437c-9162-fcf72c7ed74f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:26.113404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-00d626e4-a124-437c-9162-fcf72c7ed74f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052937 Aug 28 01:01:26.114504 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 351/702] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:26 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:26.373086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0c726396-938a-437f-be4d-fd810bc79154 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:26.374051 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0c726396-938a-437f-be4d-fd810bc79154 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:26.390962 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c726396-938a-437f-be4d-fd810bc79154 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:26.391852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c726396-938a-437f-be4d-fd810bc79154 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:26.392632 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c726396-938a-437f-be4d-fd810bc79154 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:26.393913 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c726396-938a-437f-be4d-fd810bc79154 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:26.394699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c726396-938a-437f-be4d-fd810bc79154 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:26.395473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c726396-938a-437f-be4d-fd810bc79154 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:26.470093 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0c726396-938a-437f-be4d-fd810bc79154 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:26.479665 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0c726396-938a-437f-be4d-fd810bc79154 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.110821 Aug 28 01:01:26.480357 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 352/703] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:26 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:27.001438 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e91ae85b-0826-4bf6-80f5-8a182216f036 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:27.003597 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e91ae85b-0826-4bf6-80f5-8a182216f036 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:27.012394 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e91ae85b-0826-4bf6-80f5-8a182216f036 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:27.012875 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e91ae85b-0826-4bf6-80f5-8a182216f036 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:27.013369 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e91ae85b-0826-4bf6-80f5-8a182216f036 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:27.014121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e91ae85b-0826-4bf6-80f5-8a182216f036 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:27.014575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e91ae85b-0826-4bf6-80f5-8a182216f036 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:27.015115 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e91ae85b-0826-4bf6-80f5-8a182216f036 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:27.048365 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e91ae85b-0826-4bf6-80f5-8a182216f036 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051700 Aug 28 01:01:27.048950 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 352/704] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:26 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:27.129504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-433900df-ae36-4b06-b2c5-765b92b4284c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:27.131895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-433900df-ae36-4b06-b2c5-765b92b4284c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:27.139246 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-433900df-ae36-4b06-b2c5-765b92b4284c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:27.139743 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-433900df-ae36-4b06-b2c5-765b92b4284c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:27.140175 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-433900df-ae36-4b06-b2c5-765b92b4284c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:27.140910 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-433900df-ae36-4b06-b2c5-765b92b4284c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:27.141386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-433900df-ae36-4b06-b2c5-765b92b4284c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:27.141806 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-433900df-ae36-4b06-b2c5-765b92b4284c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:27.175997 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-433900df-ae36-4b06-b2c5-765b92b4284c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050503 Aug 28 01:01:27.176620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 353/705] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:27 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:27.501302 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-82a06053-e1df-4b0b-bb1f-283cb04d00a8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:27.502015 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-82a06053-e1df-4b0b-bb1f-283cb04d00a8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:27.508923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82a06053-e1df-4b0b-bb1f-283cb04d00a8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:27.509445 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82a06053-e1df-4b0b-bb1f-283cb04d00a8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:27.509945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82a06053-e1df-4b0b-bb1f-283cb04d00a8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:27.510819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82a06053-e1df-4b0b-bb1f-283cb04d00a8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:27.511450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82a06053-e1df-4b0b-bb1f-283cb04d00a8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:27.511965 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82a06053-e1df-4b0b-bb1f-283cb04d00a8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:27.582499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-82a06053-e1df-4b0b-bb1f-283cb04d00a8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:27.596149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-82a06053-e1df-4b0b-bb1f-283cb04d00a8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.098922 Aug 28 01:01:27.597070 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 353/706] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:27 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:28.064218 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5eff0c9c-6716-48e1-b219-ce8f34a8132f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:28.066935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5eff0c9c-6716-48e1-b219-ce8f34a8132f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:28.074564 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5eff0c9c-6716-48e1-b219-ce8f34a8132f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:28.075082 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5eff0c9c-6716-48e1-b219-ce8f34a8132f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:28.075634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5eff0c9c-6716-48e1-b219-ce8f34a8132f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:28.076399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5eff0c9c-6716-48e1-b219-ce8f34a8132f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:28.077166 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5eff0c9c-6716-48e1-b219-ce8f34a8132f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:28.077636 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5eff0c9c-6716-48e1-b219-ce8f34a8132f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:28.111908 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5eff0c9c-6716-48e1-b219-ce8f34a8132f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051907 Aug 28 01:01:28.112509 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 354/707] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:28 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:28.188670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-73706200-8080-4b5b-ac2b-e0007a5ac4ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:28.190879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-73706200-8080-4b5b-ac2b-e0007a5ac4ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:28.198527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73706200-8080-4b5b-ac2b-e0007a5ac4ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:28.199070 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73706200-8080-4b5b-ac2b-e0007a5ac4ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:28.199572 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73706200-8080-4b5b-ac2b-e0007a5ac4ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:28.200328 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73706200-8080-4b5b-ac2b-e0007a5ac4ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:28.200888 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73706200-8080-4b5b-ac2b-e0007a5ac4ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:28.201328 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73706200-8080-4b5b-ac2b-e0007a5ac4ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:28.237517 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-73706200-8080-4b5b-ac2b-e0007a5ac4ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052756 Aug 28 01:01:28.238107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 354/708] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:28 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:28.627963 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-01cea208-0c79-43ba-b1eb-98f031b149ff tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:28.629203 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-01cea208-0c79-43ba-b1eb-98f031b149ff tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:28.637830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-01cea208-0c79-43ba-b1eb-98f031b149ff tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:28.638364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-01cea208-0c79-43ba-b1eb-98f031b149ff tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:28.638774 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-01cea208-0c79-43ba-b1eb-98f031b149ff tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:28.639485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-01cea208-0c79-43ba-b1eb-98f031b149ff tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:28.639941 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-01cea208-0c79-43ba-b1eb-98f031b149ff tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:28.640354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-01cea208-0c79-43ba-b1eb-98f031b149ff tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:28.707156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-01cea208-0c79-43ba-b1eb-98f031b149ff tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:28.716611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-01cea208-0c79-43ba-b1eb-98f031b149ff tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.095773 Aug 28 01:01:28.716611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 355/709] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:28 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:29.124133 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5b38384c-4bb8-4243-ba7b-756ceb533bf8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:29.126581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5b38384c-4bb8-4243-ba7b-756ceb533bf8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:29.133060 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b38384c-4bb8-4243-ba7b-756ceb533bf8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:29.133555 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b38384c-4bb8-4243-ba7b-756ceb533bf8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:29.133976 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b38384c-4bb8-4243-ba7b-756ceb533bf8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:29.134684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b38384c-4bb8-4243-ba7b-756ceb533bf8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:29.135119 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b38384c-4bb8-4243-ba7b-756ceb533bf8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:29.135536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b38384c-4bb8-4243-ba7b-756ceb533bf8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:29.170945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5b38384c-4bb8-4243-ba7b-756ceb533bf8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050792 Aug 28 01:01:29.171541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 355/710] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:29 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:29.249611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8aeae758-14fd-4705-a16d-9506c5e935b9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:29.251852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8aeae758-14fd-4705-a16d-9506c5e935b9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:29.258899 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8aeae758-14fd-4705-a16d-9506c5e935b9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:29.259399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8aeae758-14fd-4705-a16d-9506c5e935b9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:29.259842 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8aeae758-14fd-4705-a16d-9506c5e935b9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:29.260647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8aeae758-14fd-4705-a16d-9506c5e935b9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:29.261133 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8aeae758-14fd-4705-a16d-9506c5e935b9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:29.261555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8aeae758-14fd-4705-a16d-9506c5e935b9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:29.299904 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8aeae758-14fd-4705-a16d-9506c5e935b9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054361 Aug 28 01:01:29.300499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 356/711] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:29 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:29.741464 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b29b2e3f-d8a2-407a-b959-9a3e8ed50403 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:29.742078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b29b2e3f-d8a2-407a-b959-9a3e8ed50403 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:29.749383 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b29b2e3f-d8a2-407a-b959-9a3e8ed50403 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:29.749878 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b29b2e3f-d8a2-407a-b959-9a3e8ed50403 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:29.750312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b29b2e3f-d8a2-407a-b959-9a3e8ed50403 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:29.751065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b29b2e3f-d8a2-407a-b959-9a3e8ed50403 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:29.751520 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b29b2e3f-d8a2-407a-b959-9a3e8ed50403 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:29.751938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b29b2e3f-d8a2-407a-b959-9a3e8ed50403 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:29.825848 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b29b2e3f-d8a2-407a-b959-9a3e8ed50403 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:29.841144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b29b2e3f-d8a2-407a-b959-9a3e8ed50403 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.103562 Aug 28 01:01:29.842405 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 356/712] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:29 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:30.184189 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-25440e73-b6a7-472b-b347-6480e42f1c69 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:30.186460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-25440e73-b6a7-472b-b347-6480e42f1c69 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:30.194145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-25440e73-b6a7-472b-b347-6480e42f1c69 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:30.194646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-25440e73-b6a7-472b-b347-6480e42f1c69 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:30.195084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-25440e73-b6a7-472b-b347-6480e42f1c69 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:30.195816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-25440e73-b6a7-472b-b347-6480e42f1c69 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:30.196273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-25440e73-b6a7-472b-b347-6480e42f1c69 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:30.196700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-25440e73-b6a7-472b-b347-6480e42f1c69 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:30.231002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-25440e73-b6a7-472b-b347-6480e42f1c69 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050841 Aug 28 01:01:30.231609 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 357/713] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:30 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 28 01:01:30.313235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c3f98496-62a9-4a29-a0ac-cbacf1c8c2ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:30.315448 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c3f98496-62a9-4a29-a0ac-cbacf1c8c2ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:30.322641 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3f98496-62a9-4a29-a0ac-cbacf1c8c2ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:30.323139 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3f98496-62a9-4a29-a0ac-cbacf1c8c2ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:30.323542 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3f98496-62a9-4a29-a0ac-cbacf1c8c2ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:30.324288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3f98496-62a9-4a29-a0ac-cbacf1c8c2ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:30.324738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3f98496-62a9-4a29-a0ac-cbacf1c8c2ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:30.325217 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3f98496-62a9-4a29-a0ac-cbacf1c8c2ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:30.359002 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c3f98496-62a9-4a29-a0ac-cbacf1c8c2ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050020 Aug 28 01:01:30.359574 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 357/714] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:30 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:30.864244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-61098363-20bf-4fda-ba18-dda8abbe157d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:30.864933 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-61098363-20bf-4fda-ba18-dda8abbe157d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:30.871980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61098363-20bf-4fda-ba18-dda8abbe157d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:30.872488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61098363-20bf-4fda-ba18-dda8abbe157d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:30.873048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61098363-20bf-4fda-ba18-dda8abbe157d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:30.874106 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61098363-20bf-4fda-ba18-dda8abbe157d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:30.874677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61098363-20bf-4fda-ba18-dda8abbe157d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:30.875152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61098363-20bf-4fda-ba18-dda8abbe157d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:30.947421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-61098363-20bf-4fda-ba18-dda8abbe157d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:30.960075 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-61098363-20bf-4fda-ba18-dda8abbe157d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.099923 Aug 28 01:01:30.961174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 358/715] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:30 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:31.244773 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-20776bef-776b-4023-bc0e-3843fe617ec1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:31.247075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-20776bef-776b-4023-bc0e-3843fe617ec1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:31.254420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20776bef-776b-4023-bc0e-3843fe617ec1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:31.255138 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20776bef-776b-4023-bc0e-3843fe617ec1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:31.255588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20776bef-776b-4023-bc0e-3843fe617ec1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:31.256401 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20776bef-776b-4023-bc0e-3843fe617ec1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:31.256877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20776bef-776b-4023-bc0e-3843fe617ec1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:31.257412 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20776bef-776b-4023-bc0e-3843fe617ec1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:31.293534 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-20776bef-776b-4023-bc0e-3843fe617ec1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053113 Aug 28 01:01:31.294176 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 358/716] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:31 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:31.372235 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0c95a670-b855-4bf1-9d05-dcef3371f5f6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:31.374623 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0c95a670-b855-4bf1-9d05-dcef3371f5f6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:31.382099 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c95a670-b855-4bf1-9d05-dcef3371f5f6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:31.382605 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c95a670-b855-4bf1-9d05-dcef3371f5f6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:31.383045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c95a670-b855-4bf1-9d05-dcef3371f5f6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:31.383762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c95a670-b855-4bf1-9d05-dcef3371f5f6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:31.384218 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c95a670-b855-4bf1-9d05-dcef3371f5f6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:31.384642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c95a670-b855-4bf1-9d05-dcef3371f5f6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:31.419172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0c95a670-b855-4bf1-9d05-dcef3371f5f6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050936 Aug 28 01:01:31.419772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 359/717] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:31 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:31.995202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-80ba3d37-6a04-462f-9375-38391f7cbd05 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:31.996044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-80ba3d37-6a04-462f-9375-38391f7cbd05 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:32.003457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-80ba3d37-6a04-462f-9375-38391f7cbd05 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:32.004007 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-80ba3d37-6a04-462f-9375-38391f7cbd05 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:32.004487 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-80ba3d37-6a04-462f-9375-38391f7cbd05 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:32.005321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-80ba3d37-6a04-462f-9375-38391f7cbd05 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:32.005773 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-80ba3d37-6a04-462f-9375-38391f7cbd05 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:32.006193 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-80ba3d37-6a04-462f-9375-38391f7cbd05 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:32.072788 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-80ba3d37-6a04-462f-9375-38391f7cbd05 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:32.081920 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-80ba3d37-6a04-462f-9375-38391f7cbd05 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.090694 Aug 28 01:01:32.082550 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 359/718] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:31 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:32.309242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-584e506f-86fa-4d43-917d-6f4b8d1aa43c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:32.312781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-584e506f-86fa-4d43-917d-6f4b8d1aa43c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:32.323101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-584e506f-86fa-4d43-917d-6f4b8d1aa43c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:32.323961 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-584e506f-86fa-4d43-917d-6f4b8d1aa43c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:32.324734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-584e506f-86fa-4d43-917d-6f4b8d1aa43c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:32.326047 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-584e506f-86fa-4d43-917d-6f4b8d1aa43c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:32.326810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-584e506f-86fa-4d43-917d-6f4b8d1aa43c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:32.327549 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-584e506f-86fa-4d43-917d-6f4b8d1aa43c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:32.369623 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-584e506f-86fa-4d43-917d-6f4b8d1aa43c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.066330 Aug 28 01:01:32.370703 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 360/719] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:32 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:32.435324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3cc129c6-7dff-4d7b-9aa4-9941f9eea2f2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:32.437554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3cc129c6-7dff-4d7b-9aa4-9941f9eea2f2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:32.445339 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cc129c6-7dff-4d7b-9aa4-9941f9eea2f2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:32.446086 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cc129c6-7dff-4d7b-9aa4-9941f9eea2f2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:32.446577 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cc129c6-7dff-4d7b-9aa4-9941f9eea2f2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:32.447488 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cc129c6-7dff-4d7b-9aa4-9941f9eea2f2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:32.447977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cc129c6-7dff-4d7b-9aa4-9941f9eea2f2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:32.448489 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cc129c6-7dff-4d7b-9aa4-9941f9eea2f2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:32.484606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3cc129c6-7dff-4d7b-9aa4-9941f9eea2f2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053479 Aug 28 01:01:32.485231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 360/720] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:32 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:33.106348 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-54f29b72-44e2-4de4-9739-638ce25f5a13 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:33.107433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-54f29b72-44e2-4de4-9739-638ce25f5a13 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:33.119231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54f29b72-44e2-4de4-9739-638ce25f5a13 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:33.120051 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54f29b72-44e2-4de4-9739-638ce25f5a13 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:33.120739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54f29b72-44e2-4de4-9739-638ce25f5a13 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:33.121971 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54f29b72-44e2-4de4-9739-638ce25f5a13 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:33.122705 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54f29b72-44e2-4de4-9739-638ce25f5a13 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:33.123323 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54f29b72-44e2-4de4-9739-638ce25f5a13 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:33.205645 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-54f29b72-44e2-4de4-9739-638ce25f5a13 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:33.218627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-54f29b72-44e2-4de4-9739-638ce25f5a13 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.119044 Aug 28 01:01:33.219685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 361/721] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:33 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:33.383355 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-39538b11-19ba-4be6-95fe-23a196e21a6d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:33.385736 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-39538b11-19ba-4be6-95fe-23a196e21a6d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:33.392544 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-39538b11-19ba-4be6-95fe-23a196e21a6d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:33.393042 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-39538b11-19ba-4be6-95fe-23a196e21a6d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:33.393485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-39538b11-19ba-4be6-95fe-23a196e21a6d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:33.394212 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-39538b11-19ba-4be6-95fe-23a196e21a6d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:33.394671 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-39538b11-19ba-4be6-95fe-23a196e21a6d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:33.395081 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-39538b11-19ba-4be6-95fe-23a196e21a6d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:33.429680 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-39538b11-19ba-4be6-95fe-23a196e21a6d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050495 Aug 28 01:01:33.430301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 361/722] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:33 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:33.497956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-baabe83d-7c85-44d6-8dc3-0b39fb51d78f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:33.500236 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-baabe83d-7c85-44d6-8dc3-0b39fb51d78f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:33.507221 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-baabe83d-7c85-44d6-8dc3-0b39fb51d78f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:33.507719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-baabe83d-7c85-44d6-8dc3-0b39fb51d78f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:33.508154 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-baabe83d-7c85-44d6-8dc3-0b39fb51d78f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:33.508929 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-baabe83d-7c85-44d6-8dc3-0b39fb51d78f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:33.509451 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-baabe83d-7c85-44d6-8dc3-0b39fb51d78f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:33.509911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-baabe83d-7c85-44d6-8dc3-0b39fb51d78f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:33.555907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-baabe83d-7c85-44d6-8dc3-0b39fb51d78f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061967 Aug 28 01:01:33.556727 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 362/723] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:33 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:34.249676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-dd941402-f838-476b-b493-045801f803b1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:34.250346 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-dd941402-f838-476b-b493-045801f803b1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:34.258309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dd941402-f838-476b-b493-045801f803b1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:34.258803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dd941402-f838-476b-b493-045801f803b1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:34.259235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dd941402-f838-476b-b493-045801f803b1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:34.259926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dd941402-f838-476b-b493-045801f803b1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:34.260405 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dd941402-f838-476b-b493-045801f803b1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:34.260821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dd941402-f838-476b-b493-045801f803b1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:34.329928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-dd941402-f838-476b-b493-045801f803b1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:34.338299 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-dd941402-f838-476b-b493-045801f803b1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.092809 Aug 28 01:01:34.338875 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 362/724] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:34 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:34.447311 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-11e59f73-e855-4106-beed-60a3f0bd53ce tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:34.451393 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-11e59f73-e855-4106-beed-60a3f0bd53ce tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:34.455922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=90 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:01:34.466462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-11e59f73-e855-4106-beed-60a3f0bd53ce tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:34.467285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-11e59f73-e855-4106-beed-60a3f0bd53ce tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:34.468167 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-11e59f73-e855-4106-beed-60a3f0bd53ce tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:34.469702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-11e59f73-e855-4106-beed-60a3f0bd53ce tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:34.470473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-11e59f73-e855-4106-beed-60a3f0bd53ce tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:34.471375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-11e59f73-e855-4106-beed-60a3f0bd53ce tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:34.533353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-11e59f73-e855-4106-beed-60a3f0bd53ce tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.093907 Aug 28 01:01:34.534945 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 363/725] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:34 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:34.570660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1ed407b1-97be-4ecd-9f49-1b77c3bc6651 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:34.572953 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1ed407b1-97be-4ecd-9f49-1b77c3bc6651 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:34.575040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:01:34.581200 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1ed407b1-97be-4ecd-9f49-1b77c3bc6651 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:34.581696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1ed407b1-97be-4ecd-9f49-1b77c3bc6651 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:34.582129 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1ed407b1-97be-4ecd-9f49-1b77c3bc6651 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:34.582925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1ed407b1-97be-4ecd-9f49-1b77c3bc6651 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:34.583379 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1ed407b1-97be-4ecd-9f49-1b77c3bc6651 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:34.583805 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1ed407b1-97be-4ecd-9f49-1b77c3bc6651 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:34.628134 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1ed407b1-97be-4ecd-9f49-1b77c3bc6651 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062322 Aug 28 01:01:34.628727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 363/726] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:34 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:35.358959 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4068879f-4169-46c5-98b5-958a9d63bfcb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:35.359579 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4068879f-4169-46c5-98b5-958a9d63bfcb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:35.366315 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4068879f-4169-46c5-98b5-958a9d63bfcb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:35.366886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4068879f-4169-46c5-98b5-958a9d63bfcb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:35.367500 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4068879f-4169-46c5-98b5-958a9d63bfcb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:35.368245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4068879f-4169-46c5-98b5-958a9d63bfcb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:35.368699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4068879f-4169-46c5-98b5-958a9d63bfcb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:35.369155 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4068879f-4169-46c5-98b5-958a9d63bfcb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:35.444452 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4068879f-4169-46c5-98b5-958a9d63bfcb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:35.453111 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4068879f-4169-46c5-98b5-958a9d63bfcb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.097935 Aug 28 01:01:35.453840 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 364/727] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:35 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:35.550541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-94ad17d4-79f3-42e8-a527-b9bfdc218447 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:35.552759 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-94ad17d4-79f3-42e8-a527-b9bfdc218447 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:35.560356 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94ad17d4-79f3-42e8-a527-b9bfdc218447 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:35.560849 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94ad17d4-79f3-42e8-a527-b9bfdc218447 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:35.561348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94ad17d4-79f3-42e8-a527-b9bfdc218447 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:35.562060 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94ad17d4-79f3-42e8-a527-b9bfdc218447 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:35.562507 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94ad17d4-79f3-42e8-a527-b9bfdc218447 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:35.562928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94ad17d4-79f3-42e8-a527-b9bfdc218447 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:35.596542 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-94ad17d4-79f3-42e8-a527-b9bfdc218447 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050196 Aug 28 01:01:35.597384 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 364/728] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:35 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:35.644436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-34215e11-82ce-4dd5-a527-e16c46bd7a33 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:35.646930 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-34215e11-82ce-4dd5-a527-e16c46bd7a33 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:35.654089 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-34215e11-82ce-4dd5-a527-e16c46bd7a33 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:35.654630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-34215e11-82ce-4dd5-a527-e16c46bd7a33 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:35.655073 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-34215e11-82ce-4dd5-a527-e16c46bd7a33 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:35.655865 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-34215e11-82ce-4dd5-a527-e16c46bd7a33 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:35.656394 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-34215e11-82ce-4dd5-a527-e16c46bd7a33 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:35.656857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-34215e11-82ce-4dd5-a527-e16c46bd7a33 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:35.690343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-34215e11-82ce-4dd5-a527-e16c46bd7a33 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049933 Aug 28 01:01:35.691070 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 365/729] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:35 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:36.475148 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bd3f5c9c-45a1-46b7-b714-19bbb520ef44 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:36.475765 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bd3f5c9c-45a1-46b7-b714-19bbb520ef44 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:36.482588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd3f5c9c-45a1-46b7-b714-19bbb520ef44 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:36.483078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd3f5c9c-45a1-46b7-b714-19bbb520ef44 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:36.483513 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd3f5c9c-45a1-46b7-b714-19bbb520ef44 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:36.484224 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd3f5c9c-45a1-46b7-b714-19bbb520ef44 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:36.484654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd3f5c9c-45a1-46b7-b714-19bbb520ef44 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:36.485128 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd3f5c9c-45a1-46b7-b714-19bbb520ef44 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:36.505488 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:01:36.555873 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bd3f5c9c-45a1-46b7-b714-19bbb520ef44 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:36.571894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bd3f5c9c-45a1-46b7-b714-19bbb520ef44 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.100621 Aug 28 01:01:36.572892 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 365/730] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:36 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:36.609996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2ad37944-d3eb-4625-83bc-e0dc24c02fb1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:36.612202 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2ad37944-d3eb-4625-83bc-e0dc24c02fb1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:36.619806 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ad37944-d3eb-4625-83bc-e0dc24c02fb1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:36.620301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ad37944-d3eb-4625-83bc-e0dc24c02fb1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:36.620737 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ad37944-d3eb-4625-83bc-e0dc24c02fb1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:36.621510 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ad37944-d3eb-4625-83bc-e0dc24c02fb1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:36.621959 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ad37944-d3eb-4625-83bc-e0dc24c02fb1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:36.622379 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ad37944-d3eb-4625-83bc-e0dc24c02fb1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:36.658885 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2ad37944-d3eb-4625-83bc-e0dc24c02fb1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052985 Aug 28 01:01:36.659471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 366/731] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:36 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:36.703791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3d3f6663-351e-4a54-9e95-00708e85e431 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:36.706097 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3d3f6663-351e-4a54-9e95-00708e85e431 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:36.712690 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d3f6663-351e-4a54-9e95-00708e85e431 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:36.713231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d3f6663-351e-4a54-9e95-00708e85e431 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:36.713684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d3f6663-351e-4a54-9e95-00708e85e431 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:36.714396 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d3f6663-351e-4a54-9e95-00708e85e431 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:36.714850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d3f6663-351e-4a54-9e95-00708e85e431 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:36.715272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d3f6663-351e-4a54-9e95-00708e85e431 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:36.749341 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3d3f6663-351e-4a54-9e95-00708e85e431 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049334 Aug 28 01:01:36.749846 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 366/732] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:36 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:37.604621 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-03b1b0c1-6621-4133-b605-c15f1d3f8a65 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:37.605279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-03b1b0c1-6621-4133-b605-c15f1d3f8a65 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:37.612226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03b1b0c1-6621-4133-b605-c15f1d3f8a65 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:37.612921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03b1b0c1-6621-4133-b605-c15f1d3f8a65 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:37.613200 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03b1b0c1-6621-4133-b605-c15f1d3f8a65 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:37.614029 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03b1b0c1-6621-4133-b605-c15f1d3f8a65 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:37.614490 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03b1b0c1-6621-4133-b605-c15f1d3f8a65 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:37.614894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03b1b0c1-6621-4133-b605-c15f1d3f8a65 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:37.636768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:01:37.672520 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-91525355-e019-4ecd-8e7c-b57e0422669c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:37.674808 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-91525355-e019-4ecd-8e7c-b57e0422669c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:37.699064 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-91525355-e019-4ecd-8e7c-b57e0422669c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:37.699574 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-91525355-e019-4ecd-8e7c-b57e0422669c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:37.700009 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-91525355-e019-4ecd-8e7c-b57e0422669c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:37.700712 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-91525355-e019-4ecd-8e7c-b57e0422669c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:37.702248 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-91525355-e019-4ecd-8e7c-b57e0422669c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:37.702248 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-91525355-e019-4ecd-8e7c-b57e0422669c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:37.704556 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-03b1b0c1-6621-4133-b605-c15f1d3f8a65 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:37.714732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-03b1b0c1-6621-4133-b605-c15f1d3f8a65 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.113830 Aug 28 01:01:37.715536 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 367/733] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:37 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:37.740784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-91525355-e019-4ecd-8e7c-b57e0422669c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.072489 Aug 28 01:01:37.741610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 367/734] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:37 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:37.766424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d55cc662-6216-4681-952a-a334fd585aca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:37.770435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d55cc662-6216-4681-952a-a334fd585aca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:37.781499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d55cc662-6216-4681-952a-a334fd585aca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:37.782340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d55cc662-6216-4681-952a-a334fd585aca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:37.783099 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d55cc662-6216-4681-952a-a334fd585aca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:37.784348 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d55cc662-6216-4681-952a-a334fd585aca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:37.785190 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d55cc662-6216-4681-952a-a334fd585aca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:37.786075 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d55cc662-6216-4681-952a-a334fd585aca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:37.846873 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d55cc662-6216-4681-952a-a334fd585aca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.087755 Aug 28 01:01:37.847824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 368/735] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:37 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:38.740045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f96d1761-b939-4204-bf24-b81663c0739f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:38.741197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f96d1761-b939-4204-bf24-b81663c0739f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:38.752348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f96d1761-b939-4204-bf24-b81663c0739f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:38.753232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f96d1761-b939-4204-bf24-b81663c0739f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:38.754006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f96d1761-b939-4204-bf24-b81663c0739f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:38.755331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f96d1761-b939-4204-bf24-b81663c0739f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:38.756132 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f96d1761-b939-4204-bf24-b81663c0739f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:38.756314 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d4d52a1e-cf97-475a-a9fa-de067b79c99f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:38.756813 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f96d1761-b939-4204-bf24-b81663c0739f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:38.758615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d4d52a1e-cf97-475a-a9fa-de067b79c99f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:38.766297 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4d52a1e-cf97-475a-a9fa-de067b79c99f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:38.766835 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4d52a1e-cf97-475a-a9fa-de067b79c99f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:38.767284 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4d52a1e-cf97-475a-a9fa-de067b79c99f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:38.768012 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4d52a1e-cf97-475a-a9fa-de067b79c99f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:38.768447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4d52a1e-cf97-475a-a9fa-de067b79c99f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:38.768873 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4d52a1e-cf97-475a-a9fa-de067b79c99f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:38.804035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d4d52a1e-cf97-475a-a9fa-de067b79c99f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051895 Aug 28 01:01:38.804621 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 369/736] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:38 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:38.853160 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f96d1761-b939-4204-bf24-b81663c0739f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:38.862971 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f96d1761-b939-4204-bf24-b81663c0739f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.127760 Aug 28 01:01:38.863250 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-24749823-e521-425a-a138-5524b4076db2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:38.863568 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 368/737] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:38 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:38.865908 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-24749823-e521-425a-a138-5524b4076db2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:38.873562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24749823-e521-425a-a138-5524b4076db2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:38.874092 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24749823-e521-425a-a138-5524b4076db2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:38.874494 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24749823-e521-425a-a138-5524b4076db2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:38.875254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24749823-e521-425a-a138-5524b4076db2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:38.875783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24749823-e521-425a-a138-5524b4076db2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:38.876143 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24749823-e521-425a-a138-5524b4076db2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:38.909876 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-24749823-e521-425a-a138-5524b4076db2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051255 Aug 28 01:01:38.910478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 370/738] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:38 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:39.821234 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-38adf488-a95a-4fc7-9085-afd0db839a5a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:39.823771 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-38adf488-a95a-4fc7-9085-afd0db839a5a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:39.831289 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38adf488-a95a-4fc7-9085-afd0db839a5a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:39.831792 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38adf488-a95a-4fc7-9085-afd0db839a5a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:39.832227 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38adf488-a95a-4fc7-9085-afd0db839a5a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:39.833151 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38adf488-a95a-4fc7-9085-afd0db839a5a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:39.833610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38adf488-a95a-4fc7-9085-afd0db839a5a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:39.834047 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38adf488-a95a-4fc7-9085-afd0db839a5a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:39.868120 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-38adf488-a95a-4fc7-9085-afd0db839a5a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051943 Aug 28 01:01:39.868726 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 369/739] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:39 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:39.896209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fc1de970-b512-47cf-b711-2970483055f3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:39.896902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fc1de970-b512-47cf-b711-2970483055f3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:39.903911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc1de970-b512-47cf-b711-2970483055f3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:39.904400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc1de970-b512-47cf-b711-2970483055f3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:39.904817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc1de970-b512-47cf-b711-2970483055f3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:39.905575 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc1de970-b512-47cf-b711-2970483055f3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:39.906064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc1de970-b512-47cf-b711-2970483055f3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:39.906477 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc1de970-b512-47cf-b711-2970483055f3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:39.924308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8467a53c-bb99-43db-8405-7baf716281ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:39.926671 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8467a53c-bb99-43db-8405-7baf716281ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:39.933235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8467a53c-bb99-43db-8405-7baf716281ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:39.933730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8467a53c-bb99-43db-8405-7baf716281ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:39.934171 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8467a53c-bb99-43db-8405-7baf716281ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:39.935020 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8467a53c-bb99-43db-8405-7baf716281ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:39.935475 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8467a53c-bb99-43db-8405-7baf716281ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:39.935900 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8467a53c-bb99-43db-8405-7baf716281ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:39.970400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8467a53c-bb99-43db-8405-7baf716281ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050822 Aug 28 01:01:39.975149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 370/740] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:39 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:39.986710 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fc1de970-b512-47cf-b711-2970483055f3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:39.999090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fc1de970-b512-47cf-b711-2970483055f3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.106844 Aug 28 01:01:39.999899 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 371/741] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:39 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:40.881078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c7dfccda-3c21-486f-9ba2-c4ede4cecd41 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:40.883866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c7dfccda-3c21-486f-9ba2-c4ede4cecd41 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:40.891559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c7dfccda-3c21-486f-9ba2-c4ede4cecd41 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:40.892104 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c7dfccda-3c21-486f-9ba2-c4ede4cecd41 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:40.892660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c7dfccda-3c21-486f-9ba2-c4ede4cecd41 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:40.893529 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c7dfccda-3c21-486f-9ba2-c4ede4cecd41 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:40.894026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c7dfccda-3c21-486f-9ba2-c4ede4cecd41 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:40.894538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c7dfccda-3c21-486f-9ba2-c4ede4cecd41 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:40.929229 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c7dfccda-3c21-486f-9ba2-c4ede4cecd41 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052269 Aug 28 01:01:40.929870 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 371/742] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:40 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:40.983807 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6ae03de4-fde6-4239-a6e6-6631290c97d6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:40.986301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6ae03de4-fde6-4239-a6e6-6631290c97d6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:40.993898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ae03de4-fde6-4239-a6e6-6631290c97d6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:40.994389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ae03de4-fde6-4239-a6e6-6631290c97d6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:40.994817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ae03de4-fde6-4239-a6e6-6631290c97d6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:40.995555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ae03de4-fde6-4239-a6e6-6631290c97d6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:40.996004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ae03de4-fde6-4239-a6e6-6631290c97d6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:40.996426 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ae03de4-fde6-4239-a6e6-6631290c97d6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:41.028213 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c83d6139-8048-40dc-b723-db0a73e5b660 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:41.029267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c83d6139-8048-40dc-b723-db0a73e5b660 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:41.038767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6ae03de4-fde6-4239-a6e6-6631290c97d6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058895 Aug 28 01:01:41.039424 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c83d6139-8048-40dc-b723-db0a73e5b660 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:41.039732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 372/743] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:40 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:41.040100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c83d6139-8048-40dc-b723-db0a73e5b660 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:41.040589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c83d6139-8048-40dc-b723-db0a73e5b660 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:41.041439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c83d6139-8048-40dc-b723-db0a73e5b660 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:41.042065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c83d6139-8048-40dc-b723-db0a73e5b660 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:41.042618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c83d6139-8048-40dc-b723-db0a73e5b660 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:41.120606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c83d6139-8048-40dc-b723-db0a73e5b660 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:41.135110 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c83d6139-8048-40dc-b723-db0a73e5b660 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.113317 Aug 28 01:01:41.135854 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 372/744] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:41 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:01:41.943303 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4b3dc9cb-9906-4153-b069-614e97cc2a88 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:41.945875 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4b3dc9cb-9906-4153-b069-614e97cc2a88 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:41.952926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b3dc9cb-9906-4153-b069-614e97cc2a88 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:41.953598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b3dc9cb-9906-4153-b069-614e97cc2a88 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:41.954123 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b3dc9cb-9906-4153-b069-614e97cc2a88 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:41.954955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b3dc9cb-9906-4153-b069-614e97cc2a88 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:41.955830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b3dc9cb-9906-4153-b069-614e97cc2a88 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:41.956377 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b3dc9cb-9906-4153-b069-614e97cc2a88 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:41.991640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4b3dc9cb-9906-4153-b069-614e97cc2a88 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052713 Aug 28 01:01:41.992416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 373/745] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:41 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:42.053424 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5593c5da-5c09-4c2b-a473-24c298acc884 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:42.055685 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5593c5da-5c09-4c2b-a473-24c298acc884 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:42.063682 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5593c5da-5c09-4c2b-a473-24c298acc884 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:42.064186 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5593c5da-5c09-4c2b-a473-24c298acc884 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:42.064612 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5593c5da-5c09-4c2b-a473-24c298acc884 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:42.065369 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5593c5da-5c09-4c2b-a473-24c298acc884 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:42.065819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5593c5da-5c09-4c2b-a473-24c298acc884 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:42.066234 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5593c5da-5c09-4c2b-a473-24c298acc884 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:42.102664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5593c5da-5c09-4c2b-a473-24c298acc884 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053631 Aug 28 01:01:42.103257 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 373/746] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:42 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:42.156533 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2b3281ae-acc0-4b03-b32c-0a6e81f131b2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:42.157163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2b3281ae-acc0-4b03-b32c-0a6e81f131b2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:42.163843 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2b3281ae-acc0-4b03-b32c-0a6e81f131b2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:42.164331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2b3281ae-acc0-4b03-b32c-0a6e81f131b2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:42.164765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2b3281ae-acc0-4b03-b32c-0a6e81f131b2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:42.165545 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2b3281ae-acc0-4b03-b32c-0a6e81f131b2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:42.166103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2b3281ae-acc0-4b03-b32c-0a6e81f131b2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:42.166535 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2b3281ae-acc0-4b03-b32c-0a6e81f131b2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:42.235829 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2b3281ae-acc0-4b03-b32c-0a6e81f131b2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:42.244077 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2b3281ae-acc0-4b03-b32c-0a6e81f131b2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.091381 Aug 28 01:01:42.244666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 374/747] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:42 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:43.004852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-24a85d28-50cf-4fd8-b544-440900a2126c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:43.007261 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-24a85d28-50cf-4fd8-b544-440900a2126c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:43.013997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24a85d28-50cf-4fd8-b544-440900a2126c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:43.014494 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24a85d28-50cf-4fd8-b544-440900a2126c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:43.014930 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24a85d28-50cf-4fd8-b544-440900a2126c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:43.015661 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24a85d28-50cf-4fd8-b544-440900a2126c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:43.016110 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24a85d28-50cf-4fd8-b544-440900a2126c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:43.016538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24a85d28-50cf-4fd8-b544-440900a2126c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:43.051100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-24a85d28-50cf-4fd8-b544-440900a2126c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050723 Aug 28 01:01:43.051720 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 374/748] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:42 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:43.116267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-78535f3f-9905-48c8-bbb1-d1a18ce665bc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:43.119483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-78535f3f-9905-48c8-bbb1-d1a18ce665bc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:43.126854 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-78535f3f-9905-48c8-bbb1-d1a18ce665bc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:43.127381 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-78535f3f-9905-48c8-bbb1-d1a18ce665bc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:43.127835 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-78535f3f-9905-48c8-bbb1-d1a18ce665bc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:43.128651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-78535f3f-9905-48c8-bbb1-d1a18ce665bc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:43.129117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-78535f3f-9905-48c8-bbb1-d1a18ce665bc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:43.129631 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-78535f3f-9905-48c8-bbb1-d1a18ce665bc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:43.166346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-78535f3f-9905-48c8-bbb1-d1a18ce665bc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054370 Aug 28 01:01:43.166972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 375/749] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:43 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:43.266555 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-20062520-9718-49d3-80c7-f397243b25b6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:43.267149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-20062520-9718-49d3-80c7-f397243b25b6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:43.274581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20062520-9718-49d3-80c7-f397243b25b6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:43.275069 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20062520-9718-49d3-80c7-f397243b25b6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:43.275709 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20062520-9718-49d3-80c7-f397243b25b6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:43.276526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20062520-9718-49d3-80c7-f397243b25b6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:43.277060 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20062520-9718-49d3-80c7-f397243b25b6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:43.277485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20062520-9718-49d3-80c7-f397243b25b6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:43.349405 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-20062520-9718-49d3-80c7-f397243b25b6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:43.358906 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-20062520-9718-49d3-80c7-f397243b25b6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.096820 Aug 28 01:01:43.359584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 375/750] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:43 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:44.068798 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-979b0831-0351-46fd-945d-055c05d7d4a2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:44.072529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-979b0831-0351-46fd-945d-055c05d7d4a2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:44.083995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-979b0831-0351-46fd-945d-055c05d7d4a2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:44.084879 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-979b0831-0351-46fd-945d-055c05d7d4a2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:44.085718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-979b0831-0351-46fd-945d-055c05d7d4a2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:44.087061 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-979b0831-0351-46fd-945d-055c05d7d4a2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:44.088051 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-979b0831-0351-46fd-945d-055c05d7d4a2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:44.088761 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-979b0831-0351-46fd-945d-055c05d7d4a2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:44.149575 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-979b0831-0351-46fd-945d-055c05d7d4a2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.088408 Aug 28 01:01:44.150615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 376/751] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:44 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:44.179949 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d9690e28-effd-48ea-ab40-efa90f5037da tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:44.182274 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d9690e28-effd-48ea-ab40-efa90f5037da tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:44.189334 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d9690e28-effd-48ea-ab40-efa90f5037da tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:44.189831 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d9690e28-effd-48ea-ab40-efa90f5037da tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:44.190272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d9690e28-effd-48ea-ab40-efa90f5037da tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:44.191001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d9690e28-effd-48ea-ab40-efa90f5037da tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:44.191567 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d9690e28-effd-48ea-ab40-efa90f5037da tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:44.191985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d9690e28-effd-48ea-ab40-efa90f5037da tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:44.225582 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d9690e28-effd-48ea-ab40-efa90f5037da tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049851 Aug 28 01:01:44.226175 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 376/752] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:44 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:44.393431 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e6d8d43e-5385-4f5f-984b-82eb2349118d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:44.394102 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e6d8d43e-5385-4f5f-984b-82eb2349118d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:44.401006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6d8d43e-5385-4f5f-984b-82eb2349118d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:44.401517 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6d8d43e-5385-4f5f-984b-82eb2349118d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:44.401952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6d8d43e-5385-4f5f-984b-82eb2349118d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:44.402652 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6d8d43e-5385-4f5f-984b-82eb2349118d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:44.403107 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6d8d43e-5385-4f5f-984b-82eb2349118d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:44.403529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6d8d43e-5385-4f5f-984b-82eb2349118d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:44.492486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e6d8d43e-5385-4f5f-984b-82eb2349118d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:44.502545 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e6d8d43e-5385-4f5f-984b-82eb2349118d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.113127 Aug 28 01:01:44.503267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 377/753] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:44 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:45.168355 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4f32e671-efe2-417e-be91-fac0f07396e6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:45.172232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4f32e671-efe2-417e-be91-fac0f07396e6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:45.184875 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4f32e671-efe2-417e-be91-fac0f07396e6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:45.185827 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4f32e671-efe2-417e-be91-fac0f07396e6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:45.186644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4f32e671-efe2-417e-be91-fac0f07396e6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:45.188059 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4f32e671-efe2-417e-be91-fac0f07396e6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:45.188911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4f32e671-efe2-417e-be91-fac0f07396e6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:45.189677 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4f32e671-efe2-417e-be91-fac0f07396e6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:45.228127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4f32e671-efe2-417e-be91-fac0f07396e6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.067724 Aug 28 01:01:45.228735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 377/754] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:45 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:45.239021 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-299fc2b9-77a1-47d7-9cff-269abbf4629f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:45.241424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-299fc2b9-77a1-47d7-9cff-269abbf4629f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:45.249112 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-299fc2b9-77a1-47d7-9cff-269abbf4629f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:45.249664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-299fc2b9-77a1-47d7-9cff-269abbf4629f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:45.250117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-299fc2b9-77a1-47d7-9cff-269abbf4629f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:45.250890 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-299fc2b9-77a1-47d7-9cff-269abbf4629f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:45.251344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-299fc2b9-77a1-47d7-9cff-269abbf4629f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:45.251762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-299fc2b9-77a1-47d7-9cff-269abbf4629f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:45.288432 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-299fc2b9-77a1-47d7-9cff-269abbf4629f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053562 Aug 28 01:01:45.289058 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 378/755] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:45 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:45.528250 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c60032a7-34ad-4ff0-aa02-e2a50c7495fe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:45.529450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c60032a7-34ad-4ff0-aa02-e2a50c7495fe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:45.540199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c60032a7-34ad-4ff0-aa02-e2a50c7495fe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:45.541073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c60032a7-34ad-4ff0-aa02-e2a50c7495fe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:45.541806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c60032a7-34ad-4ff0-aa02-e2a50c7495fe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:45.543096 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c60032a7-34ad-4ff0-aa02-e2a50c7495fe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:45.543916 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c60032a7-34ad-4ff0-aa02-e2a50c7495fe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:45.544678 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c60032a7-34ad-4ff0-aa02-e2a50c7495fe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:45.638748 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c60032a7-34ad-4ff0-aa02-e2a50c7495fe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:45.647637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c60032a7-34ad-4ff0-aa02-e2a50c7495fe tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.126978 Aug 28 01:01:45.648286 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 378/756] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:45 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:46.241604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-76d09616-a9e7-43f7-8df2-de88304f49a9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:46.244175 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-76d09616-a9e7-43f7-8df2-de88304f49a9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:46.252442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76d09616-a9e7-43f7-8df2-de88304f49a9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:46.252939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76d09616-a9e7-43f7-8df2-de88304f49a9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:46.253414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76d09616-a9e7-43f7-8df2-de88304f49a9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:46.254271 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76d09616-a9e7-43f7-8df2-de88304f49a9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:46.254751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76d09616-a9e7-43f7-8df2-de88304f49a9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:46.255174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76d09616-a9e7-43f7-8df2-de88304f49a9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:46.290719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-76d09616-a9e7-43f7-8df2-de88304f49a9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053293 Aug 28 01:01:46.291309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 379/757] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:46 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:46.302044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-461dfe18-c943-4b45-bd14-6aaf241f3e07 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:46.304304 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-461dfe18-c943-4b45-bd14-6aaf241f3e07 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:46.311723 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-461dfe18-c943-4b45-bd14-6aaf241f3e07 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:46.312219 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-461dfe18-c943-4b45-bd14-6aaf241f3e07 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:46.312650 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-461dfe18-c943-4b45-bd14-6aaf241f3e07 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:46.313482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-461dfe18-c943-4b45-bd14-6aaf241f3e07 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:46.313936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-461dfe18-c943-4b45-bd14-6aaf241f3e07 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:46.314374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-461dfe18-c943-4b45-bd14-6aaf241f3e07 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:46.367754 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-461dfe18-c943-4b45-bd14-6aaf241f3e07 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.069729 Aug 28 01:01:46.368536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 379/758] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:46 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:46.669186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e84a9728-5a0b-4d1e-9771-36ff73890720 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:46.669940 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e84a9728-5a0b-4d1e-9771-36ff73890720 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:46.677883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e84a9728-5a0b-4d1e-9771-36ff73890720 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:46.678373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e84a9728-5a0b-4d1e-9771-36ff73890720 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:46.678799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e84a9728-5a0b-4d1e-9771-36ff73890720 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:46.679619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e84a9728-5a0b-4d1e-9771-36ff73890720 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:46.679959 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e84a9728-5a0b-4d1e-9771-36ff73890720 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:46.680377 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e84a9728-5a0b-4d1e-9771-36ff73890720 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:46.749539 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e84a9728-5a0b-4d1e-9771-36ff73890720 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:46.759036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e84a9728-5a0b-4d1e-9771-36ff73890720 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.093724 Aug 28 01:01:46.759775 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 380/759] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:46 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:47.303737 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-83b6f3d4-dd92-48c4-be6f-1f81a44c5ec9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:47.306415 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-83b6f3d4-dd92-48c4-be6f-1f81a44c5ec9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:47.314268 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-83b6f3d4-dd92-48c4-be6f-1f81a44c5ec9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:47.314910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-83b6f3d4-dd92-48c4-be6f-1f81a44c5ec9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:47.315372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-83b6f3d4-dd92-48c4-be6f-1f81a44c5ec9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:47.316271 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-83b6f3d4-dd92-48c4-be6f-1f81a44c5ec9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:47.316907 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-83b6f3d4-dd92-48c4-be6f-1f81a44c5ec9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:47.317443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-83b6f3d4-dd92-48c4-be6f-1f81a44c5ec9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:47.357709 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-83b6f3d4-dd92-48c4-be6f-1f81a44c5ec9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058071 Aug 28 01:01:47.358330 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 380/760] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:47 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:47.381108 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0a8b4d27-bfd7-4029-a48a-0bb596fea928 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:47.383325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0a8b4d27-bfd7-4029-a48a-0bb596fea928 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:47.390416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a8b4d27-bfd7-4029-a48a-0bb596fea928 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:47.390992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a8b4d27-bfd7-4029-a48a-0bb596fea928 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:47.391467 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a8b4d27-bfd7-4029-a48a-0bb596fea928 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:47.392308 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a8b4d27-bfd7-4029-a48a-0bb596fea928 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:47.392797 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a8b4d27-bfd7-4029-a48a-0bb596fea928 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:47.393406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a8b4d27-bfd7-4029-a48a-0bb596fea928 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:47.426630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0a8b4d27-bfd7-4029-a48a-0bb596fea928 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049490 Aug 28 01:01:47.427196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 381/761] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:47 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:01:47.780982 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7e1239e1-8821-435c-983d-541a8c8b3f8f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:47.781641 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7e1239e1-8821-435c-983d-541a8c8b3f8f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:47.788933 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e1239e1-8821-435c-983d-541a8c8b3f8f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:47.789462 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e1239e1-8821-435c-983d-541a8c8b3f8f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:47.789910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e1239e1-8821-435c-983d-541a8c8b3f8f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:47.790615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e1239e1-8821-435c-983d-541a8c8b3f8f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:47.791055 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e1239e1-8821-435c-983d-541a8c8b3f8f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:47.791496 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e1239e1-8821-435c-983d-541a8c8b3f8f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:47.860456 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7e1239e1-8821-435c-983d-541a8c8b3f8f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:47.871005 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7e1239e1-8821-435c-983d-541a8c8b3f8f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.093889 Aug 28 01:01:47.871687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 381/762] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:47 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:48.373660 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e3160de4-430a-428e-8a4a-50f79491afc7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:48.375746 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e3160de4-430a-428e-8a4a-50f79491afc7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:48.382554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e3160de4-430a-428e-8a4a-50f79491afc7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:48.383286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e3160de4-430a-428e-8a4a-50f79491afc7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:48.383435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e3160de4-430a-428e-8a4a-50f79491afc7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:48.384152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e3160de4-430a-428e-8a4a-50f79491afc7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:48.384592 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e3160de4-430a-428e-8a4a-50f79491afc7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:48.385049 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e3160de4-430a-428e-8a4a-50f79491afc7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:48.429264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e3160de4-430a-428e-8a4a-50f79491afc7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.059645 Aug 28 01:01:48.430262 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 382/763] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:48 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:48.439695 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-52554933-6d82-47c4-9984-7719e01dec33 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:48.442097 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-52554933-6d82-47c4-9984-7719e01dec33 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:48.449926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-52554933-6d82-47c4-9984-7719e01dec33 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:48.450457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-52554933-6d82-47c4-9984-7719e01dec33 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:48.450893 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-52554933-6d82-47c4-9984-7719e01dec33 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:48.451618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-52554933-6d82-47c4-9984-7719e01dec33 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:48.452065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-52554933-6d82-47c4-9984-7719e01dec33 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:48.452482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-52554933-6d82-47c4-9984-7719e01dec33 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:48.456807 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:01:48.486757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-52554933-6d82-47c4-9984-7719e01dec33 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051404 Aug 28 01:01:48.487333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 382/764] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:48 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:48.893025 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-44269ff0-30c3-47b3-92cc-e7f391a13607 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:48.893582 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-44269ff0-30c3-47b3-92cc-e7f391a13607 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:48.901138 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-44269ff0-30c3-47b3-92cc-e7f391a13607 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:48.901628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-44269ff0-30c3-47b3-92cc-e7f391a13607 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:48.902059 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-44269ff0-30c3-47b3-92cc-e7f391a13607 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:48.902774 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-44269ff0-30c3-47b3-92cc-e7f391a13607 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:48.903224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-44269ff0-30c3-47b3-92cc-e7f391a13607 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:48.903620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-44269ff0-30c3-47b3-92cc-e7f391a13607 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:48.979057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-44269ff0-30c3-47b3-92cc-e7f391a13607 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:48.988365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-44269ff0-30c3-47b3-92cc-e7f391a13607 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.099910 Aug 28 01:01:48.988992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 383/765] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:48 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:49.446924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0e62f036-6a67-4b26-b906-d450c645be65 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:49.449289 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0e62f036-6a67-4b26-b906-d450c645be65 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:49.456523 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e62f036-6a67-4b26-b906-d450c645be65 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:49.457063 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e62f036-6a67-4b26-b906-d450c645be65 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:49.457610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e62f036-6a67-4b26-b906-d450c645be65 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:49.458366 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e62f036-6a67-4b26-b906-d450c645be65 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:49.458831 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e62f036-6a67-4b26-b906-d450c645be65 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:49.459247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e62f036-6a67-4b26-b906-d450c645be65 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:49.493444 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0e62f036-6a67-4b26-b906-d450c645be65 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051015 Aug 28 01:01:49.494585 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 383/766] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:49 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:49.509385 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b160e0b8-e7ca-4615-9493-5af49ad888ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:49.514605 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b160e0b8-e7ca-4615-9493-5af49ad888ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:49.525294 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b160e0b8-e7ca-4615-9493-5af49ad888ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:49.525858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b160e0b8-e7ca-4615-9493-5af49ad888ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:49.526841 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b160e0b8-e7ca-4615-9493-5af49ad888ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:49.527212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b160e0b8-e7ca-4615-9493-5af49ad888ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:49.527724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b160e0b8-e7ca-4615-9493-5af49ad888ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:49.528154 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b160e0b8-e7ca-4615-9493-5af49ad888ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:49.532670 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:01:49.569807 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b160e0b8-e7ca-4615-9493-5af49ad888ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.069767 Aug 28 01:01:49.570589 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 384/767] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:49 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:50.010260 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b210ed3b-03ea-475e-8cfd-44efcc2b978e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:50.011058 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b210ed3b-03ea-475e-8cfd-44efcc2b978e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:50.018969 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b210ed3b-03ea-475e-8cfd-44efcc2b978e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:50.019468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b210ed3b-03ea-475e-8cfd-44efcc2b978e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:50.020031 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b210ed3b-03ea-475e-8cfd-44efcc2b978e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:50.020793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b210ed3b-03ea-475e-8cfd-44efcc2b978e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:50.021278 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b210ed3b-03ea-475e-8cfd-44efcc2b978e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:50.021727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b210ed3b-03ea-475e-8cfd-44efcc2b978e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:50.091692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b210ed3b-03ea-475e-8cfd-44efcc2b978e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:50.100488 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b210ed3b-03ea-475e-8cfd-44efcc2b978e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.094527 Aug 28 01:01:50.101117 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 384/768] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:50 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:50.507319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-eb2e0155-c900-45b3-92b2-45fedd2eb0ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:50.509817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-eb2e0155-c900-45b3-92b2-45fedd2eb0ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:50.516632 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb2e0155-c900-45b3-92b2-45fedd2eb0ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:50.517139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb2e0155-c900-45b3-92b2-45fedd2eb0ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:50.517594 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb2e0155-c900-45b3-92b2-45fedd2eb0ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:50.518319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb2e0155-c900-45b3-92b2-45fedd2eb0ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:50.518764 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb2e0155-c900-45b3-92b2-45fedd2eb0ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:50.519179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb2e0155-c900-45b3-92b2-45fedd2eb0ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:50.551485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-eb2e0155-c900-45b3-92b2-45fedd2eb0ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048385 Aug 28 01:01:50.552090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 385/769] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:50 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:50.583404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a3963158-6092-4b0b-9a72-c1216d44cdc7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:50.585700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a3963158-6092-4b0b-9a72-c1216d44cdc7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:50.592155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3963158-6092-4b0b-9a72-c1216d44cdc7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:50.592641 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3963158-6092-4b0b-9a72-c1216d44cdc7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:50.593095 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3963158-6092-4b0b-9a72-c1216d44cdc7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:50.593791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3963158-6092-4b0b-9a72-c1216d44cdc7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:50.594244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3963158-6092-4b0b-9a72-c1216d44cdc7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:50.594652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3963158-6092-4b0b-9a72-c1216d44cdc7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:50.630871 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a3963158-6092-4b0b-9a72-c1216d44cdc7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052199 Aug 28 01:01:50.631463 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 385/770] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:50 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:51.134977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1eaf5a7f-b110-4f44-97ab-74c7a0d43907 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:51.135748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1eaf5a7f-b110-4f44-97ab-74c7a0d43907 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:51.145822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1eaf5a7f-b110-4f44-97ab-74c7a0d43907 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:51.146534 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1eaf5a7f-b110-4f44-97ab-74c7a0d43907 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:51.147134 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1eaf5a7f-b110-4f44-97ab-74c7a0d43907 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:51.148107 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1eaf5a7f-b110-4f44-97ab-74c7a0d43907 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:51.148906 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1eaf5a7f-b110-4f44-97ab-74c7a0d43907 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:51.149719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1eaf5a7f-b110-4f44-97ab-74c7a0d43907 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:51.243247 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1eaf5a7f-b110-4f44-97ab-74c7a0d43907 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:51.252626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1eaf5a7f-b110-4f44-97ab-74c7a0d43907 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.122319 Aug 28 01:01:51.253423 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 386/771] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:51 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:51.571536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6b60b62a-af04-456e-9cff-8b9344ab860e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:51.575245 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6b60b62a-af04-456e-9cff-8b9344ab860e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:51.587052 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b60b62a-af04-456e-9cff-8b9344ab860e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:51.587922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b60b62a-af04-456e-9cff-8b9344ab860e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:51.588681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b60b62a-af04-456e-9cff-8b9344ab860e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:51.589986 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b60b62a-af04-456e-9cff-8b9344ab860e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:51.591041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b60b62a-af04-456e-9cff-8b9344ab860e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:51.591808 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b60b62a-af04-456e-9cff-8b9344ab860e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:51.644416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2744f1ab-00c9-4186-8183-14d10bc9f65b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:51.646858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2744f1ab-00c9-4186-8183-14d10bc9f65b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:51.653138 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6b60b62a-af04-456e-9cff-8b9344ab860e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.089413 Aug 28 01:01:51.654298 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 386/772] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:51 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:51.654901 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2744f1ab-00c9-4186-8183-14d10bc9f65b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:51.655502 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2744f1ab-00c9-4186-8183-14d10bc9f65b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:51.655968 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2744f1ab-00c9-4186-8183-14d10bc9f65b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:51.656758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2744f1ab-00c9-4186-8183-14d10bc9f65b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:51.657249 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2744f1ab-00c9-4186-8183-14d10bc9f65b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:51.657678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2744f1ab-00c9-4186-8183-14d10bc9f65b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:51.690757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2744f1ab-00c9-4186-8183-14d10bc9f65b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051291 Aug 28 01:01:51.691345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 387/773] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:51 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:52.274456 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-260b5b31-b12e-4ff5-ae6d-1591ec922351 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:52.275097 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-260b5b31-b12e-4ff5-ae6d-1591ec922351 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:52.282306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-260b5b31-b12e-4ff5-ae6d-1591ec922351 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:52.282790 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-260b5b31-b12e-4ff5-ae6d-1591ec922351 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:52.283219 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-260b5b31-b12e-4ff5-ae6d-1591ec922351 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:52.284000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-260b5b31-b12e-4ff5-ae6d-1591ec922351 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:52.284461 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-260b5b31-b12e-4ff5-ae6d-1591ec922351 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:52.284898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-260b5b31-b12e-4ff5-ae6d-1591ec922351 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:52.356530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-260b5b31-b12e-4ff5-ae6d-1591ec922351 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:52.368062 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-260b5b31-b12e-4ff5-ae6d-1591ec922351 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.097552 Aug 28 01:01:52.369064 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 387/774] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:52 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:01:52.669936 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-aee802e8-8a73-452d-9a72-1242abb4d974 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:52.672159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-aee802e8-8a73-452d-9a72-1242abb4d974 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:52.679514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aee802e8-8a73-452d-9a72-1242abb4d974 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:52.680063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aee802e8-8a73-452d-9a72-1242abb4d974 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:52.680540 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aee802e8-8a73-452d-9a72-1242abb4d974 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:52.681417 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aee802e8-8a73-452d-9a72-1242abb4d974 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:52.681923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aee802e8-8a73-452d-9a72-1242abb4d974 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:52.682395 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aee802e8-8a73-452d-9a72-1242abb4d974 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:52.704480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0efef9c2-e7ce-43cb-b626-77c34c29d449 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:52.706823 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0efef9c2-e7ce-43cb-b626-77c34c29d449 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:52.715800 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0efef9c2-e7ce-43cb-b626-77c34c29d449 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:52.716580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0efef9c2-e7ce-43cb-b626-77c34c29d449 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:52.717044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0efef9c2-e7ce-43cb-b626-77c34c29d449 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:52.717775 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0efef9c2-e7ce-43cb-b626-77c34c29d449 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:52.718231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0efef9c2-e7ce-43cb-b626-77c34c29d449 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:52.718663 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0efef9c2-e7ce-43cb-b626-77c34c29d449 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:52.733079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-aee802e8-8a73-452d-9a72-1242abb4d974 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.066786 Aug 28 01:01:52.734011 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 388/775] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:52 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:52.752744 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0efef9c2-e7ce-43cb-b626-77c34c29d449 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052421 Aug 28 01:01:52.753372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 388/776] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:52 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:53.408519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-24ab0399-23dd-40c8-8798-df1ff02a8a8a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:53.410020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-24ab0399-23dd-40c8-8798-df1ff02a8a8a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:53.424073 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24ab0399-23dd-40c8-8798-df1ff02a8a8a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:53.425188 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24ab0399-23dd-40c8-8798-df1ff02a8a8a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:53.426124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24ab0399-23dd-40c8-8798-df1ff02a8a8a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:53.427753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24ab0399-23dd-40c8-8798-df1ff02a8a8a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:53.428688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24ab0399-23dd-40c8-8798-df1ff02a8a8a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:53.429648 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24ab0399-23dd-40c8-8798-df1ff02a8a8a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:53.753804 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f9fc4449-48c5-4e5e-9e3d-aa6d089ee164 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:53.756811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f9fc4449-48c5-4e5e-9e3d-aa6d089ee164 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:53.766205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9fc4449-48c5-4e5e-9e3d-aa6d089ee164 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:53.766861 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9fc4449-48c5-4e5e-9e3d-aa6d089ee164 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:53.767398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9fc4449-48c5-4e5e-9e3d-aa6d089ee164 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:53.768333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9fc4449-48c5-4e5e-9e3d-aa6d089ee164 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:53.768921 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9fc4449-48c5-4e5e-9e3d-aa6d089ee164 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:53.769535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9fc4449-48c5-4e5e-9e3d-aa6d089ee164 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:53.808623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f9fc4449-48c5-4e5e-9e3d-aa6d089ee164 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.062653 Aug 28 01:01:53.809417 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 389/777] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:53 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:53.815160 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-805cb0da-a6ff-4c6c-8396-8b8d04bdffd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:53.817837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-805cb0da-a6ff-4c6c-8396-8b8d04bdffd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:53.825295 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-805cb0da-a6ff-4c6c-8396-8b8d04bdffd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:53.825935 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-805cb0da-a6ff-4c6c-8396-8b8d04bdffd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:53.826475 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-805cb0da-a6ff-4c6c-8396-8b8d04bdffd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:53.827469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-805cb0da-a6ff-4c6c-8396-8b8d04bdffd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:53.828022 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-805cb0da-a6ff-4c6c-8396-8b8d04bdffd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:53.828514 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-805cb0da-a6ff-4c6c-8396-8b8d04bdffd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:53.865431 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-805cb0da-a6ff-4c6c-8396-8b8d04bdffd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054658 Aug 28 01:01:53.866136 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 390/778] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:53 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:53.895353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-24ab0399-23dd-40c8-8798-df1ff02a8a8a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:53.905976 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-24ab0399-23dd-40c8-8798-df1ff02a8a8a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.506972 Aug 28 01:01:53.919883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 389/779] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:53 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 510 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:54.829983 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-826ad021-ea50-4225-8649-326aa78329ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:54.834013 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-826ad021-ea50-4225-8649-326aa78329ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:54.846523 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-826ad021-ea50-4225-8649-326aa78329ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:54.847367 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-826ad021-ea50-4225-8649-326aa78329ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:54.848636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-826ad021-ea50-4225-8649-326aa78329ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:54.849927 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-826ad021-ea50-4225-8649-326aa78329ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:54.850745 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-826ad021-ea50-4225-8649-326aa78329ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:54.851477 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-826ad021-ea50-4225-8649-326aa78329ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:54.888719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c709f3d8-b524-4405-bd68-f925cc7d307f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:54.892987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c709f3d8-b524-4405-bd68-f925cc7d307f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:54.905339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c709f3d8-b524-4405-bd68-f925cc7d307f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:54.906126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c709f3d8-b524-4405-bd68-f925cc7d307f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:54.906821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c709f3d8-b524-4405-bd68-f925cc7d307f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:54.908011 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c709f3d8-b524-4405-bd68-f925cc7d307f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:54.908759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c709f3d8-b524-4405-bd68-f925cc7d307f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:54.909640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c709f3d8-b524-4405-bd68-f925cc7d307f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:54.912440 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-826ad021-ea50-4225-8649-326aa78329ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.089912 Aug 28 01:01:54.913764 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 391/780] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:54 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:54.934748 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d4bd2d99-61f0-40ed-bd01-0df3dc735ec6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:54.935829 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d4bd2d99-61f0-40ed-bd01-0df3dc735ec6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:54.947497 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d4bd2d99-61f0-40ed-bd01-0df3dc735ec6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:54.948354 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d4bd2d99-61f0-40ed-bd01-0df3dc735ec6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:54.949169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d4bd2d99-61f0-40ed-bd01-0df3dc735ec6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:54.950408 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d4bd2d99-61f0-40ed-bd01-0df3dc735ec6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:54.951373 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d4bd2d99-61f0-40ed-bd01-0df3dc735ec6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:54.952265 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d4bd2d99-61f0-40ed-bd01-0df3dc735ec6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:54.970732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c709f3d8-b524-4405-bd68-f925cc7d307f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.091322 Aug 28 01:01:54.971713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 390/781] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:54 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:55.053587 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d4bd2d99-61f0-40ed-bd01-0df3dc735ec6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:55.069084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d4bd2d99-61f0-40ed-bd01-0df3dc735ec6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.141132 Aug 28 01:01:55.069959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 392/782] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:54 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:01:55.929805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e6829db7-95a4-4f8f-8ec9-52b5b67b9c92 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:55.933045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e6829db7-95a4-4f8f-8ec9-52b5b67b9c92 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:55.940799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6829db7-95a4-4f8f-8ec9-52b5b67b9c92 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:55.941656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6829db7-95a4-4f8f-8ec9-52b5b67b9c92 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:55.942206 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6829db7-95a4-4f8f-8ec9-52b5b67b9c92 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:55.943057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6829db7-95a4-4f8f-8ec9-52b5b67b9c92 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:55.944006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6829db7-95a4-4f8f-8ec9-52b5b67b9c92 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:55.944527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6829db7-95a4-4f8f-8ec9-52b5b67b9c92 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:55.988523 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-da5ef5c8-3672-4a30-93ac-2f7e6267a43e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:55.992202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-da5ef5c8-3672-4a30-93ac-2f7e6267a43e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:55.996251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e6829db7-95a4-4f8f-8ec9-52b5b67b9c92 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.070825 Aug 28 01:01:55.997366 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 391/783] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:55 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:56.001502 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da5ef5c8-3672-4a30-93ac-2f7e6267a43e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:56.002049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da5ef5c8-3672-4a30-93ac-2f7e6267a43e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:56.002527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da5ef5c8-3672-4a30-93ac-2f7e6267a43e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:56.003383 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da5ef5c8-3672-4a30-93ac-2f7e6267a43e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:56.004006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da5ef5c8-3672-4a30-93ac-2f7e6267a43e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:56.004561 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da5ef5c8-3672-4a30-93ac-2f7e6267a43e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:56.059543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-da5ef5c8-3672-4a30-93ac-2f7e6267a43e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.076816 Aug 28 01:01:56.060341 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 393/784] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:55 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:01:56.106086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d2910a28-42dd-4157-b12c-e25330e42e5b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:56.107153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d2910a28-42dd-4157-b12c-e25330e42e5b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:56.114508 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d2910a28-42dd-4157-b12c-e25330e42e5b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:56.115166 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d2910a28-42dd-4157-b12c-e25330e42e5b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:56.115880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d2910a28-42dd-4157-b12c-e25330e42e5b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:56.116902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d2910a28-42dd-4157-b12c-e25330e42e5b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:56.117639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d2910a28-42dd-4157-b12c-e25330e42e5b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:56.118242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d2910a28-42dd-4157-b12c-e25330e42e5b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:56.204057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d2910a28-42dd-4157-b12c-e25330e42e5b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:56.213432 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d2910a28-42dd-4157-b12c-e25330e42e5b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.111577 Aug 28 01:01:56.214058 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 392/785] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:56 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:57.011759 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c4bb0eb7-0694-4e57-b563-c2c0fc6d8385 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:57.017887 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c4bb0eb7-0694-4e57-b563-c2c0fc6d8385 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:57.030303 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c4bb0eb7-0694-4e57-b563-c2c0fc6d8385 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:57.031458 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c4bb0eb7-0694-4e57-b563-c2c0fc6d8385 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:57.032447 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c4bb0eb7-0694-4e57-b563-c2c0fc6d8385 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:57.034109 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c4bb0eb7-0694-4e57-b563-c2c0fc6d8385 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:57.035198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c4bb0eb7-0694-4e57-b563-c2c0fc6d8385 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:57.036145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c4bb0eb7-0694-4e57-b563-c2c0fc6d8385 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:57.077062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0578c9ae-ee35-4fad-8d52-3f612b90af51 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:57.080487 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0578c9ae-ee35-4fad-8d52-3f612b90af51 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:57.092437 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0578c9ae-ee35-4fad-8d52-3f612b90af51 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:57.092922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0578c9ae-ee35-4fad-8d52-3f612b90af51 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:57.093393 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0578c9ae-ee35-4fad-8d52-3f612b90af51 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:57.093980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c4bb0eb7-0694-4e57-b563-c2c0fc6d8385 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.088529 Aug 28 01:01:57.094187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0578c9ae-ee35-4fad-8d52-3f612b90af51 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:57.094563 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0578c9ae-ee35-4fad-8d52-3f612b90af51 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:57.095175 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 394/786] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:57 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:57.095373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0578c9ae-ee35-4fad-8d52-3f612b90af51 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:57.134763 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0578c9ae-ee35-4fad-8d52-3f612b90af51 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064466 Aug 28 01:01:57.135572 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 393/787] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:57 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:01:57.243188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-96499daa-dc4f-4e83-bf3b-830078fa86cb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:57.243935 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-96499daa-dc4f-4e83-bf3b-830078fa86cb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:57.254277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96499daa-dc4f-4e83-bf3b-830078fa86cb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:57.255071 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96499daa-dc4f-4e83-bf3b-830078fa86cb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:57.255802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96499daa-dc4f-4e83-bf3b-830078fa86cb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:57.256939 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96499daa-dc4f-4e83-bf3b-830078fa86cb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:57.257753 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96499daa-dc4f-4e83-bf3b-830078fa86cb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:57.258367 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96499daa-dc4f-4e83-bf3b-830078fa86cb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:57.345285 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-96499daa-dc4f-4e83-bf3b-830078fa86cb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:57.360345 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-96499daa-dc4f-4e83-bf3b-830078fa86cb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.123663 Aug 28 01:01:57.361115 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 395/788] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:57 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:01:58.111411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cb3e6fbe-6c51-4704-b28d-ca6b1bbde1ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:58.115618 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cb3e6fbe-6c51-4704-b28d-ca6b1bbde1ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:58.123650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cb3e6fbe-6c51-4704-b28d-ca6b1bbde1ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:58.124203 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cb3e6fbe-6c51-4704-b28d-ca6b1bbde1ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:58.125422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cb3e6fbe-6c51-4704-b28d-ca6b1bbde1ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:58.126182 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cb3e6fbe-6c51-4704-b28d-ca6b1bbde1ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:58.127038 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cb3e6fbe-6c51-4704-b28d-ca6b1bbde1ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:58.127677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cb3e6fbe-6c51-4704-b28d-ca6b1bbde1ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:58.150408 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7a46c75f-4271-4ee0-8df8-283a9827e1ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:58.153840 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7a46c75f-4271-4ee0-8df8-283a9827e1ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:58.166069 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7a46c75f-4271-4ee0-8df8-283a9827e1ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:58.167251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7a46c75f-4271-4ee0-8df8-283a9827e1ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:58.168206 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7a46c75f-4271-4ee0-8df8-283a9827e1ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:58.169563 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7a46c75f-4271-4ee0-8df8-283a9827e1ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:58.170519 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7a46c75f-4271-4ee0-8df8-283a9827e1ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:58.171191 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7a46c75f-4271-4ee0-8df8-283a9827e1ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:58.176861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cb3e6fbe-6c51-4704-b28d-ca6b1bbde1ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.072284 Aug 28 01:01:58.178490 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 394/789] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:58 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:58.227815 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7a46c75f-4271-4ee0-8df8-283a9827e1ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.081634 Aug 28 01:01:58.228866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 396/790] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:58 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:01:58.393294 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a9ff4230-ba39-4a7d-ac07-4042ece46f4e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:58.394119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a9ff4230-ba39-4a7d-ac07-4042ece46f4e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:58.401644 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9ff4230-ba39-4a7d-ac07-4042ece46f4e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:58.402162 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9ff4230-ba39-4a7d-ac07-4042ece46f4e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:58.402627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9ff4230-ba39-4a7d-ac07-4042ece46f4e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:58.403330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9ff4230-ba39-4a7d-ac07-4042ece46f4e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:58.403818 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9ff4230-ba39-4a7d-ac07-4042ece46f4e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:58.404214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9ff4230-ba39-4a7d-ac07-4042ece46f4e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:58.478174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a9ff4230-ba39-4a7d-ac07-4042ece46f4e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:58.491306 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a9ff4230-ba39-4a7d-ac07-4042ece46f4e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.101811 Aug 28 01:01:58.492154 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 395/791] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:58 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:01:59.198173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-efc0a620-48bc-4411-a22a-a156bd9d360c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:59.201075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-efc0a620-48bc-4411-a22a-a156bd9d360c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:59.208837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-efc0a620-48bc-4411-a22a-a156bd9d360c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:59.209423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-efc0a620-48bc-4411-a22a-a156bd9d360c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:59.213877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-efc0a620-48bc-4411-a22a-a156bd9d360c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:59.213877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-efc0a620-48bc-4411-a22a-a156bd9d360c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:59.213877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-efc0a620-48bc-4411-a22a-a156bd9d360c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:59.213877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-efc0a620-48bc-4411-a22a-a156bd9d360c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:59.243292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-70968460-487d-4ad3-be21-77f06fde11d1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:59.245610 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-70968460-487d-4ad3-be21-77f06fde11d1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:59.252160 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-efc0a620-48bc-4411-a22a-a156bd9d360c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.064238 Aug 28 01:01:59.252518 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70968460-487d-4ad3-be21-77f06fde11d1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:59.253007 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70968460-487d-4ad3-be21-77f06fde11d1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:59.257800 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 397/792] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:01:59 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:01:59.258072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70968460-487d-4ad3-be21-77f06fde11d1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:59.258072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70968460-487d-4ad3-be21-77f06fde11d1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:59.258072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70968460-487d-4ad3-be21-77f06fde11d1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:59.258072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70968460-487d-4ad3-be21-77f06fde11d1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:59.297961 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-70968460-487d-4ad3-be21-77f06fde11d1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059359 Aug 28 01:01:59.298665 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 396/793] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:01:59 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:01:59.524059 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-54ddce27-66ed-4e19-a33a-e73a189282bd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:01:59.524649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-54ddce27-66ed-4e19-a33a-e73a189282bd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:01:59.531570 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54ddce27-66ed-4e19-a33a-e73a189282bd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:59.532113 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54ddce27-66ed-4e19-a33a-e73a189282bd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:59.532593 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54ddce27-66ed-4e19-a33a-e73a189282bd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:59.533427 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54ddce27-66ed-4e19-a33a-e73a189282bd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:01:59.533925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54ddce27-66ed-4e19-a33a-e73a189282bd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:01:59.534480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54ddce27-66ed-4e19-a33a-e73a189282bd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:01:59.618514 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-54ddce27-66ed-4e19-a33a-e73a189282bd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:01:59.630961 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:01:59.636951 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-54ddce27-66ed-4e19-a33a-e73a189282bd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.116931 Aug 28 01:01:59.637969 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 398/794] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:01:59 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:02:00.266211 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5b433c34-1159-49dd-bb6d-1578eebb27e2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:00.268421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5b433c34-1159-49dd-bb6d-1578eebb27e2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:00.275962 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b433c34-1159-49dd-bb6d-1578eebb27e2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:00.276373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b433c34-1159-49dd-bb6d-1578eebb27e2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:00.276814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b433c34-1159-49dd-bb6d-1578eebb27e2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:00.277585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b433c34-1159-49dd-bb6d-1578eebb27e2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:00.278039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b433c34-1159-49dd-bb6d-1578eebb27e2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:00.278464 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b433c34-1159-49dd-bb6d-1578eebb27e2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:00.314441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5b433c34-1159-49dd-bb6d-1578eebb27e2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052329 Aug 28 01:02:00.315140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 397/795] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:00 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:00.316489 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2bf8dbe5-67b7-4e99-8839-3a13d45e4fb9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:00.320576 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2bf8dbe5-67b7-4e99-8839-3a13d45e4fb9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:00.332621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2bf8dbe5-67b7-4e99-8839-3a13d45e4fb9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:00.333542 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2bf8dbe5-67b7-4e99-8839-3a13d45e4fb9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:00.334315 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2bf8dbe5-67b7-4e99-8839-3a13d45e4fb9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:00.335589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2bf8dbe5-67b7-4e99-8839-3a13d45e4fb9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:00.336429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2bf8dbe5-67b7-4e99-8839-3a13d45e4fb9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:00.337229 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2bf8dbe5-67b7-4e99-8839-3a13d45e4fb9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:00.396909 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2bf8dbe5-67b7-4e99-8839-3a13d45e4fb9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.087689 Aug 28 01:02:00.398040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 399/796] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:00 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:02:00.667344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f16413a6-9018-4f15-872c-775d2c595166 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:00.667825 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f16413a6-9018-4f15-872c-775d2c595166 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:00.674595 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f16413a6-9018-4f15-872c-775d2c595166 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:00.675083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f16413a6-9018-4f15-872c-775d2c595166 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:00.675746 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f16413a6-9018-4f15-872c-775d2c595166 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:00.676851 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f16413a6-9018-4f15-872c-775d2c595166 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:00.677343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f16413a6-9018-4f15-872c-775d2c595166 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:00.677768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f16413a6-9018-4f15-872c-775d2c595166 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:00.756285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f16413a6-9018-4f15-872c-775d2c595166 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:00.764928 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f16413a6-9018-4f15-872c-775d2c595166 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.102026 Aug 28 01:02:00.765540 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 398/797] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:00 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:01.329318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-42b81aaa-55d4-4f12-bd9d-ea9214aed9ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:01.331601 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-42b81aaa-55d4-4f12-bd9d-ea9214aed9ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:01.338969 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42b81aaa-55d4-4f12-bd9d-ea9214aed9ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:01.339451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42b81aaa-55d4-4f12-bd9d-ea9214aed9ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:01.339885 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42b81aaa-55d4-4f12-bd9d-ea9214aed9ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:01.340590 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42b81aaa-55d4-4f12-bd9d-ea9214aed9ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:01.341073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42b81aaa-55d4-4f12-bd9d-ea9214aed9ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:01.341525 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42b81aaa-55d4-4f12-bd9d-ea9214aed9ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:01.377655 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-42b81aaa-55d4-4f12-bd9d-ea9214aed9ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053160 Aug 28 01:02:01.378255 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 400/798] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:01 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 28 01:02:01.412870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5471d52d-5d8b-4ed1-8da5-76819672166f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:01.415312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5471d52d-5d8b-4ed1-8da5-76819672166f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:01.423716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5471d52d-5d8b-4ed1-8da5-76819672166f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:01.424335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5471d52d-5d8b-4ed1-8da5-76819672166f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:01.424832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5471d52d-5d8b-4ed1-8da5-76819672166f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:01.425832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5471d52d-5d8b-4ed1-8da5-76819672166f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:01.426352 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5471d52d-5d8b-4ed1-8da5-76819672166f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:01.426880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5471d52d-5d8b-4ed1-8da5-76819672166f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:01.463870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5471d52d-5d8b-4ed1-8da5-76819672166f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055258 Aug 28 01:02:01.464523 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 399/799] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:01 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:02:01.794697 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-23a3c217-8515-4bed-939f-bfea1f8989c1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:01.795369 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-23a3c217-8515-4bed-939f-bfea1f8989c1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:01.802204 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23a3c217-8515-4bed-939f-bfea1f8989c1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:01.802698 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23a3c217-8515-4bed-939f-bfea1f8989c1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:01.803134 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23a3c217-8515-4bed-939f-bfea1f8989c1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:01.804025 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23a3c217-8515-4bed-939f-bfea1f8989c1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:01.804644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23a3c217-8515-4bed-939f-bfea1f8989c1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:01.805102 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23a3c217-8515-4bed-939f-bfea1f8989c1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:01.880735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-23a3c217-8515-4bed-939f-bfea1f8989c1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:01.889121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-23a3c217-8515-4bed-939f-bfea1f8989c1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.098387 Aug 28 01:02:01.889719 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 401/800] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:01 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:02.392649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ccd4bf8a-1a20-428c-926d-7c53985e1abf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:02.395510 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ccd4bf8a-1a20-428c-926d-7c53985e1abf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:02.404812 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ccd4bf8a-1a20-428c-926d-7c53985e1abf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:02.405717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ccd4bf8a-1a20-428c-926d-7c53985e1abf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:02.406489 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ccd4bf8a-1a20-428c-926d-7c53985e1abf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:02.407771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ccd4bf8a-1a20-428c-926d-7c53985e1abf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:02.408562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ccd4bf8a-1a20-428c-926d-7c53985e1abf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:02.409347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ccd4bf8a-1a20-428c-926d-7c53985e1abf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:02.459872 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:02:02.470204 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ccd4bf8a-1a20-428c-926d-7c53985e1abf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.082426 Aug 28 01:02:02.471259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 400/801] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:02 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:02.479627 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-050c6555-5ee9-4c84-97a3-8c2a192f8d99 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:02.482169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-050c6555-5ee9-4c84-97a3-8c2a192f8d99 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:02.490177 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-050c6555-5ee9-4c84-97a3-8c2a192f8d99 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:02.490729 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-050c6555-5ee9-4c84-97a3-8c2a192f8d99 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:02.491205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-050c6555-5ee9-4c84-97a3-8c2a192f8d99 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:02.491968 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-050c6555-5ee9-4c84-97a3-8c2a192f8d99 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:02.492531 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-050c6555-5ee9-4c84-97a3-8c2a192f8d99 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:02.493052 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-050c6555-5ee9-4c84-97a3-8c2a192f8d99 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:02.533679 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-050c6555-5ee9-4c84-97a3-8c2a192f8d99 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060886 Aug 28 01:02:02.534506 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 402/802] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:02 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:02:02.910511 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7c74e6d0-377a-4941-a865-5498487b74a8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:02.911086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7c74e6d0-377a-4941-a865-5498487b74a8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:02.918630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c74e6d0-377a-4941-a865-5498487b74a8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:02.919166 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c74e6d0-377a-4941-a865-5498487b74a8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:02.919640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c74e6d0-377a-4941-a865-5498487b74a8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:02.920414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c74e6d0-377a-4941-a865-5498487b74a8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:02.921406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c74e6d0-377a-4941-a865-5498487b74a8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:02.921406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c74e6d0-377a-4941-a865-5498487b74a8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:03.008339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7c74e6d0-377a-4941-a865-5498487b74a8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:03.017417 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7c74e6d0-377a-4941-a865-5498487b74a8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.111402 Aug 28 01:02:03.018171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 401/803] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:02 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:03.486782 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-754c9ed5-69e2-410e-9136-187c1f96e935 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:03.489421 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-754c9ed5-69e2-410e-9136-187c1f96e935 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:03.496904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-754c9ed5-69e2-410e-9136-187c1f96e935 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:03.497530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-754c9ed5-69e2-410e-9136-187c1f96e935 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:03.498008 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-754c9ed5-69e2-410e-9136-187c1f96e935 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:03.498787 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-754c9ed5-69e2-410e-9136-187c1f96e935 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:03.499390 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-754c9ed5-69e2-410e-9136-187c1f96e935 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:03.499860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-754c9ed5-69e2-410e-9136-187c1f96e935 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:03.535735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-754c9ed5-69e2-410e-9136-187c1f96e935 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053115 Aug 28 01:02:03.536303 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 403/804] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:03 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:03.548015 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-caaafc7a-cd4c-4f76-a7d9-cc4d16db68ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:03.550336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-caaafc7a-cd4c-4f76-a7d9-cc4d16db68ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:03.557435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-caaafc7a-cd4c-4f76-a7d9-cc4d16db68ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:03.557934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-caaafc7a-cd4c-4f76-a7d9-cc4d16db68ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:03.558377 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-caaafc7a-cd4c-4f76-a7d9-cc4d16db68ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:03.559106 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-caaafc7a-cd4c-4f76-a7d9-cc4d16db68ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:03.559583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-caaafc7a-cd4c-4f76-a7d9-cc4d16db68ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:03.560009 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-caaafc7a-cd4c-4f76-a7d9-cc4d16db68ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:03.598521 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-caaafc7a-cd4c-4f76-a7d9-cc4d16db68ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054687 Aug 28 01:02:03.599115 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 402/805] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:03 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:04.044125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5175b533-d20c-41d9-8e41-1baa23c19c94 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:04.044885 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5175b533-d20c-41d9-8e41-1baa23c19c94 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:04.052722 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5175b533-d20c-41d9-8e41-1baa23c19c94 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:04.053404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5175b533-d20c-41d9-8e41-1baa23c19c94 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:04.053971 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5175b533-d20c-41d9-8e41-1baa23c19c94 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:04.054799 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5175b533-d20c-41d9-8e41-1baa23c19c94 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:04.055340 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5175b533-d20c-41d9-8e41-1baa23c19c94 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:04.055871 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5175b533-d20c-41d9-8e41-1baa23c19c94 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:04.141159 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5175b533-d20c-41d9-8e41-1baa23c19c94 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:04.150173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5175b533-d20c-41d9-8e41-1baa23c19c94 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.110853 Aug 28 01:02:04.150831 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 404/806] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:04 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:04.551234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-32991d41-7dbb-409f-83db-1643e93ee845 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:04.553916 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-32991d41-7dbb-409f-83db-1643e93ee845 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:04.556902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=80 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:02:04.564086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32991d41-7dbb-409f-83db-1643e93ee845 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:04.564677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32991d41-7dbb-409f-83db-1643e93ee845 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:04.565282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32991d41-7dbb-409f-83db-1643e93ee845 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:04.566245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32991d41-7dbb-409f-83db-1643e93ee845 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:04.566773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32991d41-7dbb-409f-83db-1643e93ee845 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:04.567383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32991d41-7dbb-409f-83db-1643e93ee845 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:04.608587 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-32991d41-7dbb-409f-83db-1643e93ee845 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.062295 Aug 28 01:02:04.609267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 403/807] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:04 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:04.615113 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0fdd2850-57ae-450b-9f2d-d4f2dadf190c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:04.618665 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0fdd2850-57ae-450b-9f2d-d4f2dadf190c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:04.624037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=84 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:02:04.630942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0fdd2850-57ae-450b-9f2d-d4f2dadf190c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:04.631757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0fdd2850-57ae-450b-9f2d-d4f2dadf190c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:04.632425 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0fdd2850-57ae-450b-9f2d-d4f2dadf190c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:04.633590 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0fdd2850-57ae-450b-9f2d-d4f2dadf190c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:04.634288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0fdd2850-57ae-450b-9f2d-d4f2dadf190c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:04.634882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0fdd2850-57ae-450b-9f2d-d4f2dadf190c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:04.682605 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0fdd2850-57ae-450b-9f2d-d4f2dadf190c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.073857 Aug 28 01:02:04.683595 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 405/808] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:04 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:02:05.172442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4a91c8f5-31ae-4245-ae70-faf80010e61d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:05.173078 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4a91c8f5-31ae-4245-ae70-faf80010e61d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:05.179689 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a91c8f5-31ae-4245-ae70-faf80010e61d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:05.180184 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a91c8f5-31ae-4245-ae70-faf80010e61d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:05.180630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a91c8f5-31ae-4245-ae70-faf80010e61d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:05.181468 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a91c8f5-31ae-4245-ae70-faf80010e61d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:05.181859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a91c8f5-31ae-4245-ae70-faf80010e61d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:05.182279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a91c8f5-31ae-4245-ae70-faf80010e61d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:05.264013 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4a91c8f5-31ae-4245-ae70-faf80010e61d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:05.274102 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4a91c8f5-31ae-4245-ae70-faf80010e61d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.105560 Aug 28 01:02:05.274769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 404/809] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:05 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:02:05.622097 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-afdf2187-c0aa-40f9-a07a-3cc349d125d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:05.624494 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-afdf2187-c0aa-40f9-a07a-3cc349d125d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:05.634992 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-afdf2187-c0aa-40f9-a07a-3cc349d125d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:05.635823 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-afdf2187-c0aa-40f9-a07a-3cc349d125d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:05.636531 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-afdf2187-c0aa-40f9-a07a-3cc349d125d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:05.637849 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-afdf2187-c0aa-40f9-a07a-3cc349d125d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:05.638707 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-afdf2187-c0aa-40f9-a07a-3cc349d125d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:05.639622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-afdf2187-c0aa-40f9-a07a-3cc349d125d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:05.686771 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-afdf2187-c0aa-40f9-a07a-3cc349d125d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.068554 Aug 28 01:02:05.687256 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 406/810] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:05 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:05.701790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fcbe3951-2a9e-41df-99fb-2a601c0fe2e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:05.703996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fcbe3951-2a9e-41df-99fb-2a601c0fe2e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:05.715852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fcbe3951-2a9e-41df-99fb-2a601c0fe2e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:05.716655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fcbe3951-2a9e-41df-99fb-2a601c0fe2e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:05.717413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fcbe3951-2a9e-41df-99fb-2a601c0fe2e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:05.718449 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fcbe3951-2a9e-41df-99fb-2a601c0fe2e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:05.718911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fcbe3951-2a9e-41df-99fb-2a601c0fe2e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:05.719330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fcbe3951-2a9e-41df-99fb-2a601c0fe2e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:05.754934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fcbe3951-2a9e-41df-99fb-2a601c0fe2e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059172 Aug 28 01:02:05.755528 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 405/811] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:05 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:06.298424 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c9c046b4-1089-4a7a-b701-5675b3d26ab5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:06.299508 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c9c046b4-1089-4a7a-b701-5675b3d26ab5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:06.307657 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9c046b4-1089-4a7a-b701-5675b3d26ab5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:06.308134 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9c046b4-1089-4a7a-b701-5675b3d26ab5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:06.308573 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9c046b4-1089-4a7a-b701-5675b3d26ab5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:06.309318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9c046b4-1089-4a7a-b701-5675b3d26ab5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:06.309762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9c046b4-1089-4a7a-b701-5675b3d26ab5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:06.310165 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9c046b4-1089-4a7a-b701-5675b3d26ab5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:06.390129 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c9c046b4-1089-4a7a-b701-5675b3d26ab5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:06.401003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c9c046b4-1089-4a7a-b701-5675b3d26ab5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.108172 Aug 28 01:02:06.401600 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 407/812] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:06 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:06.700905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-15cc84fe-1535-4bda-9b4e-383b059734ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:06.703195 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-15cc84fe-1535-4bda-9b4e-383b059734ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:06.710738 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15cc84fe-1535-4bda-9b4e-383b059734ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:06.711239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15cc84fe-1535-4bda-9b4e-383b059734ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:06.711670 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15cc84fe-1535-4bda-9b4e-383b059734ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:06.712392 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15cc84fe-1535-4bda-9b4e-383b059734ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:06.712838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15cc84fe-1535-4bda-9b4e-383b059734ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:06.713293 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15cc84fe-1535-4bda-9b4e-383b059734ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:06.747064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-15cc84fe-1535-4bda-9b4e-383b059734ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050251 Aug 28 01:02:06.747652 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 406/813] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:06 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:06.768166 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8920a38b-e8f3-42e1-b94b-e5b27ea2ed47 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:06.770504 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8920a38b-e8f3-42e1-b94b-e5b27ea2ed47 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:06.777794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8920a38b-e8f3-42e1-b94b-e5b27ea2ed47 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:06.778282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8920a38b-e8f3-42e1-b94b-e5b27ea2ed47 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:06.778712 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8920a38b-e8f3-42e1-b94b-e5b27ea2ed47 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:06.779456 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8920a38b-e8f3-42e1-b94b-e5b27ea2ed47 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:06.779899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8920a38b-e8f3-42e1-b94b-e5b27ea2ed47 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:06.780422 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8920a38b-e8f3-42e1-b94b-e5b27ea2ed47 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:06.813969 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8920a38b-e8f3-42e1-b94b-e5b27ea2ed47 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049772 Aug 28 01:02:06.814589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 408/814] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:06 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:02:07.424677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ab2891d3-01d0-4cb9-9053-1f6b906061a7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:07.425371 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ab2891d3-01d0-4cb9-9053-1f6b906061a7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:07.433314 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab2891d3-01d0-4cb9-9053-1f6b906061a7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:07.433831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab2891d3-01d0-4cb9-9053-1f6b906061a7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:07.434261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab2891d3-01d0-4cb9-9053-1f6b906061a7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:07.434977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab2891d3-01d0-4cb9-9053-1f6b906061a7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:07.435422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab2891d3-01d0-4cb9-9053-1f6b906061a7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:07.435829 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab2891d3-01d0-4cb9-9053-1f6b906061a7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:07.504783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ab2891d3-01d0-4cb9-9053-1f6b906061a7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:07.518340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ab2891d3-01d0-4cb9-9053-1f6b906061a7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.097305 Aug 28 01:02:07.519479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 407/815] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:07 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:07.760877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4e3213a7-9420-41bc-a7ef-03c4d7d238d1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:07.763170 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4e3213a7-9420-41bc-a7ef-03c4d7d238d1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:07.770579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e3213a7-9420-41bc-a7ef-03c4d7d238d1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:07.771296 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e3213a7-9420-41bc-a7ef-03c4d7d238d1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:07.771697 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e3213a7-9420-41bc-a7ef-03c4d7d238d1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:07.772421 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e3213a7-9420-41bc-a7ef-03c4d7d238d1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:07.772928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e3213a7-9420-41bc-a7ef-03c4d7d238d1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:07.773465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e3213a7-9420-41bc-a7ef-03c4d7d238d1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:07.810785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4e3213a7-9420-41bc-a7ef-03c4d7d238d1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054476 Aug 28 01:02:07.811418 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 409/816] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:07 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:07.828132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6d16d579-40da-4a77-8783-1e3d2ab75bfb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:07.830507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6d16d579-40da-4a77-8783-1e3d2ab75bfb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:07.838142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d16d579-40da-4a77-8783-1e3d2ab75bfb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:07.838726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d16d579-40da-4a77-8783-1e3d2ab75bfb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:07.839085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d16d579-40da-4a77-8783-1e3d2ab75bfb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:07.839852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d16d579-40da-4a77-8783-1e3d2ab75bfb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:07.840330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d16d579-40da-4a77-8783-1e3d2ab75bfb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:07.840933 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d16d579-40da-4a77-8783-1e3d2ab75bfb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:07.875457 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6d16d579-40da-4a77-8783-1e3d2ab75bfb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051804 Aug 28 01:02:07.876064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 408/817] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:07 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:08.552639 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fc900670-ab10-4805-8f7a-6593a52a6bf2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:08.553501 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fc900670-ab10-4805-8f7a-6593a52a6bf2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:08.565595 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc900670-ab10-4805-8f7a-6593a52a6bf2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:08.566153 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc900670-ab10-4805-8f7a-6593a52a6bf2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:08.566750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc900670-ab10-4805-8f7a-6593a52a6bf2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:08.567480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc900670-ab10-4805-8f7a-6593a52a6bf2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:08.569016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc900670-ab10-4805-8f7a-6593a52a6bf2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:08.569473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc900670-ab10-4805-8f7a-6593a52a6bf2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:08.638739 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fc900670-ab10-4805-8f7a-6593a52a6bf2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:08.651877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fc900670-ab10-4805-8f7a-6593a52a6bf2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.103952 Aug 28 01:02:08.652540 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 410/818] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:08 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:08.825402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-60d1ccf1-89da-4d22-9487-6da3d54fe5d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:08.827541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-60d1ccf1-89da-4d22-9487-6da3d54fe5d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:08.834741 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60d1ccf1-89da-4d22-9487-6da3d54fe5d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:08.835668 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60d1ccf1-89da-4d22-9487-6da3d54fe5d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:08.835668 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60d1ccf1-89da-4d22-9487-6da3d54fe5d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:08.836360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60d1ccf1-89da-4d22-9487-6da3d54fe5d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:08.836813 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60d1ccf1-89da-4d22-9487-6da3d54fe5d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:08.837271 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60d1ccf1-89da-4d22-9487-6da3d54fe5d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:08.873248 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-60d1ccf1-89da-4d22-9487-6da3d54fe5d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052003 Aug 28 01:02:08.873890 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 409/819] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:08 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:08.888613 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9094d6ab-1cf4-4113-bf6e-4d654854fd89 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:08.891036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9094d6ab-1cf4-4113-bf6e-4d654854fd89 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:08.899985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9094d6ab-1cf4-4113-bf6e-4d654854fd89 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:08.900779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9094d6ab-1cf4-4113-bf6e-4d654854fd89 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:08.901567 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9094d6ab-1cf4-4113-bf6e-4d654854fd89 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:08.902804 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9094d6ab-1cf4-4113-bf6e-4d654854fd89 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:08.903613 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9094d6ab-1cf4-4113-bf6e-4d654854fd89 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:08.904352 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9094d6ab-1cf4-4113-bf6e-4d654854fd89 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:08.963477 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9094d6ab-1cf4-4113-bf6e-4d654854fd89 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.078840 Aug 28 01:02:08.964491 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 411/820] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:08 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:09.674852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9f16f23f-c6f5-40d0-980b-c01b89c5c2e1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:09.675618 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9f16f23f-c6f5-40d0-980b-c01b89c5c2e1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:09.682977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9f16f23f-c6f5-40d0-980b-c01b89c5c2e1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:09.683539 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9f16f23f-c6f5-40d0-980b-c01b89c5c2e1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:09.684005 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9f16f23f-c6f5-40d0-980b-c01b89c5c2e1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:09.684747 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9f16f23f-c6f5-40d0-980b-c01b89c5c2e1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:09.685349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9f16f23f-c6f5-40d0-980b-c01b89c5c2e1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:09.685825 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9f16f23f-c6f5-40d0-980b-c01b89c5c2e1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:09.757319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9f16f23f-c6f5-40d0-980b-c01b89c5c2e1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:09.766762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9f16f23f-c6f5-40d0-980b-c01b89c5c2e1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.096008 Aug 28 01:02:09.767349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 410/821] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:09 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:09.889003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-21c47c44-5b5e-4111-8201-02eb2ec6fd63 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:09.891608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-21c47c44-5b5e-4111-8201-02eb2ec6fd63 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:09.898682 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-21c47c44-5b5e-4111-8201-02eb2ec6fd63 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:09.899144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-21c47c44-5b5e-4111-8201-02eb2ec6fd63 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:09.899598 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-21c47c44-5b5e-4111-8201-02eb2ec6fd63 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:09.900310 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-21c47c44-5b5e-4111-8201-02eb2ec6fd63 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:09.900727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-21c47c44-5b5e-4111-8201-02eb2ec6fd63 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:09.901209 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-21c47c44-5b5e-4111-8201-02eb2ec6fd63 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:09.936231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-21c47c44-5b5e-4111-8201-02eb2ec6fd63 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051325 Aug 28 01:02:09.936715 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 412/822] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:09 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:09.979463 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f4f1a772-7786-433e-ad3d-81271b34eee0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:09.981829 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f4f1a772-7786-433e-ad3d-81271b34eee0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:09.989409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f4f1a772-7786-433e-ad3d-81271b34eee0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:09.989952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f4f1a772-7786-433e-ad3d-81271b34eee0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:09.990457 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f4f1a772-7786-433e-ad3d-81271b34eee0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:09.991322 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f4f1a772-7786-433e-ad3d-81271b34eee0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:09.991818 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f4f1a772-7786-433e-ad3d-81271b34eee0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:09.992271 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f4f1a772-7786-433e-ad3d-81271b34eee0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:10.027796 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f4f1a772-7786-433e-ad3d-81271b34eee0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052362 Aug 28 01:02:10.028403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 411/823] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:09 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:10.790453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a802e823-071a-4db7-bc19-bbad2ad91e44 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:10.791067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a802e823-071a-4db7-bc19-bbad2ad91e44 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:10.797764 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a802e823-071a-4db7-bc19-bbad2ad91e44 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:10.798258 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a802e823-071a-4db7-bc19-bbad2ad91e44 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:10.798684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a802e823-071a-4db7-bc19-bbad2ad91e44 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:10.799406 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a802e823-071a-4db7-bc19-bbad2ad91e44 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:10.799844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a802e823-071a-4db7-bc19-bbad2ad91e44 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:10.800264 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a802e823-071a-4db7-bc19-bbad2ad91e44 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:10.870556 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a802e823-071a-4db7-bc19-bbad2ad91e44 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:10.879768 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a802e823-071a-4db7-bc19-bbad2ad91e44 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.093217 Aug 28 01:02:10.880364 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 413/824] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:10 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:10.949450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b9ca4b7c-6c4b-4f7c-b33d-e18524d2c769 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:10.951620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b9ca4b7c-6c4b-4f7c-b33d-e18524d2c769 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:10.958978 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9ca4b7c-6c4b-4f7c-b33d-e18524d2c769 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:10.959562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9ca4b7c-6c4b-4f7c-b33d-e18524d2c769 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:10.960040 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9ca4b7c-6c4b-4f7c-b33d-e18524d2c769 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:10.960807 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9ca4b7c-6c4b-4f7c-b33d-e18524d2c769 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:10.961369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9ca4b7c-6c4b-4f7c-b33d-e18524d2c769 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:10.961843 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9ca4b7c-6c4b-4f7c-b33d-e18524d2c769 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:10.998374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b9ca4b7c-6c4b-4f7c-b33d-e18524d2c769 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053185 Aug 28 01:02:10.999153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 412/825] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:10 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:11.040557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9d3f2ced-9bc3-4364-98c3-e743025567c1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:11.042992 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9d3f2ced-9bc3-4364-98c3-e743025567c1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:11.050615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d3f2ced-9bc3-4364-98c3-e743025567c1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:11.051108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d3f2ced-9bc3-4364-98c3-e743025567c1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:11.051539 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d3f2ced-9bc3-4364-98c3-e743025567c1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:11.052261 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d3f2ced-9bc3-4364-98c3-e743025567c1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:11.052708 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d3f2ced-9bc3-4364-98c3-e743025567c1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:11.053168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d3f2ced-9bc3-4364-98c3-e743025567c1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:11.101948 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9d3f2ced-9bc3-4364-98c3-e743025567c1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064990 Aug 28 01:02:11.102984 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 414/826] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:11 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:11.902116 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-defaaf4c-0bdd-4d3e-8067-6eb75e98f401 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:11.902831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-defaaf4c-0bdd-4d3e-8067-6eb75e98f401 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:11.910474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-defaaf4c-0bdd-4d3e-8067-6eb75e98f401 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:11.911429 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-defaaf4c-0bdd-4d3e-8067-6eb75e98f401 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:11.912012 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-defaaf4c-0bdd-4d3e-8067-6eb75e98f401 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:11.913017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-defaaf4c-0bdd-4d3e-8067-6eb75e98f401 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:11.913600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-defaaf4c-0bdd-4d3e-8067-6eb75e98f401 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:11.914119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-defaaf4c-0bdd-4d3e-8067-6eb75e98f401 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:11.983632 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-defaaf4c-0bdd-4d3e-8067-6eb75e98f401 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:11.993065 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-defaaf4c-0bdd-4d3e-8067-6eb75e98f401 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.095320 Aug 28 01:02:11.993847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 413/827] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:11 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:12.012215 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c0bd86db-4c7a-4349-8822-e942b27bba01 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:12.014885 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c0bd86db-4c7a-4349-8822-e942b27bba01 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:12.022132 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0bd86db-4c7a-4349-8822-e942b27bba01 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:12.022621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0bd86db-4c7a-4349-8822-e942b27bba01 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:12.023051 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0bd86db-4c7a-4349-8822-e942b27bba01 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:12.023792 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0bd86db-4c7a-4349-8822-e942b27bba01 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:12.024237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0bd86db-4c7a-4349-8822-e942b27bba01 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:12.024646 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0bd86db-4c7a-4349-8822-e942b27bba01 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:12.043498 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:02:12.061132 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c0bd86db-4c7a-4349-8822-e942b27bba01 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053158 Aug 28 01:02:12.061710 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 415/828] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:12 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:12.117024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3ee7c3d8-ec87-4a4f-b4a1-e763c07e824c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:12.119034 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3ee7c3d8-ec87-4a4f-b4a1-e763c07e824c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:12.125873 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ee7c3d8-ec87-4a4f-b4a1-e763c07e824c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:12.126366 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ee7c3d8-ec87-4a4f-b4a1-e763c07e824c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:12.126814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ee7c3d8-ec87-4a4f-b4a1-e763c07e824c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:12.127521 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ee7c3d8-ec87-4a4f-b4a1-e763c07e824c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:12.127979 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ee7c3d8-ec87-4a4f-b4a1-e763c07e824c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:12.128402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ee7c3d8-ec87-4a4f-b4a1-e763c07e824c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:12.162675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3ee7c3d8-ec87-4a4f-b4a1-e763c07e824c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051482 Aug 28 01:02:12.163191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 414/829] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:12 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:13.016384 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6324888f-18a7-4993-99d3-a70aaf0e0910 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:13.017087 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6324888f-18a7-4993-99d3-a70aaf0e0910 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:13.024542 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6324888f-18a7-4993-99d3-a70aaf0e0910 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:13.025067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6324888f-18a7-4993-99d3-a70aaf0e0910 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:13.025503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6324888f-18a7-4993-99d3-a70aaf0e0910 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:13.026217 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6324888f-18a7-4993-99d3-a70aaf0e0910 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:13.026667 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6324888f-18a7-4993-99d3-a70aaf0e0910 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:13.027091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6324888f-18a7-4993-99d3-a70aaf0e0910 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:13.077106 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d2017ddf-6bf8-45d8-9f84-6134590aed44 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:13.079998 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d2017ddf-6bf8-45d8-9f84-6134590aed44 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:13.089365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d2017ddf-6bf8-45d8-9f84-6134590aed44 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:13.090134 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d2017ddf-6bf8-45d8-9f84-6134590aed44 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:13.096795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d2017ddf-6bf8-45d8-9f84-6134590aed44 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:13.096795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d2017ddf-6bf8-45d8-9f84-6134590aed44 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:13.096795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d2017ddf-6bf8-45d8-9f84-6134590aed44 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:13.096795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d2017ddf-6bf8-45d8-9f84-6134590aed44 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:13.100005 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6324888f-18a7-4993-99d3-a70aaf0e0910 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:13.111757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6324888f-18a7-4993-99d3-a70aaf0e0910 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.098773 Aug 28 01:02:13.112688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 416/830] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:13 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:13.144933 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d2017ddf-6bf8-45d8-9f84-6134590aed44 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.074114 Aug 28 01:02:13.146013 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 415/831] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:13 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:13.176344 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5d245d77-3c58-4c59-8afe-265163305583 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:13.178888 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5d245d77-3c58-4c59-8afe-265163305583 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:13.186726 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5d245d77-3c58-4c59-8afe-265163305583 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:13.187229 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5d245d77-3c58-4c59-8afe-265163305583 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:13.187659 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5d245d77-3c58-4c59-8afe-265163305583 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:13.188390 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5d245d77-3c58-4c59-8afe-265163305583 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:13.188843 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5d245d77-3c58-4c59-8afe-265163305583 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:13.189286 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5d245d77-3c58-4c59-8afe-265163305583 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:13.223426 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5d245d77-3c58-4c59-8afe-265163305583 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051653 Aug 28 01:02:13.224301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 417/832] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:13 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:02:14.132426 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-745bb43a-aa18-491a-830a-b2f4880e5bd0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:14.133191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-745bb43a-aa18-491a-830a-b2f4880e5bd0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:14.140571 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-745bb43a-aa18-491a-830a-b2f4880e5bd0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:14.141191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-745bb43a-aa18-491a-830a-b2f4880e5bd0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:14.141669 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-745bb43a-aa18-491a-830a-b2f4880e5bd0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:14.142562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-745bb43a-aa18-491a-830a-b2f4880e5bd0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:14.143146 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-745bb43a-aa18-491a-830a-b2f4880e5bd0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:14.143706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-745bb43a-aa18-491a-830a-b2f4880e5bd0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:14.161581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-eed552e8-5e8b-4462-8be7-89d646a39c9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:14.164654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-eed552e8-5e8b-4462-8be7-89d646a39c9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:14.173794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eed552e8-5e8b-4462-8be7-89d646a39c9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:14.174639 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eed552e8-5e8b-4462-8be7-89d646a39c9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:14.175423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eed552e8-5e8b-4462-8be7-89d646a39c9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:14.176284 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eed552e8-5e8b-4462-8be7-89d646a39c9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:14.176844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eed552e8-5e8b-4462-8be7-89d646a39c9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:14.177398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eed552e8-5e8b-4462-8be7-89d646a39c9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:14.233113 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-745bb43a-aa18-491a-830a-b2f4880e5bd0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:14.234278 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-eed552e8-5e8b-4462-8be7-89d646a39c9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.077333 Aug 28 01:02:14.235093 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 418/833] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:14 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 28 01:02:14.242849 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-745bb43a-aa18-491a-830a-b2f4880e5bd0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.114628 Aug 28 01:02:14.243339 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1b1ee185-eb45-4067-a6d7-22a09e9f63e7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:14.243773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 416/834] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:14 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:14.246018 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1b1ee185-eb45-4067-a6d7-22a09e9f63e7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:14.253277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b1ee185-eb45-4067-a6d7-22a09e9f63e7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:14.253900 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b1ee185-eb45-4067-a6d7-22a09e9f63e7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:14.254539 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b1ee185-eb45-4067-a6d7-22a09e9f63e7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:14.255500 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b1ee185-eb45-4067-a6d7-22a09e9f63e7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:14.255998 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b1ee185-eb45-4067-a6d7-22a09e9f63e7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:14.256460 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b1ee185-eb45-4067-a6d7-22a09e9f63e7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:14.290998 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1b1ee185-eb45-4067-a6d7-22a09e9f63e7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054361 Aug 28 01:02:14.291816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 419/835] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:14 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:15.250561 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8fb0653b-6e91-43b0-b00f-41f8bac38103 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:15.252796 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8fb0653b-6e91-43b0-b00f-41f8bac38103 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:15.265010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8fb0653b-6e91-43b0-b00f-41f8bac38103 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:15.265772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8fb0653b-6e91-43b0-b00f-41f8bac38103 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:15.266314 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8fb0653b-6e91-43b0-b00f-41f8bac38103 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:15.267108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-32cc2445-42dc-449f-857c-43eba6913af0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:15.267906 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-32cc2445-42dc-449f-857c-43eba6913af0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:15.268200 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8fb0653b-6e91-43b0-b00f-41f8bac38103 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:15.268764 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8fb0653b-6e91-43b0-b00f-41f8bac38103 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:15.269402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8fb0653b-6e91-43b0-b00f-41f8bac38103 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:15.275404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-32cc2445-42dc-449f-857c-43eba6913af0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:15.275905 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-32cc2445-42dc-449f-857c-43eba6913af0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:15.276334 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-32cc2445-42dc-449f-857c-43eba6913af0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:15.277200 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-32cc2445-42dc-449f-857c-43eba6913af0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:15.277560 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-32cc2445-42dc-449f-857c-43eba6913af0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:15.278019 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-32cc2445-42dc-449f-857c-43eba6913af0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:15.293026 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:02:15.309303 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8fb0653b-6e91-43b0-b00f-41f8bac38103 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.062711 Aug 28 01:02:15.309880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 417/836] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:15 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:15.315145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0f8239f5-0056-4210-ac7a-ac37c17d7dda tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:15.318266 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0f8239f5-0056-4210-ac7a-ac37c17d7dda tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:15.324783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f8239f5-0056-4210-ac7a-ac37c17d7dda tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:15.325317 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f8239f5-0056-4210-ac7a-ac37c17d7dda tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:15.325750 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f8239f5-0056-4210-ac7a-ac37c17d7dda tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:15.326472 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f8239f5-0056-4210-ac7a-ac37c17d7dda tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:15.326923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f8239f5-0056-4210-ac7a-ac37c17d7dda tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:15.327345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f8239f5-0056-4210-ac7a-ac37c17d7dda tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:15.362287 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0f8239f5-0056-4210-ac7a-ac37c17d7dda tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051262 Aug 28 01:02:15.362876 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 418/837] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:15 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:15.388078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-32cc2445-42dc-449f-857c-43eba6913af0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:15.396953 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-32cc2445-42dc-449f-857c-43eba6913af0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.134312 Aug 28 01:02:15.397587 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 420/838] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:15 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:16.326242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3a709c8b-ffd6-42c8-8289-f3ee08e4ae6b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:16.329704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3a709c8b-ffd6-42c8-8289-f3ee08e4ae6b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:16.340076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a709c8b-ffd6-42c8-8289-f3ee08e4ae6b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:16.340816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a709c8b-ffd6-42c8-8289-f3ee08e4ae6b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:16.341507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a709c8b-ffd6-42c8-8289-f3ee08e4ae6b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:16.342615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a709c8b-ffd6-42c8-8289-f3ee08e4ae6b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:16.343318 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a709c8b-ffd6-42c8-8289-f3ee08e4ae6b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:16.343966 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a709c8b-ffd6-42c8-8289-f3ee08e4ae6b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:16.379153 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f6c4a0b4-4aa2-4c70-85b2-d9b7fc36519d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:16.382725 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f6c4a0b4-4aa2-4c70-85b2-d9b7fc36519d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:16.393657 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f6c4a0b4-4aa2-4c70-85b2-d9b7fc36519d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:16.394072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3a709c8b-ffd6-42c8-8289-f3ee08e4ae6b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.073856 Aug 28 01:02:16.394454 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f6c4a0b4-4aa2-4c70-85b2-d9b7fc36519d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:16.395067 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 419/839] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:16 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:16.395245 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f6c4a0b4-4aa2-4c70-85b2-d9b7fc36519d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:16.395988 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f6c4a0b4-4aa2-4c70-85b2-d9b7fc36519d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:16.396514 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f6c4a0b4-4aa2-4c70-85b2-d9b7fc36519d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:16.397193 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f6c4a0b4-4aa2-4c70-85b2-d9b7fc36519d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:16.419428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d1245bb8-2402-4e5a-a909-0c3ff505d27a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:16.420125 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d1245bb8-2402-4e5a-a909-0c3ff505d27a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:16.427300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d1245bb8-2402-4e5a-a909-0c3ff505d27a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:16.428082 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d1245bb8-2402-4e5a-a909-0c3ff505d27a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:16.428728 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d1245bb8-2402-4e5a-a909-0c3ff505d27a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:16.429634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d1245bb8-2402-4e5a-a909-0c3ff505d27a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:16.430215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d1245bb8-2402-4e5a-a909-0c3ff505d27a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:16.430817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d1245bb8-2402-4e5a-a909-0c3ff505d27a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:16.433389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f6c4a0b4-4aa2-4c70-85b2-d9b7fc36519d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061269 Aug 28 01:02:16.433978 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 421/840] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:16 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:02:16.502248 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d1245bb8-2402-4e5a-a909-0c3ff505d27a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:16.516019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d1245bb8-2402-4e5a-a909-0c3ff505d27a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.100391 Aug 28 01:02:16.516881 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 420/841] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:16 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:02:17.411381 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c7755b43-ad74-49f4-a3b7-5e83a4464055 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:17.413788 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c7755b43-ad74-49f4-a3b7-5e83a4464055 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:17.421002 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c7755b43-ad74-49f4-a3b7-5e83a4464055 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:17.421508 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c7755b43-ad74-49f4-a3b7-5e83a4464055 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:17.421940 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c7755b43-ad74-49f4-a3b7-5e83a4464055 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:17.422695 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c7755b43-ad74-49f4-a3b7-5e83a4464055 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:17.423104 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c7755b43-ad74-49f4-a3b7-5e83a4464055 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:17.423528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c7755b43-ad74-49f4-a3b7-5e83a4464055 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:17.450889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0f7246e7-5406-4d35-b1f1-106cb6132f45 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:17.454555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0f7246e7-5406-4d35-b1f1-106cb6132f45 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:17.460302 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c7755b43-ad74-49f4-a3b7-5e83a4464055 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053110 Aug 28 01:02:17.461302 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 422/842] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:17 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 28 01:02:17.466179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f7246e7-5406-4d35-b1f1-106cb6132f45 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:17.466963 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f7246e7-5406-4d35-b1f1-106cb6132f45 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:17.467635 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f7246e7-5406-4d35-b1f1-106cb6132f45 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:17.468693 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f7246e7-5406-4d35-b1f1-106cb6132f45 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:17.469391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f7246e7-5406-4d35-b1f1-106cb6132f45 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:17.470019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f7246e7-5406-4d35-b1f1-106cb6132f45 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:17.513071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0f7246e7-5406-4d35-b1f1-106cb6132f45 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.069648 Aug 28 01:02:17.513667 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 421/843] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:17 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:17.547267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f127f48f-53ef-4bb0-866b-4e7b73c259e2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:17.547874 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f127f48f-53ef-4bb0-866b-4e7b73c259e2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:17.554761 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f127f48f-53ef-4bb0-866b-4e7b73c259e2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:17.555242 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f127f48f-53ef-4bb0-866b-4e7b73c259e2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:17.555679 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f127f48f-53ef-4bb0-866b-4e7b73c259e2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:17.556397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f127f48f-53ef-4bb0-866b-4e7b73c259e2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:17.556842 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f127f48f-53ef-4bb0-866b-4e7b73c259e2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:17.557286 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f127f48f-53ef-4bb0-866b-4e7b73c259e2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:17.626832 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f127f48f-53ef-4bb0-866b-4e7b73c259e2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:17.635778 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f127f48f-53ef-4bb0-866b-4e7b73c259e2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.092507 Aug 28 01:02:17.636423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 423/844] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:17 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:18.474187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0637c02b-f78e-4c90-b5a9-ca6b6a1efbe7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:18.476467 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0637c02b-f78e-4c90-b5a9-ca6b6a1efbe7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:18.483106 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0637c02b-f78e-4c90-b5a9-ca6b6a1efbe7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:18.483603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0637c02b-f78e-4c90-b5a9-ca6b6a1efbe7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:18.484037 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0637c02b-f78e-4c90-b5a9-ca6b6a1efbe7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:18.484767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0637c02b-f78e-4c90-b5a9-ca6b6a1efbe7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:18.485257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0637c02b-f78e-4c90-b5a9-ca6b6a1efbe7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:18.485683 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0637c02b-f78e-4c90-b5a9-ca6b6a1efbe7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:18.523375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0637c02b-f78e-4c90-b5a9-ca6b6a1efbe7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053069 Aug 28 01:02:18.524318 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 422/845] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:18 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:18.528142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9b391e0c-f902-4104-b2a4-23c14d7e844f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:18.531016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9b391e0c-f902-4104-b2a4-23c14d7e844f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:18.538787 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b391e0c-f902-4104-b2a4-23c14d7e844f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:18.539333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b391e0c-f902-4104-b2a4-23c14d7e844f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:18.539762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b391e0c-f902-4104-b2a4-23c14d7e844f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:18.540509 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b391e0c-f902-4104-b2a4-23c14d7e844f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:18.541034 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b391e0c-f902-4104-b2a4-23c14d7e844f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:18.541458 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b391e0c-f902-4104-b2a4-23c14d7e844f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:18.576448 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9b391e0c-f902-4104-b2a4-23c14d7e844f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053770 Aug 28 01:02:18.576952 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 424/846] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:18 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:18.658650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9c9f929b-3177-4024-a522-1d97d4911847 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:18.659274 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9c9f929b-3177-4024-a522-1d97d4911847 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:18.666100 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c9f929b-3177-4024-a522-1d97d4911847 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:18.666603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c9f929b-3177-4024-a522-1d97d4911847 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:18.667039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c9f929b-3177-4024-a522-1d97d4911847 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:18.667768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c9f929b-3177-4024-a522-1d97d4911847 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:18.668228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c9f929b-3177-4024-a522-1d97d4911847 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:18.668652 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c9f929b-3177-4024-a522-1d97d4911847 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:18.737022 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9c9f929b-3177-4024-a522-1d97d4911847 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:18.746460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9c9f929b-3177-4024-a522-1d97d4911847 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.092210 Aug 28 01:02:18.747065 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 423/847] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:18 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:19.536902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-47c964f8-97af-46b9-b3d8-171f73e01272 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:19.539180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-47c964f8-97af-46b9-b3d8-171f73e01272 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:19.546289 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-47c964f8-97af-46b9-b3d8-171f73e01272 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:19.546767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-47c964f8-97af-46b9-b3d8-171f73e01272 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:19.547206 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-47c964f8-97af-46b9-b3d8-171f73e01272 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:19.547910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-47c964f8-97af-46b9-b3d8-171f73e01272 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:19.548359 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-47c964f8-97af-46b9-b3d8-171f73e01272 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:19.548789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-47c964f8-97af-46b9-b3d8-171f73e01272 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:19.582780 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-47c964f8-97af-46b9-b3d8-171f73e01272 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049831 Aug 28 01:02:19.583360 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 425/848] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:19 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:19.589762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-60545c33-93bb-40c9-beea-da9fba4bbe93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:19.592139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-60545c33-93bb-40c9-beea-da9fba4bbe93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:19.599491 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60545c33-93bb-40c9-beea-da9fba4bbe93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:19.599970 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60545c33-93bb-40c9-beea-da9fba4bbe93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:19.600399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60545c33-93bb-40c9-beea-da9fba4bbe93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:19.601140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60545c33-93bb-40c9-beea-da9fba4bbe93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:19.601584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60545c33-93bb-40c9-beea-da9fba4bbe93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:19.602001 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60545c33-93bb-40c9-beea-da9fba4bbe93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:19.633676 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-60545c33-93bb-40c9-beea-da9fba4bbe93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048436 Aug 28 01:02:19.634253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 424/849] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:19 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:19.768850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-707a785d-1ba8-486e-bd1e-50f38b4207f5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:19.769521 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-707a785d-1ba8-486e-bd1e-50f38b4207f5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:19.776063 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-707a785d-1ba8-486e-bd1e-50f38b4207f5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:19.776461 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-707a785d-1ba8-486e-bd1e-50f38b4207f5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:19.776884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-707a785d-1ba8-486e-bd1e-50f38b4207f5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:19.777638 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-707a785d-1ba8-486e-bd1e-50f38b4207f5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:19.778089 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-707a785d-1ba8-486e-bd1e-50f38b4207f5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:19.778516 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-707a785d-1ba8-486e-bd1e-50f38b4207f5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:19.851005 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-707a785d-1ba8-486e-bd1e-50f38b4207f5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:19.859725 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-707a785d-1ba8-486e-bd1e-50f38b4207f5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.095465 Aug 28 01:02:19.860324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 426/850] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:19 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:20.596015 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1eaf9307-5f93-4052-a407-7d907a540bb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:20.598186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1eaf9307-5f93-4052-a407-7d907a540bb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:20.605103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1eaf9307-5f93-4052-a407-7d907a540bb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:20.605578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1eaf9307-5f93-4052-a407-7d907a540bb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:20.605990 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1eaf9307-5f93-4052-a407-7d907a540bb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:20.606681 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1eaf9307-5f93-4052-a407-7d907a540bb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:20.607122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1eaf9307-5f93-4052-a407-7d907a540bb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:20.607534 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1eaf9307-5f93-4052-a407-7d907a540bb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:20.639204 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1eaf9307-5f93-4052-a407-7d907a540bb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047260 Aug 28 01:02:20.639815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 425/851] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:20 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:20.646932 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-370a14e8-5477-4142-b9eb-f59f80a66fc6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:20.649146 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-370a14e8-5477-4142-b9eb-f59f80a66fc6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:20.655878 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-370a14e8-5477-4142-b9eb-f59f80a66fc6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:20.656368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-370a14e8-5477-4142-b9eb-f59f80a66fc6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:20.656798 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-370a14e8-5477-4142-b9eb-f59f80a66fc6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:20.657551 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-370a14e8-5477-4142-b9eb-f59f80a66fc6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:20.657997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-370a14e8-5477-4142-b9eb-f59f80a66fc6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:20.658408 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-370a14e8-5477-4142-b9eb-f59f80a66fc6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:20.690348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-370a14e8-5477-4142-b9eb-f59f80a66fc6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047652 Aug 28 01:02:20.690935 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 427/852] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:20 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:21.021311 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9570060d-83ef-4ad6-bfc6-d9fa0c840c4b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:21.021930 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9570060d-83ef-4ad6-bfc6-d9fa0c840c4b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:21.028675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9570060d-83ef-4ad6-bfc6-d9fa0c840c4b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:21.029208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9570060d-83ef-4ad6-bfc6-d9fa0c840c4b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:21.031319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9570060d-83ef-4ad6-bfc6-d9fa0c840c4b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:21.031319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9570060d-83ef-4ad6-bfc6-d9fa0c840c4b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:21.031319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9570060d-83ef-4ad6-bfc6-d9fa0c840c4b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:21.031319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9570060d-83ef-4ad6-bfc6-d9fa0c840c4b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:21.107630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9570060d-83ef-4ad6-bfc6-d9fa0c840c4b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:21.118392 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9570060d-83ef-4ad6-bfc6-d9fa0c840c4b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.241330 Aug 28 01:02:21.119860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 426/853] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:20 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:21.653179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7e047c5c-23ac-4dd0-acba-a431af7ad064 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:21.655488 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7e047c5c-23ac-4dd0-acba-a431af7ad064 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:21.662713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e047c5c-23ac-4dd0-acba-a431af7ad064 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:21.663219 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e047c5c-23ac-4dd0-acba-a431af7ad064 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:21.663636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e047c5c-23ac-4dd0-acba-a431af7ad064 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:21.664346 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e047c5c-23ac-4dd0-acba-a431af7ad064 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:21.664795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e047c5c-23ac-4dd0-acba-a431af7ad064 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:21.665257 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e047c5c-23ac-4dd0-acba-a431af7ad064 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:21.701439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7e047c5c-23ac-4dd0-acba-a431af7ad064 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052064 Aug 28 01:02:21.702367 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 428/854] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:21 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 28 01:02:21.705261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fb859fdf-0137-48d5-af4e-da0b048e1f1b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:21.707723 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fb859fdf-0137-48d5-af4e-da0b048e1f1b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:21.714856 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fb859fdf-0137-48d5-af4e-da0b048e1f1b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:21.715350 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fb859fdf-0137-48d5-af4e-da0b048e1f1b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:21.715783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fb859fdf-0137-48d5-af4e-da0b048e1f1b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:21.716502 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fb859fdf-0137-48d5-af4e-da0b048e1f1b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:21.716981 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fb859fdf-0137-48d5-af4e-da0b048e1f1b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:21.717412 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fb859fdf-0137-48d5-af4e-da0b048e1f1b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:21.754451 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fb859fdf-0137-48d5-af4e-da0b048e1f1b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054344 Aug 28 01:02:21.755143 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 427/855] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:21 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:22.139979 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1fb52338-bb42-4ef9-9b5b-3f945f5b0575 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:22.140615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1fb52338-bb42-4ef9-9b5b-3f945f5b0575 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:22.147591 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1fb52338-bb42-4ef9-9b5b-3f945f5b0575 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:22.148089 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1fb52338-bb42-4ef9-9b5b-3f945f5b0575 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:22.148530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1fb52338-bb42-4ef9-9b5b-3f945f5b0575 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:22.149288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1fb52338-bb42-4ef9-9b5b-3f945f5b0575 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:22.149742 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1fb52338-bb42-4ef9-9b5b-3f945f5b0575 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:22.150165 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1fb52338-bb42-4ef9-9b5b-3f945f5b0575 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:22.224382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1fb52338-bb42-4ef9-9b5b-3f945f5b0575 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:22.233179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1fb52338-bb42-4ef9-9b5b-3f945f5b0575 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.097040 Aug 28 01:02:22.233839 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 429/856] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:22 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:22.715715 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ab9ec5b8-0820-4cfa-a605-8e8dd70c8f45 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:22.717971 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ab9ec5b8-0820-4cfa-a605-8e8dd70c8f45 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:22.725334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab9ec5b8-0820-4cfa-a605-8e8dd70c8f45 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:22.725860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab9ec5b8-0820-4cfa-a605-8e8dd70c8f45 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:22.726270 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab9ec5b8-0820-4cfa-a605-8e8dd70c8f45 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:22.727017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab9ec5b8-0820-4cfa-a605-8e8dd70c8f45 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:22.727470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab9ec5b8-0820-4cfa-a605-8e8dd70c8f45 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:22.727875 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab9ec5b8-0820-4cfa-a605-8e8dd70c8f45 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:22.761955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ab9ec5b8-0820-4cfa-a605-8e8dd70c8f45 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050242 Aug 28 01:02:22.762623 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 428/857] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:22 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:22.771300 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-24d84a00-6cce-4958-9d5d-226fe5e6ea54 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:22.773758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-24d84a00-6cce-4958-9d5d-226fe5e6ea54 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:22.780239 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24d84a00-6cce-4958-9d5d-226fe5e6ea54 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:22.780758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24d84a00-6cce-4958-9d5d-226fe5e6ea54 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:22.781220 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24d84a00-6cce-4958-9d5d-226fe5e6ea54 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:22.781971 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24d84a00-6cce-4958-9d5d-226fe5e6ea54 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:22.782673 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24d84a00-6cce-4958-9d5d-226fe5e6ea54 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:22.783099 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24d84a00-6cce-4958-9d5d-226fe5e6ea54 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:22.815917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-24d84a00-6cce-4958-9d5d-226fe5e6ea54 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051488 Aug 28 01:02:22.816470 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 430/858] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:22 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:02:23.254381 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a5f1bf06-25e2-4a51-9d7b-dbc843f5e752 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:23.254999 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a5f1bf06-25e2-4a51-9d7b-dbc843f5e752 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:23.261978 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5f1bf06-25e2-4a51-9d7b-dbc843f5e752 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:23.262462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5f1bf06-25e2-4a51-9d7b-dbc843f5e752 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:23.262892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5f1bf06-25e2-4a51-9d7b-dbc843f5e752 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:23.263626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5f1bf06-25e2-4a51-9d7b-dbc843f5e752 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:23.264074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5f1bf06-25e2-4a51-9d7b-dbc843f5e752 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:23.264488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5f1bf06-25e2-4a51-9d7b-dbc843f5e752 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:23.335512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a5f1bf06-25e2-4a51-9d7b-dbc843f5e752 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:23.344714 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a5f1bf06-25e2-4a51-9d7b-dbc843f5e752 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.094246 Aug 28 01:02:23.345381 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 429/859] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:23 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:23.778141 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-707e1cc0-839c-40e8-bf23-9f156b9d54ed tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:23.780804 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-707e1cc0-839c-40e8-bf23-9f156b9d54ed tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:23.788453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-707e1cc0-839c-40e8-bf23-9f156b9d54ed tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:23.789192 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-707e1cc0-839c-40e8-bf23-9f156b9d54ed tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:23.789679 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-707e1cc0-839c-40e8-bf23-9f156b9d54ed tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:23.790483 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-707e1cc0-839c-40e8-bf23-9f156b9d54ed tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:23.791076 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-707e1cc0-839c-40e8-bf23-9f156b9d54ed tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:23.791557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-707e1cc0-839c-40e8-bf23-9f156b9d54ed tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:23.827345 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-707e1cc0-839c-40e8-bf23-9f156b9d54ed tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053309 Aug 28 01:02:23.827945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 431/860] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:23 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:23.830501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1c27fef5-3666-4a5d-a804-1be41fc01700 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:23.833445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1c27fef5-3666-4a5d-a804-1be41fc01700 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:23.840698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1c27fef5-3666-4a5d-a804-1be41fc01700 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:23.841248 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1c27fef5-3666-4a5d-a804-1be41fc01700 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:23.841687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1c27fef5-3666-4a5d-a804-1be41fc01700 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:23.842378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1c27fef5-3666-4a5d-a804-1be41fc01700 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:23.842850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1c27fef5-3666-4a5d-a804-1be41fc01700 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:23.843268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1c27fef5-3666-4a5d-a804-1be41fc01700 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:23.884259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1c27fef5-3666-4a5d-a804-1be41fc01700 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058427 Aug 28 01:02:23.884865 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 430/861] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:23 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:02:24.366686 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-991682d5-c9e8-4df2-80d1-62ba1cad87f0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:24.367306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-991682d5-c9e8-4df2-80d1-62ba1cad87f0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:24.374433 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-991682d5-c9e8-4df2-80d1-62ba1cad87f0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:24.374842 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-991682d5-c9e8-4df2-80d1-62ba1cad87f0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:24.375259 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-991682d5-c9e8-4df2-80d1-62ba1cad87f0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:24.376026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-991682d5-c9e8-4df2-80d1-62ba1cad87f0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:24.376485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-991682d5-c9e8-4df2-80d1-62ba1cad87f0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:24.376896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-991682d5-c9e8-4df2-80d1-62ba1cad87f0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:24.381940 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:02:24.448938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-991682d5-c9e8-4df2-80d1-62ba1cad87f0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:24.457531 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-991682d5-c9e8-4df2-80d1-62ba1cad87f0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.094740 Aug 28 01:02:24.458118 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 432/862] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:24 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:24.843596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ec70f4e0-b549-48b9-b31b-fb4d4aa24cc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:24.845858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ec70f4e0-b549-48b9-b31b-fb4d4aa24cc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:24.853479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec70f4e0-b549-48b9-b31b-fb4d4aa24cc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:24.854111 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec70f4e0-b549-48b9-b31b-fb4d4aa24cc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:24.854636 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec70f4e0-b549-48b9-b31b-fb4d4aa24cc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:24.855439 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec70f4e0-b549-48b9-b31b-fb4d4aa24cc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:24.855979 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec70f4e0-b549-48b9-b31b-fb4d4aa24cc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:24.856489 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec70f4e0-b549-48b9-b31b-fb4d4aa24cc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:24.891584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ec70f4e0-b549-48b9-b31b-fb4d4aa24cc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052251 Aug 28 01:02:24.892455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 431/863] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:24 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:24.898313 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-399f12cd-31ff-45d2-b678-0062ab4c9a7c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:24.900532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-399f12cd-31ff-45d2-b678-0062ab4c9a7c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:24.908053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-399f12cd-31ff-45d2-b678-0062ab4c9a7c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:24.908453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-399f12cd-31ff-45d2-b678-0062ab4c9a7c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:24.908978 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-399f12cd-31ff-45d2-b678-0062ab4c9a7c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:24.909709 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-399f12cd-31ff-45d2-b678-0062ab4c9a7c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:24.910150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-399f12cd-31ff-45d2-b678-0062ab4c9a7c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:24.910570 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-399f12cd-31ff-45d2-b678-0062ab4c9a7c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:24.945873 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-399f12cd-31ff-45d2-b678-0062ab4c9a7c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051830 Aug 28 01:02:24.946572 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 433/864] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:24 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:02:25.482722 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b6476305-41ea-4c55-9cb2-f61ecdc9ee06 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:25.483803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b6476305-41ea-4c55-9cb2-f61ecdc9ee06 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:25.494606 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b6476305-41ea-4c55-9cb2-f61ecdc9ee06 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:25.495443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b6476305-41ea-4c55-9cb2-f61ecdc9ee06 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:25.496184 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b6476305-41ea-4c55-9cb2-f61ecdc9ee06 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:25.497635 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b6476305-41ea-4c55-9cb2-f61ecdc9ee06 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:25.498436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b6476305-41ea-4c55-9cb2-f61ecdc9ee06 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:25.499193 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b6476305-41ea-4c55-9cb2-f61ecdc9ee06 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:25.590714 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b6476305-41ea-4c55-9cb2-f61ecdc9ee06 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:25.599338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b6476305-41ea-4c55-9cb2-f61ecdc9ee06 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.123983 Aug 28 01:02:25.599958 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 432/865] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:25 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:25.908808 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0eebbf52-1ef3-4635-9755-274c3d9675c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:25.912154 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0eebbf52-1ef3-4635-9755-274c3d9675c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:25.922475 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0eebbf52-1ef3-4635-9755-274c3d9675c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:25.923321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0eebbf52-1ef3-4635-9755-274c3d9675c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:25.924110 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0eebbf52-1ef3-4635-9755-274c3d9675c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:25.925403 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0eebbf52-1ef3-4635-9755-274c3d9675c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:25.926210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0eebbf52-1ef3-4635-9755-274c3d9675c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:25.926978 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0eebbf52-1ef3-4635-9755-274c3d9675c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:25.961060 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f5bfab02-b727-4a92-a0dc-a574070a1601 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:25.963614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f5bfab02-b727-4a92-a0dc-a574070a1601 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:25.970448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5bfab02-b727-4a92-a0dc-a574070a1601 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:25.971021 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5bfab02-b727-4a92-a0dc-a574070a1601 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:25.971552 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5bfab02-b727-4a92-a0dc-a574070a1601 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:25.972253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5bfab02-b727-4a92-a0dc-a574070a1601 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:25.972682 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5bfab02-b727-4a92-a0dc-a574070a1601 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:25.973301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5bfab02-b727-4a92-a0dc-a574070a1601 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:25.985039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0eebbf52-1ef3-4635-9755-274c3d9675c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.083029 Aug 28 01:02:25.986064 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 434/866] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:25 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:26.011034 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f5bfab02-b727-4a92-a0dc-a574070a1601 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055088 Aug 28 01:02:26.011640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 433/867] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:25 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:26.624494 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9aec42da-eadf-40e9-a1b4-9ce0edc5aa0c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:26.625678 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9aec42da-eadf-40e9-a1b4-9ce0edc5aa0c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:26.636725 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9aec42da-eadf-40e9-a1b4-9ce0edc5aa0c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:26.637603 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9aec42da-eadf-40e9-a1b4-9ce0edc5aa0c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:26.638360 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9aec42da-eadf-40e9-a1b4-9ce0edc5aa0c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:26.639612 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9aec42da-eadf-40e9-a1b4-9ce0edc5aa0c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:26.640414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9aec42da-eadf-40e9-a1b4-9ce0edc5aa0c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:26.641246 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9aec42da-eadf-40e9-a1b4-9ce0edc5aa0c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:26.734051 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9aec42da-eadf-40e9-a1b4-9ce0edc5aa0c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:26.743002 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9aec42da-eadf-40e9-a1b4-9ce0edc5aa0c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.125850 Aug 28 01:02:26.743575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 435/868] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:26 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:26.999368 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-20a37bc6-9018-4363-9fb6-035ef2dfdf15 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:27.001700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-20a37bc6-9018-4363-9fb6-035ef2dfdf15 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:27.009353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-20a37bc6-9018-4363-9fb6-035ef2dfdf15 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:27.010001 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-20a37bc6-9018-4363-9fb6-035ef2dfdf15 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:27.010560 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-20a37bc6-9018-4363-9fb6-035ef2dfdf15 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:27.011295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-20a37bc6-9018-4363-9fb6-035ef2dfdf15 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:27.012102 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-20a37bc6-9018-4363-9fb6-035ef2dfdf15 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:27.012550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-20a37bc6-9018-4363-9fb6-035ef2dfdf15 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:27.017046 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:02:27.029829 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ec1d4730-5f9c-43fe-9d22-f303e87b7558 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:27.033559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ec1d4730-5f9c-43fe-9d22-f303e87b7558 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:27.042460 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ec1d4730-5f9c-43fe-9d22-f303e87b7558 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:27.043163 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ec1d4730-5f9c-43fe-9d22-f303e87b7558 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:27.043702 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ec1d4730-5f9c-43fe-9d22-f303e87b7558 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:27.044591 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ec1d4730-5f9c-43fe-9d22-f303e87b7558 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:27.045174 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ec1d4730-5f9c-43fe-9d22-f303e87b7558 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:27.045891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ec1d4730-5f9c-43fe-9d22-f303e87b7558 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:27.050315 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-20a37bc6-9018-4363-9fb6-035ef2dfdf15 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055048 Aug 28 01:02:27.051044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 434/869] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:26 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:27.083478 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ec1d4730-5f9c-43fe-9d22-f303e87b7558 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060187 Aug 28 01:02:27.084065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 436/870] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:27 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:02:27.765832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c0de6575-ee0a-494c-b79a-ccfc08dee650 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:27.766443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c0de6575-ee0a-494c-b79a-ccfc08dee650 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:27.774139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0de6575-ee0a-494c-b79a-ccfc08dee650 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:27.774682 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0de6575-ee0a-494c-b79a-ccfc08dee650 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:27.775194 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0de6575-ee0a-494c-b79a-ccfc08dee650 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:27.775951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0de6575-ee0a-494c-b79a-ccfc08dee650 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:27.776445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0de6575-ee0a-494c-b79a-ccfc08dee650 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:27.776907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0de6575-ee0a-494c-b79a-ccfc08dee650 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:27.849962 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c0de6575-ee0a-494c-b79a-ccfc08dee650 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:27.858428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c0de6575-ee0a-494c-b79a-ccfc08dee650 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.096486 Aug 28 01:02:27.858988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 435/871] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:27 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:28.064112 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cb9cc3e0-d915-4aa3-ad4b-28ae2078f410 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:28.066517 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cb9cc3e0-d915-4aa3-ad4b-28ae2078f410 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:28.074858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb9cc3e0-d915-4aa3-ad4b-28ae2078f410 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:28.075353 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb9cc3e0-d915-4aa3-ad4b-28ae2078f410 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:28.075780 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb9cc3e0-d915-4aa3-ad4b-28ae2078f410 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:28.076512 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb9cc3e0-d915-4aa3-ad4b-28ae2078f410 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:28.076940 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb9cc3e0-d915-4aa3-ad4b-28ae2078f410 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:28.077574 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb9cc3e0-d915-4aa3-ad4b-28ae2078f410 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:28.099548 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-60d3ce5c-3956-4646-a587-dea51dc50e11 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:28.103008 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-60d3ce5c-3956-4646-a587-dea51dc50e11 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:28.114034 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cb9cc3e0-d915-4aa3-ad4b-28ae2078f410 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054154 Aug 28 01:02:28.114371 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60d3ce5c-3956-4646-a587-dea51dc50e11 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:28.119858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 437/872] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:28 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:28.120088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60d3ce5c-3956-4646-a587-dea51dc50e11 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:28.120088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60d3ce5c-3956-4646-a587-dea51dc50e11 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:28.120088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60d3ce5c-3956-4646-a587-dea51dc50e11 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:28.120088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60d3ce5c-3956-4646-a587-dea51dc50e11 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:28.120408 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60d3ce5c-3956-4646-a587-dea51dc50e11 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:28.164497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-60d3ce5c-3956-4646-a587-dea51dc50e11 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.071914 Aug 28 01:02:28.165128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 436/873] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:28 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:28.887756 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-85dd8751-89cb-49d5-8442-e494c207d5d0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:28.888465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-85dd8751-89cb-49d5-8442-e494c207d5d0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:28.896664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85dd8751-89cb-49d5-8442-e494c207d5d0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:28.897211 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85dd8751-89cb-49d5-8442-e494c207d5d0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:28.897682 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85dd8751-89cb-49d5-8442-e494c207d5d0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:28.898449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85dd8751-89cb-49d5-8442-e494c207d5d0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:28.898899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85dd8751-89cb-49d5-8442-e494c207d5d0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:28.899318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85dd8751-89cb-49d5-8442-e494c207d5d0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:28.985868 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-85dd8751-89cb-49d5-8442-e494c207d5d0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:28.995073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-85dd8751-89cb-49d5-8442-e494c207d5d0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.111476 Aug 28 01:02:28.995672 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 438/874] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:28 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:29.127979 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-392994ef-0141-469b-bafd-20b2973e5819 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:29.130311 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-392994ef-0141-469b-bafd-20b2973e5819 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:29.137654 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-392994ef-0141-469b-bafd-20b2973e5819 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:29.138127 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-392994ef-0141-469b-bafd-20b2973e5819 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:29.138563 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-392994ef-0141-469b-bafd-20b2973e5819 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:29.139279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-392994ef-0141-469b-bafd-20b2973e5819 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:29.139732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-392994ef-0141-469b-bafd-20b2973e5819 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:29.140157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-392994ef-0141-469b-bafd-20b2973e5819 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:29.178680 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6dc1d3e1-08b4-4ad5-af79-59316bc18847 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:29.179770 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-392994ef-0141-469b-bafd-20b2973e5819 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055392 Aug 28 01:02:29.180885 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 437/875] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:29 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:29.181650 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6dc1d3e1-08b4-4ad5-af79-59316bc18847 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:29.189395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6dc1d3e1-08b4-4ad5-af79-59316bc18847 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:29.189891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6dc1d3e1-08b4-4ad5-af79-59316bc18847 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:29.190344 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6dc1d3e1-08b4-4ad5-af79-59316bc18847 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:29.191081 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6dc1d3e1-08b4-4ad5-af79-59316bc18847 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:29.191527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6dc1d3e1-08b4-4ad5-af79-59316bc18847 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:29.191942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6dc1d3e1-08b4-4ad5-af79-59316bc18847 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:29.224639 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6dc1d3e1-08b4-4ad5-af79-59316bc18847 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051357 Aug 28 01:02:29.225260 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 439/876] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:29 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:02:30.016068 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5ceba619-f6d0-44ea-abd2-c0489933467e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:30.016676 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5ceba619-f6d0-44ea-abd2-c0489933467e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:30.023461 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ceba619-f6d0-44ea-abd2-c0489933467e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:30.023949 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ceba619-f6d0-44ea-abd2-c0489933467e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:30.024374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ceba619-f6d0-44ea-abd2-c0489933467e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:30.025156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ceba619-f6d0-44ea-abd2-c0489933467e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:30.025609 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ceba619-f6d0-44ea-abd2-c0489933467e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:30.026023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ceba619-f6d0-44ea-abd2-c0489933467e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:30.095841 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5ceba619-f6d0-44ea-abd2-c0489933467e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:30.104609 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5ceba619-f6d0-44ea-abd2-c0489933467e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.092373 Aug 28 01:02:30.105222 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 438/877] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:30 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:30.196035 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-38995054-1f68-4c02-b3d6-7015b725772c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:30.198330 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-38995054-1f68-4c02-b3d6-7015b725772c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:30.205684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38995054-1f68-4c02-b3d6-7015b725772c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:30.206172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38995054-1f68-4c02-b3d6-7015b725772c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:30.206604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38995054-1f68-4c02-b3d6-7015b725772c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:30.208169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38995054-1f68-4c02-b3d6-7015b725772c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:30.208625 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38995054-1f68-4c02-b3d6-7015b725772c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:30.209092 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38995054-1f68-4c02-b3d6-7015b725772c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:30.237715 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-45f41acf-39f6-41b0-89f0-1b85abd90f4d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:30.240913 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-45f41acf-39f6-41b0-89f0-1b85abd90f4d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:30.246633 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-38995054-1f68-4c02-b3d6-7015b725772c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054762 Aug 28 01:02:30.247635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 440/878] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:30 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:30.248202 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45f41acf-39f6-41b0-89f0-1b85abd90f4d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:30.248757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45f41acf-39f6-41b0-89f0-1b85abd90f4d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:30.249228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45f41acf-39f6-41b0-89f0-1b85abd90f4d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:30.249958 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45f41acf-39f6-41b0-89f0-1b85abd90f4d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:30.250423 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45f41acf-39f6-41b0-89f0-1b85abd90f4d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:30.250842 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45f41acf-39f6-41b0-89f0-1b85abd90f4d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:30.285477 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-45f41acf-39f6-41b0-89f0-1b85abd90f4d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051714 Aug 28 01:02:30.286043 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 439/879] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:30 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:31.134380 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a16b8775-2512-41d6-bc17-d0a7b97f65a2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:31.135094 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a16b8775-2512-41d6-bc17-d0a7b97f65a2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:31.141952 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a16b8775-2512-41d6-bc17-d0a7b97f65a2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:31.142409 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a16b8775-2512-41d6-bc17-d0a7b97f65a2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:31.142851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a16b8775-2512-41d6-bc17-d0a7b97f65a2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:31.143569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a16b8775-2512-41d6-bc17-d0a7b97f65a2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:31.144024 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a16b8775-2512-41d6-bc17-d0a7b97f65a2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:31.144486 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a16b8775-2512-41d6-bc17-d0a7b97f65a2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:31.232996 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a16b8775-2512-41d6-bc17-d0a7b97f65a2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:31.242069 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a16b8775-2512-41d6-bc17-d0a7b97f65a2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.111825 Aug 28 01:02:31.242645 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 441/880] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:31 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:31.263361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c173495d-59db-40af-94b0-0af02489ca33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:31.265895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c173495d-59db-40af-94b0-0af02489ca33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:31.274292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c173495d-59db-40af-94b0-0af02489ca33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:31.274936 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c173495d-59db-40af-94b0-0af02489ca33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:31.275466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c173495d-59db-40af-94b0-0af02489ca33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:31.276339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c173495d-59db-40af-94b0-0af02489ca33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:31.276924 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c173495d-59db-40af-94b0-0af02489ca33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:31.277498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c173495d-59db-40af-94b0-0af02489ca33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:31.302304 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fc362d92-2051-4115-a3fb-1d4401604dce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:31.305190 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fc362d92-2051-4115-a3fb-1d4401604dce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:31.314279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c173495d-59db-40af-94b0-0af02489ca33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055544 Aug 28 01:02:31.319078 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 440/881] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:31 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:31.323021 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc362d92-2051-4115-a3fb-1d4401604dce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:31.323520 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc362d92-2051-4115-a3fb-1d4401604dce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:31.324014 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc362d92-2051-4115-a3fb-1d4401604dce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:31.324752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc362d92-2051-4115-a3fb-1d4401604dce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:31.325297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc362d92-2051-4115-a3fb-1d4401604dce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:31.325734 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc362d92-2051-4115-a3fb-1d4401604dce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:31.360872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fc362d92-2051-4115-a3fb-1d4401604dce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.065614 Aug 28 01:02:31.361540 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 442/882] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:31 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:32.264853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6e5228a9-3fcd-43d1-8481-496c7f511f26 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:32.265585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6e5228a9-3fcd-43d1-8481-496c7f511f26 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:32.272820 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e5228a9-3fcd-43d1-8481-496c7f511f26 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:32.273353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e5228a9-3fcd-43d1-8481-496c7f511f26 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:32.273787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e5228a9-3fcd-43d1-8481-496c7f511f26 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:32.274590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e5228a9-3fcd-43d1-8481-496c7f511f26 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:32.274969 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e5228a9-3fcd-43d1-8481-496c7f511f26 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:32.275406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e5228a9-3fcd-43d1-8481-496c7f511f26 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:32.332994 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8779b691-3155-4b1a-a365-d7e8916fbe91 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:32.337083 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8779b691-3155-4b1a-a365-d7e8916fbe91 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:32.349510 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8779b691-3155-4b1a-a365-d7e8916fbe91 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:32.350348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8779b691-3155-4b1a-a365-d7e8916fbe91 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:32.351103 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8779b691-3155-4b1a-a365-d7e8916fbe91 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:32.352248 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8779b691-3155-4b1a-a365-d7e8916fbe91 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:32.352933 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8779b691-3155-4b1a-a365-d7e8916fbe91 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:32.353569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8779b691-3155-4b1a-a365-d7e8916fbe91 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:32.359570 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6e5228a9-3fcd-43d1-8481-496c7f511f26 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:32.372521 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6e5228a9-3fcd-43d1-8481-496c7f511f26 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.111797 Aug 28 01:02:32.373183 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 441/883] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:32 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:32.378905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1e9042c0-a5c7-4bde-ba4a-c02678d3f56a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:32.381404 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1e9042c0-a5c7-4bde-ba4a-c02678d3f56a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:32.387666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1e9042c0-a5c7-4bde-ba4a-c02678d3f56a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:32.388161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1e9042c0-a5c7-4bde-ba4a-c02678d3f56a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:32.388586 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1e9042c0-a5c7-4bde-ba4a-c02678d3f56a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:32.389322 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1e9042c0-a5c7-4bde-ba4a-c02678d3f56a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:32.389769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1e9042c0-a5c7-4bde-ba4a-c02678d3f56a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:32.390183 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1e9042c0-a5c7-4bde-ba4a-c02678d3f56a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:32.411941 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8779b691-3155-4b1a-a365-d7e8916fbe91 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.084616 Aug 28 01:02:32.412982 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 443/884] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:32 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:32.428837 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1e9042c0-a5c7-4bde-ba4a-c02678d3f56a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054208 Aug 28 01:02:32.429466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 442/885] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:32 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:33.394703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2afb2929-ad01-47da-adf8-e7b47b578604 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:33.395318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2afb2929-ad01-47da-adf8-e7b47b578604 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:33.402293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2afb2929-ad01-47da-adf8-e7b47b578604 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:33.402784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2afb2929-ad01-47da-adf8-e7b47b578604 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:33.403214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2afb2929-ad01-47da-adf8-e7b47b578604 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:33.403909 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2afb2929-ad01-47da-adf8-e7b47b578604 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:33.404356 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2afb2929-ad01-47da-adf8-e7b47b578604 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:33.404771 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2afb2929-ad01-47da-adf8-e7b47b578604 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:33.430466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7efc480a-6cbe-4d8e-a099-d67f611066a1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:33.432749 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7efc480a-6cbe-4d8e-a099-d67f611066a1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:33.439950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7efc480a-6cbe-4d8e-a099-d67f611066a1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:33.440457 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7efc480a-6cbe-4d8e-a099-d67f611066a1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:33.440877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7efc480a-6cbe-4d8e-a099-d67f611066a1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:33.441644 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7efc480a-6cbe-4d8e-a099-d67f611066a1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:33.442095 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7efc480a-6cbe-4d8e-a099-d67f611066a1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:33.442818 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7efc480a-6cbe-4d8e-a099-d67f611066a1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:33.478456 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7efc480a-6cbe-4d8e-a099-d67f611066a1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055373 Aug 28 01:02:33.479045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 443/886] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:33 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:33.484475 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-872a7de5-d808-4c23-9a05-bd7b050ab5a1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:33.485961 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-872a7de5-d808-4c23-9a05-bd7b050ab5a1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:33.487106 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2afb2929-ad01-47da-adf8-e7b47b578604 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:33.492411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-872a7de5-d808-4c23-9a05-bd7b050ab5a1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:33.492926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-872a7de5-d808-4c23-9a05-bd7b050ab5a1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:33.493445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-872a7de5-d808-4c23-9a05-bd7b050ab5a1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:33.494170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-872a7de5-d808-4c23-9a05-bd7b050ab5a1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:33.494634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-872a7de5-d808-4c23-9a05-bd7b050ab5a1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:33.495039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-872a7de5-d808-4c23-9a05-bd7b050ab5a1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:33.499106 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2afb2929-ad01-47da-adf8-e7b47b578604 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.108402 Aug 28 01:02:33.499934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 444/887] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:33 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:33.529994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-872a7de5-d808-4c23-9a05-bd7b050ab5a1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049688 Aug 28 01:02:33.530926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 444/888] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:33 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:34.493961 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a4c81179-51b9-4cc5-a7d4-19b3738e81da tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:34.497063 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a4c81179-51b9-4cc5-a7d4-19b3738e81da tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:34.505941 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a4c81179-51b9-4cc5-a7d4-19b3738e81da tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:34.506520 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a4c81179-51b9-4cc5-a7d4-19b3738e81da tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:34.507014 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a4c81179-51b9-4cc5-a7d4-19b3738e81da tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:34.507828 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a4c81179-51b9-4cc5-a7d4-19b3738e81da tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:34.508274 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a4c81179-51b9-4cc5-a7d4-19b3738e81da tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:34.508696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a4c81179-51b9-4cc5-a7d4-19b3738e81da tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:34.521240 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8826f587-90e6-4035-bf0d-9fa462835f16 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:34.522049 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8826f587-90e6-4035-bf0d-9fa462835f16 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:34.528760 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8826f587-90e6-4035-bf0d-9fa462835f16 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:34.529763 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8826f587-90e6-4035-bf0d-9fa462835f16 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:34.530353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8826f587-90e6-4035-bf0d-9fa462835f16 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:34.531084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8826f587-90e6-4035-bf0d-9fa462835f16 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:34.531542 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8826f587-90e6-4035-bf0d-9fa462835f16 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:34.531970 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8826f587-90e6-4035-bf0d-9fa462835f16 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:34.543062 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a4c81179-51b9-4cc5-a7d4-19b3738e81da tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054615 Aug 28 01:02:34.543678 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 445/889] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:34 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 28 01:02:34.548622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2fc97a3d-2bd3-4288-a242-f19bfdfbf312 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:34.550940 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2fc97a3d-2bd3-4288-a242-f19bfdfbf312 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:34.557569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fc97a3d-2bd3-4288-a242-f19bfdfbf312 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:34.558049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fc97a3d-2bd3-4288-a242-f19bfdfbf312 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:34.558484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fc97a3d-2bd3-4288-a242-f19bfdfbf312 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:34.559213 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fc97a3d-2bd3-4288-a242-f19bfdfbf312 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:34.559659 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fc97a3d-2bd3-4288-a242-f19bfdfbf312 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:34.560181 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fc97a3d-2bd3-4288-a242-f19bfdfbf312 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:34.612791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2fc97a3d-2bd3-4288-a242-f19bfdfbf312 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067567 Aug 28 01:02:34.613851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 446/890] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:34 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:34.621228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8826f587-90e6-4035-bf0d-9fa462835f16 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:34.630516 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8826f587-90e6-4035-bf0d-9fa462835f16 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.113328 Aug 28 01:02:34.631159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 445/891] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:34 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:35.560640 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-335bb138-1734-4adb-905e-26d467c827dc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:35.562860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-335bb138-1734-4adb-905e-26d467c827dc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:35.565493 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=84 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:02:35.571385 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-335bb138-1734-4adb-905e-26d467c827dc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:35.571882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-335bb138-1734-4adb-905e-26d467c827dc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:35.572285 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-335bb138-1734-4adb-905e-26d467c827dc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:35.573070 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-335bb138-1734-4adb-905e-26d467c827dc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:35.573525 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-335bb138-1734-4adb-905e-26d467c827dc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:35.573941 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-335bb138-1734-4adb-905e-26d467c827dc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:35.607466 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-335bb138-1734-4adb-905e-26d467c827dc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051191 Aug 28 01:02:35.608052 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 447/892] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:35 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:35.627195 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6d363088-cc03-4685-a995-0c6430befc9c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:35.629728 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6d363088-cc03-4685-a995-0c6430befc9c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:35.632641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=86 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:02:35.639102 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d363088-cc03-4685-a995-0c6430befc9c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:35.639669 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d363088-cc03-4685-a995-0c6430befc9c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:35.640145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d363088-cc03-4685-a995-0c6430befc9c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:35.641220 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d363088-cc03-4685-a995-0c6430befc9c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:35.641740 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d363088-cc03-4685-a995-0c6430befc9c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:35.642244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d363088-cc03-4685-a995-0c6430befc9c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:35.656140 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d32afe2a-7a84-431b-9bc2-fcf4f790e86b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:35.657235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d32afe2a-7a84-431b-9bc2-fcf4f790e86b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:35.667975 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d32afe2a-7a84-431b-9bc2-fcf4f790e86b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:35.668768 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d32afe2a-7a84-431b-9bc2-fcf4f790e86b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:35.669429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d32afe2a-7a84-431b-9bc2-fcf4f790e86b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:35.670660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d32afe2a-7a84-431b-9bc2-fcf4f790e86b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:35.671464 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d32afe2a-7a84-431b-9bc2-fcf4f790e86b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:35.672211 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d32afe2a-7a84-431b-9bc2-fcf4f790e86b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:35.676931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6d363088-cc03-4685-a995-0c6430befc9c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053934 Aug 28 01:02:35.677544 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 446/893] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:35 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:35.741648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d32afe2a-7a84-431b-9bc2-fcf4f790e86b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:35.745724 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:02:35.751945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d32afe2a-7a84-431b-9bc2-fcf4f790e86b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.102962 Aug 28 01:02:35.752593 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 448/894] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:35 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:02:36.620626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0b458494-8579-4637-92a9-f06f0f841152 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:36.622998 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0b458494-8579-4637-92a9-f06f0f841152 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:36.629981 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0b458494-8579-4637-92a9-f06f0f841152 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:36.630783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0b458494-8579-4637-92a9-f06f0f841152 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:36.631019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0b458494-8579-4637-92a9-f06f0f841152 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:36.631793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0b458494-8579-4637-92a9-f06f0f841152 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:36.632255 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0b458494-8579-4637-92a9-f06f0f841152 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:36.632838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0b458494-8579-4637-92a9-f06f0f841152 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:36.669381 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0b458494-8579-4637-92a9-f06f0f841152 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052862 Aug 28 01:02:36.670023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 447/895] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:36 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:36.691964 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-09e0116f-5043-4036-9ef8-1ad233daba7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:36.694628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-09e0116f-5043-4036-9ef8-1ad233daba7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:36.702172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-09e0116f-5043-4036-9ef8-1ad233daba7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:36.702683 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-09e0116f-5043-4036-9ef8-1ad233daba7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:36.703118 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-09e0116f-5043-4036-9ef8-1ad233daba7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:36.703871 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-09e0116f-5043-4036-9ef8-1ad233daba7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:36.704326 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-09e0116f-5043-4036-9ef8-1ad233daba7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:36.704747 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-09e0116f-5043-4036-9ef8-1ad233daba7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:36.738794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-09e0116f-5043-4036-9ef8-1ad233daba7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051066 Aug 28 01:02:36.739471 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 449/896] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:36 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:36.785287 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-951286cc-2efe-47bf-9b8d-a737f731b233 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:36.785968 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-951286cc-2efe-47bf-9b8d-a737f731b233 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:36.792383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-951286cc-2efe-47bf-9b8d-a737f731b233 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:36.792885 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-951286cc-2efe-47bf-9b8d-a737f731b233 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:36.793353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-951286cc-2efe-47bf-9b8d-a737f731b233 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:36.794084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-951286cc-2efe-47bf-9b8d-a737f731b233 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:36.794536 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-951286cc-2efe-47bf-9b8d-a737f731b233 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:36.794958 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-951286cc-2efe-47bf-9b8d-a737f731b233 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:36.862165 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-951286cc-2efe-47bf-9b8d-a737f731b233 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:36.871062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-951286cc-2efe-47bf-9b8d-a737f731b233 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.089740 Aug 28 01:02:36.871649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 448/897] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:36 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:02:37.682830 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7cfdfb3a-5ec9-42f3-89d7-220f78d87b7a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:37.685123 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7cfdfb3a-5ec9-42f3-89d7-220f78d87b7a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:37.694566 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7cfdfb3a-5ec9-42f3-89d7-220f78d87b7a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:37.695296 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7cfdfb3a-5ec9-42f3-89d7-220f78d87b7a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:37.695452 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7cfdfb3a-5ec9-42f3-89d7-220f78d87b7a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:37.696198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7cfdfb3a-5ec9-42f3-89d7-220f78d87b7a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:37.696646 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7cfdfb3a-5ec9-42f3-89d7-220f78d87b7a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:37.697087 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7cfdfb3a-5ec9-42f3-89d7-220f78d87b7a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:37.734934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7cfdfb3a-5ec9-42f3-89d7-220f78d87b7a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.056241 Aug 28 01:02:37.735502 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 450/898] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:37 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:37.752372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e8468524-790e-470b-a559-4e4bed009a91 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:37.754641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e8468524-790e-470b-a559-4e4bed009a91 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:37.761443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e8468524-790e-470b-a559-4e4bed009a91 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:37.761918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e8468524-790e-470b-a559-4e4bed009a91 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:37.762356 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e8468524-790e-470b-a559-4e4bed009a91 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:37.763057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e8468524-790e-470b-a559-4e4bed009a91 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:37.763508 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e8468524-790e-470b-a559-4e4bed009a91 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:37.763928 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e8468524-790e-470b-a559-4e4bed009a91 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:37.798067 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e8468524-790e-470b-a559-4e4bed009a91 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049732 Aug 28 01:02:37.798673 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 449/899] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:37 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:37.893769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-28245995-f0a2-40cc-9682-7eea9eb32563 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:37.894424 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-28245995-f0a2-40cc-9682-7eea9eb32563 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:37.901217 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-28245995-f0a2-40cc-9682-7eea9eb32563 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:37.901728 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-28245995-f0a2-40cc-9682-7eea9eb32563 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:37.902173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-28245995-f0a2-40cc-9682-7eea9eb32563 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:37.902888 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-28245995-f0a2-40cc-9682-7eea9eb32563 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:37.903341 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-28245995-f0a2-40cc-9682-7eea9eb32563 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:37.903767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-28245995-f0a2-40cc-9682-7eea9eb32563 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:37.974429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-28245995-f0a2-40cc-9682-7eea9eb32563 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:37.983206 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-28245995-f0a2-40cc-9682-7eea9eb32563 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.093813 Aug 28 01:02:37.983779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 451/900] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:37 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:38.751842 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-10acb08f-4b73-49d7-9a12-1ccc8b5dad1a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:38.754588 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-10acb08f-4b73-49d7-9a12-1ccc8b5dad1a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:38.762019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10acb08f-4b73-49d7-9a12-1ccc8b5dad1a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:38.762728 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10acb08f-4b73-49d7-9a12-1ccc8b5dad1a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:38.763387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10acb08f-4b73-49d7-9a12-1ccc8b5dad1a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:38.764269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10acb08f-4b73-49d7-9a12-1ccc8b5dad1a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:38.764795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10acb08f-4b73-49d7-9a12-1ccc8b5dad1a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:38.765310 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10acb08f-4b73-49d7-9a12-1ccc8b5dad1a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:38.795681 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:02:38.802285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-10acb08f-4b73-49d7-9a12-1ccc8b5dad1a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054672 Aug 28 01:02:38.802869 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 450/901] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:38 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:38.811698 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-90ebf032-aa38-4b06-be72-af820aa21f6f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:38.814519 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-90ebf032-aa38-4b06-be72-af820aa21f6f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:38.822153 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90ebf032-aa38-4b06-be72-af820aa21f6f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:38.822656 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90ebf032-aa38-4b06-be72-af820aa21f6f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:38.823119 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90ebf032-aa38-4b06-be72-af820aa21f6f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:38.823841 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90ebf032-aa38-4b06-be72-af820aa21f6f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:38.824312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90ebf032-aa38-4b06-be72-af820aa21f6f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:38.824757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90ebf032-aa38-4b06-be72-af820aa21f6f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:38.860746 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-90ebf032-aa38-4b06-be72-af820aa21f6f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053193 Aug 28 01:02:38.861820 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 452/902] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:38 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:39.074780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-41e5d59d-1493-4e67-a891-ecdb92d7b7d4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:39.075336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-41e5d59d-1493-4e67-a891-ecdb92d7b7d4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:39.081859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-41e5d59d-1493-4e67-a891-ecdb92d7b7d4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:39.082353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-41e5d59d-1493-4e67-a891-ecdb92d7b7d4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:39.082784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-41e5d59d-1493-4e67-a891-ecdb92d7b7d4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:39.083499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-41e5d59d-1493-4e67-a891-ecdb92d7b7d4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:39.083951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-41e5d59d-1493-4e67-a891-ecdb92d7b7d4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:39.084371 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-41e5d59d-1493-4e67-a891-ecdb92d7b7d4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:39.158737 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-41e5d59d-1493-4e67-a891-ecdb92d7b7d4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:39.167225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-41e5d59d-1493-4e67-a891-ecdb92d7b7d4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.096619 Aug 28 01:02:39.167828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 451/903] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:39 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:39.825102 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-842f2393-71b7-4ffa-b5a7-a907da4e991b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:39.828689 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-842f2393-71b7-4ffa-b5a7-a907da4e991b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:39.842737 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-842f2393-71b7-4ffa-b5a7-a907da4e991b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:39.843595 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-842f2393-71b7-4ffa-b5a7-a907da4e991b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:39.844398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-842f2393-71b7-4ffa-b5a7-a907da4e991b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:39.846132 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-842f2393-71b7-4ffa-b5a7-a907da4e991b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:39.846943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-842f2393-71b7-4ffa-b5a7-a907da4e991b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:39.847713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-842f2393-71b7-4ffa-b5a7-a907da4e991b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:39.874539 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a4ca2e87-4612-4b7a-9c3a-18def2df5b37 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:39.877710 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a4ca2e87-4612-4b7a-9c3a-18def2df5b37 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:39.886116 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4ca2e87-4612-4b7a-9c3a-18def2df5b37 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:39.886806 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4ca2e87-4612-4b7a-9c3a-18def2df5b37 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:39.887511 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4ca2e87-4612-4b7a-9c3a-18def2df5b37 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:39.888510 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4ca2e87-4612-4b7a-9c3a-18def2df5b37 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:39.889157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4ca2e87-4612-4b7a-9c3a-18def2df5b37 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:39.889828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4ca2e87-4612-4b7a-9c3a-18def2df5b37 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:39.895757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-842f2393-71b7-4ffa-b5a7-a907da4e991b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.079236 Aug 28 01:02:39.896379 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 453/904] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:39 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:39.924569 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a4ca2e87-4612-4b7a-9c3a-18def2df5b37 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054253 Aug 28 01:02:39.925178 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 452/905] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:39 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:40.201149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-42d54035-c00b-4eaa-9e3c-a4db35162aa1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:40.201793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-42d54035-c00b-4eaa-9e3c-a4db35162aa1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:40.209683 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42d54035-c00b-4eaa-9e3c-a4db35162aa1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:40.210511 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42d54035-c00b-4eaa-9e3c-a4db35162aa1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:40.211241 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42d54035-c00b-4eaa-9e3c-a4db35162aa1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:40.212180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42d54035-c00b-4eaa-9e3c-a4db35162aa1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:40.212762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42d54035-c00b-4eaa-9e3c-a4db35162aa1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:40.213375 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42d54035-c00b-4eaa-9e3c-a4db35162aa1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:40.280408 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-42d54035-c00b-4eaa-9e3c-a4db35162aa1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:40.292510 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-42d54035-c00b-4eaa-9e3c-a4db35162aa1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.095265 Aug 28 01:02:40.293141 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 454/906] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:40 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:40.908758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ff4f97a8-e656-4d75-81b7-660a0abfc28f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:40.910979 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ff4f97a8-e656-4d75-81b7-660a0abfc28f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:40.918641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ff4f97a8-e656-4d75-81b7-660a0abfc28f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:40.919044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ff4f97a8-e656-4d75-81b7-660a0abfc28f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:40.919471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ff4f97a8-e656-4d75-81b7-660a0abfc28f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:40.920170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ff4f97a8-e656-4d75-81b7-660a0abfc28f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:40.920645 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ff4f97a8-e656-4d75-81b7-660a0abfc28f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:40.921159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ff4f97a8-e656-4d75-81b7-660a0abfc28f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:40.938684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c20eddf7-7084-4592-8c85-5410c3e0f6ab tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:40.941851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c20eddf7-7084-4592-8c85-5410c3e0f6ab tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:40.950163 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c20eddf7-7084-4592-8c85-5410c3e0f6ab tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:40.950784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c20eddf7-7084-4592-8c85-5410c3e0f6ab tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:40.951340 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c20eddf7-7084-4592-8c85-5410c3e0f6ab tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:40.952165 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c20eddf7-7084-4592-8c85-5410c3e0f6ab tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:40.952720 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c20eddf7-7084-4592-8c85-5410c3e0f6ab tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:40.953295 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c20eddf7-7084-4592-8c85-5410c3e0f6ab tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:40.956446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ff4f97a8-e656-4d75-81b7-660a0abfc28f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051884 Aug 28 01:02:40.957074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 453/907] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:40 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:40.986357 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c20eddf7-7084-4592-8c85-5410c3e0f6ab tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052567 Aug 28 01:02:40.986973 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 455/908] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:40 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:02:41.313723 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-dece1e72-7701-48fe-9b92-eaf8c5b847ae tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:41.314210 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-dece1e72-7701-48fe-9b92-eaf8c5b847ae tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:41.321383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dece1e72-7701-48fe-9b92-eaf8c5b847ae tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:41.321874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dece1e72-7701-48fe-9b92-eaf8c5b847ae tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:41.322299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dece1e72-7701-48fe-9b92-eaf8c5b847ae tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:41.323016 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dece1e72-7701-48fe-9b92-eaf8c5b847ae tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:41.323460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dece1e72-7701-48fe-9b92-eaf8c5b847ae tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:41.323880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dece1e72-7701-48fe-9b92-eaf8c5b847ae tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:41.400872 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dece1e72-7701-48fe-9b92-eaf8c5b847ae tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:41.410692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-dece1e72-7701-48fe-9b92-eaf8c5b847ae tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.100880 Aug 28 01:02:41.411466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 454/909] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:41 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:41.973009 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-961ff381-8dc8-49fd-8d1f-0fe8c1d44bb6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:41.976222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-961ff381-8dc8-49fd-8d1f-0fe8c1d44bb6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:41.983570 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-961ff381-8dc8-49fd-8d1f-0fe8c1d44bb6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:41.984120 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-961ff381-8dc8-49fd-8d1f-0fe8c1d44bb6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:41.984639 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-961ff381-8dc8-49fd-8d1f-0fe8c1d44bb6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:41.985373 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-961ff381-8dc8-49fd-8d1f-0fe8c1d44bb6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:41.985812 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-961ff381-8dc8-49fd-8d1f-0fe8c1d44bb6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:41.986522 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-961ff381-8dc8-49fd-8d1f-0fe8c1d44bb6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:42.007731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-35395dc0-60a4-4a52-97c9-2d8389769c32 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:42.008611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-35395dc0-60a4-4a52-97c9-2d8389769c32 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:42.021889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-35395dc0-60a4-4a52-97c9-2d8389769c32 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:42.022789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-35395dc0-60a4-4a52-97c9-2d8389769c32 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:42.023595 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-35395dc0-60a4-4a52-97c9-2d8389769c32 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:42.024025 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-961ff381-8dc8-49fd-8d1f-0fe8c1d44bb6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055474 Aug 28 01:02:42.025004 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 456/910] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:41 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:42.025389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-35395dc0-60a4-4a52-97c9-2d8389769c32 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:42.026398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-35395dc0-60a4-4a52-97c9-2d8389769c32 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:42.027132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-35395dc0-60a4-4a52-97c9-2d8389769c32 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:42.081800 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-35395dc0-60a4-4a52-97c9-2d8389769c32 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.085155 Aug 28 01:02:42.083122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 455/911] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:41 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:42.091528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-49a755e6-f11f-465d-a9e5-f40e97e328bb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:42.092036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-49a755e6-f11f-465d-a9e5-f40e97e328bb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:42.098699 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49a755e6-f11f-465d-a9e5-f40e97e328bb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:42.099199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49a755e6-f11f-465d-a9e5-f40e97e328bb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:42.099638 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49a755e6-f11f-465d-a9e5-f40e97e328bb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:42.100365 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49a755e6-f11f-465d-a9e5-f40e97e328bb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:42.100828 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49a755e6-f11f-465d-a9e5-f40e97e328bb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:42.101299 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49a755e6-f11f-465d-a9e5-f40e97e328bb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:42.432563 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-34b1cb92-787a-412d-a563-5ebab22d1790 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:42.433280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-34b1cb92-787a-412d-a563-5ebab22d1790 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:42.440632 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-34b1cb92-787a-412d-a563-5ebab22d1790 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:42.441166 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-34b1cb92-787a-412d-a563-5ebab22d1790 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:42.441604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-34b1cb92-787a-412d-a563-5ebab22d1790 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:42.442331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-34b1cb92-787a-412d-a563-5ebab22d1790 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:42.443153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-34b1cb92-787a-412d-a563-5ebab22d1790 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:42.443575 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-34b1cb92-787a-412d-a563-5ebab22d1790 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:42.512371 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-34b1cb92-787a-412d-a563-5ebab22d1790 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:42.521044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-34b1cb92-787a-412d-a563-5ebab22d1790 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.092512 Aug 28 01:02:42.521655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 456/912] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:42 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:43.037663 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5abc00da-f893-415d-b497-517b076d8be3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:43.040129 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5abc00da-f893-415d-b497-517b076d8be3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:43.047702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5abc00da-f893-415d-b497-517b076d8be3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:43.048227 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5abc00da-f893-415d-b497-517b076d8be3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:43.048801 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5abc00da-f893-415d-b497-517b076d8be3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:43.049717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5abc00da-f893-415d-b497-517b076d8be3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:43.050285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5abc00da-f893-415d-b497-517b076d8be3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:43.050824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5abc00da-f893-415d-b497-517b076d8be3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:43.086604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5abc00da-f893-415d-b497-517b076d8be3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053460 Aug 28 01:02:43.087289 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 457/913] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:43 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:43.098711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a7c389ab-75c8-4933-9c25-20e21d46649a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:43.101001 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a7c389ab-75c8-4933-9c25-20e21d46649a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:43.107918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7c389ab-75c8-4933-9c25-20e21d46649a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:43.108407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7c389ab-75c8-4933-9c25-20e21d46649a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:43.108835 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7c389ab-75c8-4933-9c25-20e21d46649a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:43.109593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7c389ab-75c8-4933-9c25-20e21d46649a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:43.110041 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7c389ab-75c8-4933-9c25-20e21d46649a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:43.110458 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7c389ab-75c8-4933-9c25-20e21d46649a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:43.144708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a7c389ab-75c8-4933-9c25-20e21d46649a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050422 Aug 28 01:02:43.145350 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 458/914] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:43 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:02:43.543432 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b7aca318-9051-4135-86c3-e4b5ad5af180 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:43.544046 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b7aca318-9051-4135-86c3-e4b5ad5af180 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:43.550858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7aca318-9051-4135-86c3-e4b5ad5af180 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:43.551331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7aca318-9051-4135-86c3-e4b5ad5af180 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:43.551794 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7aca318-9051-4135-86c3-e4b5ad5af180 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:43.552506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7aca318-9051-4135-86c3-e4b5ad5af180 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:43.552987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7aca318-9051-4135-86c3-e4b5ad5af180 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:43.553422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7aca318-9051-4135-86c3-e4b5ad5af180 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:43.628000 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b7aca318-9051-4135-86c3-e4b5ad5af180 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:43.636726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b7aca318-9051-4135-86c3-e4b5ad5af180 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.097239 Aug 28 01:02:43.637342 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 459/915] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:43 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:44.100380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4c927bf1-15d9-40de-8bee-aab1786636c6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:44.102672 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4c927bf1-15d9-40de-8bee-aab1786636c6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:44.110074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c927bf1-15d9-40de-8bee-aab1786636c6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:44.110575 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c927bf1-15d9-40de-8bee-aab1786636c6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:44.111012 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c927bf1-15d9-40de-8bee-aab1786636c6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:44.111865 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c927bf1-15d9-40de-8bee-aab1786636c6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:44.112353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c927bf1-15d9-40de-8bee-aab1786636c6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:44.112782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c927bf1-15d9-40de-8bee-aab1786636c6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:44.147607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4c927bf1-15d9-40de-8bee-aab1786636c6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051320 Aug 28 01:02:44.148205 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 460/916] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:44 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 28 01:02:44.161018 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-15d074f1-4991-45e4-8b1a-c178d0dc8b80 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:44.163284 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-15d074f1-4991-45e4-8b1a-c178d0dc8b80 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:44.169987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15d074f1-4991-45e4-8b1a-c178d0dc8b80 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:44.170508 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15d074f1-4991-45e4-8b1a-c178d0dc8b80 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:44.170939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15d074f1-4991-45e4-8b1a-c178d0dc8b80 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:44.171667 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15d074f1-4991-45e4-8b1a-c178d0dc8b80 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:44.172115 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15d074f1-4991-45e4-8b1a-c178d0dc8b80 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:44.172535 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15d074f1-4991-45e4-8b1a-c178d0dc8b80 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:44.207901 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-15d074f1-4991-45e4-8b1a-c178d0dc8b80 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051171 Aug 28 01:02:44.208477 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 461/917] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:44 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:02:44.659580 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f07ead4c-642e-4bb3-b8e9-258309ec5474 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:44.660187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f07ead4c-642e-4bb3-b8e9-258309ec5474 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:44.666985 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f07ead4c-642e-4bb3-b8e9-258309ec5474 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:44.667592 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f07ead4c-642e-4bb3-b8e9-258309ec5474 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:44.668132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f07ead4c-642e-4bb3-b8e9-258309ec5474 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:44.669269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f07ead4c-642e-4bb3-b8e9-258309ec5474 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:44.669639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f07ead4c-642e-4bb3-b8e9-258309ec5474 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:44.670176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f07ead4c-642e-4bb3-b8e9-258309ec5474 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:44.741003 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f07ead4c-642e-4bb3-b8e9-258309ec5474 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:44.755772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f07ead4c-642e-4bb3-b8e9-258309ec5474 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.100185 Aug 28 01:02:44.756788 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 462/918] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:44 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:02:45.163805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-42fc49d6-e04b-4a33-9a07-199ad62363cc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:45.166221 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-42fc49d6-e04b-4a33-9a07-199ad62363cc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:45.174569 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42fc49d6-e04b-4a33-9a07-199ad62363cc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:45.175186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42fc49d6-e04b-4a33-9a07-199ad62363cc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:45.175702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42fc49d6-e04b-4a33-9a07-199ad62363cc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:45.176447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42fc49d6-e04b-4a33-9a07-199ad62363cc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:45.177012 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42fc49d6-e04b-4a33-9a07-199ad62363cc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:45.177544 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42fc49d6-e04b-4a33-9a07-199ad62363cc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:45.215890 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-42fc49d6-e04b-4a33-9a07-199ad62363cc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.056538 Aug 28 01:02:45.216486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 463/919] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:45 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 28 01:02:45.221506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-52770d07-3e6f-4960-85fb-27c59ca45d4e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:45.223702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-52770d07-3e6f-4960-85fb-27c59ca45d4e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:45.230148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52770d07-3e6f-4960-85fb-27c59ca45d4e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:45.230650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52770d07-3e6f-4960-85fb-27c59ca45d4e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:45.231078 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52770d07-3e6f-4960-85fb-27c59ca45d4e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:45.231796 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52770d07-3e6f-4960-85fb-27c59ca45d4e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:45.232237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52770d07-3e6f-4960-85fb-27c59ca45d4e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:45.232650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52770d07-3e6f-4960-85fb-27c59ca45d4e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:45.268317 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-52770d07-3e6f-4960-85fb-27c59ca45d4e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050651 Aug 28 01:02:45.268889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 464/920] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:45 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:45.786043 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-237f7260-03fe-423a-8997-3b08e959c892 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:45.786670 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-237f7260-03fe-423a-8997-3b08e959c892 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:45.793330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-237f7260-03fe-423a-8997-3b08e959c892 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:45.793816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-237f7260-03fe-423a-8997-3b08e959c892 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:45.794245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-237f7260-03fe-423a-8997-3b08e959c892 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:45.794956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-237f7260-03fe-423a-8997-3b08e959c892 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:45.795400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-237f7260-03fe-423a-8997-3b08e959c892 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:45.795804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-237f7260-03fe-423a-8997-3b08e959c892 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:45.887261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-237f7260-03fe-423a-8997-3b08e959c892 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:45.901629 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-237f7260-03fe-423a-8997-3b08e959c892 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.119137 Aug 28 01:02:45.902473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 465/921] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:45 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:46.229542 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d7c30860-d7ac-4630-bccb-6dcb5e7502ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:46.232078 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d7c30860-d7ac-4630-bccb-6dcb5e7502ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:46.239584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c30860-d7ac-4630-bccb-6dcb5e7502ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:46.240080 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c30860-d7ac-4630-bccb-6dcb5e7502ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:46.240541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c30860-d7ac-4630-bccb-6dcb5e7502ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:46.241409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c30860-d7ac-4630-bccb-6dcb5e7502ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:46.241865 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c30860-d7ac-4630-bccb-6dcb5e7502ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:46.242288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c30860-d7ac-4630-bccb-6dcb5e7502ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:46.276218 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d7c30860-d7ac-4630-bccb-6dcb5e7502ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050746 Aug 28 01:02:46.276809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 466/922] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:46 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:46.281632 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-97453a67-7692-4a5c-98e4-d7002bcf6560 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:46.283846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-97453a67-7692-4a5c-98e4-d7002bcf6560 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:46.290893 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97453a67-7692-4a5c-98e4-d7002bcf6560 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:46.291462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97453a67-7692-4a5c-98e4-d7002bcf6560 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:46.291946 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97453a67-7692-4a5c-98e4-d7002bcf6560 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:46.292702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97453a67-7692-4a5c-98e4-d7002bcf6560 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:46.293310 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97453a67-7692-4a5c-98e4-d7002bcf6560 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:46.293740 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97453a67-7692-4a5c-98e4-d7002bcf6560 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:46.312717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:02:46.328937 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-97453a67-7692-4a5c-98e4-d7002bcf6560 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050925 Aug 28 01:02:46.329556 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 467/923] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:46 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:46.934101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c67af57b-5227-4a8c-8e2b-bf4fd7bfa5c4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:46.934769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c67af57b-5227-4a8c-8e2b-bf4fd7bfa5c4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:46.942704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c67af57b-5227-4a8c-8e2b-bf4fd7bfa5c4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:46.943225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c67af57b-5227-4a8c-8e2b-bf4fd7bfa5c4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:46.943698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c67af57b-5227-4a8c-8e2b-bf4fd7bfa5c4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:46.944420 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c67af57b-5227-4a8c-8e2b-bf4fd7bfa5c4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:46.944953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c67af57b-5227-4a8c-8e2b-bf4fd7bfa5c4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:46.945437 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c67af57b-5227-4a8c-8e2b-bf4fd7bfa5c4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:47.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c67af57b-5227-4a8c-8e2b-bf4fd7bfa5c4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:47.026980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c67af57b-5227-4a8c-8e2b-bf4fd7bfa5c4 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.097070 Aug 28 01:02:47.027580 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 468/924] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:46 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:47.291747 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-05a6dcbb-6a88-4fdc-bf21-cb0236e9e193 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:47.294559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-05a6dcbb-6a88-4fdc-bf21-cb0236e9e193 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:47.302154 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05a6dcbb-6a88-4fdc-bf21-cb0236e9e193 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:47.302713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05a6dcbb-6a88-4fdc-bf21-cb0236e9e193 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:47.303212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05a6dcbb-6a88-4fdc-bf21-cb0236e9e193 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:47.304205 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05a6dcbb-6a88-4fdc-bf21-cb0236e9e193 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:47.304692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05a6dcbb-6a88-4fdc-bf21-cb0236e9e193 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:47.305205 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05a6dcbb-6a88-4fdc-bf21-cb0236e9e193 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:47.340130 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-05a6dcbb-6a88-4fdc-bf21-cb0236e9e193 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053269 Aug 28 01:02:47.340765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 469/925] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:47 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 28 01:02:47.346434 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-de5b277e-da29-4db2-b129-fb68c7b63fd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:47.348829 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-de5b277e-da29-4db2-b129-fb68c7b63fd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:47.356397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de5b277e-da29-4db2-b129-fb68c7b63fd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:47.356913 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de5b277e-da29-4db2-b129-fb68c7b63fd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:47.357388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de5b277e-da29-4db2-b129-fb68c7b63fd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:47.358108 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de5b277e-da29-4db2-b129-fb68c7b63fd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:47.358554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de5b277e-da29-4db2-b129-fb68c7b63fd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:47.358954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de5b277e-da29-4db2-b129-fb68c7b63fd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:47.392558 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-de5b277e-da29-4db2-b129-fb68c7b63fd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050409 Aug 28 01:02:47.393518 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 470/926] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:47 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:48.048461 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ca7fb061-5f78-43f5-bf17-ced0d1d6c89c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:48.049781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ca7fb061-5f78-43f5-bf17-ced0d1d6c89c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:48.056773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ca7fb061-5f78-43f5-bf17-ced0d1d6c89c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:48.057344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ca7fb061-5f78-43f5-bf17-ced0d1d6c89c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:48.057745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ca7fb061-5f78-43f5-bf17-ced0d1d6c89c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:48.058498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ca7fb061-5f78-43f5-bf17-ced0d1d6c89c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:48.059198 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ca7fb061-5f78-43f5-bf17-ced0d1d6c89c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:48.059690 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ca7fb061-5f78-43f5-bf17-ced0d1d6c89c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:48.135035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ca7fb061-5f78-43f5-bf17-ced0d1d6c89c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:48.145996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ca7fb061-5f78-43f5-bf17-ced0d1d6c89c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.101377 Aug 28 01:02:48.146803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 471/927] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:48 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:48.355377 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8aace69e-15cc-45f1-977d-b4769a523159 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:48.357819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8aace69e-15cc-45f1-977d-b4769a523159 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:48.364996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8aace69e-15cc-45f1-977d-b4769a523159 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:48.365522 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8aace69e-15cc-45f1-977d-b4769a523159 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:48.365951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8aace69e-15cc-45f1-977d-b4769a523159 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:48.366687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8aace69e-15cc-45f1-977d-b4769a523159 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:48.367144 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8aace69e-15cc-45f1-977d-b4769a523159 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:48.367564 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8aace69e-15cc-45f1-977d-b4769a523159 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:48.405744 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8aace69e-15cc-45f1-977d-b4769a523159 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054805 Aug 28 01:02:48.406583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 472/928] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:48 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:48.412336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f7d59666-81d1-4926-8f1f-83c3629f142f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:48.414796 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f7d59666-81d1-4926-8f1f-83c3629f142f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:48.422073 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7d59666-81d1-4926-8f1f-83c3629f142f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:48.422564 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7d59666-81d1-4926-8f1f-83c3629f142f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:48.423029 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7d59666-81d1-4926-8f1f-83c3629f142f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:48.423762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7d59666-81d1-4926-8f1f-83c3629f142f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:48.424231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7d59666-81d1-4926-8f1f-83c3629f142f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:48.424654 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7d59666-81d1-4926-8f1f-83c3629f142f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:48.460813 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f7d59666-81d1-4926-8f1f-83c3629f142f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053013 Aug 28 01:02:48.461537 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 473/929] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:48 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:49.169734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-900e111c-89b5-40e9-b3e3-e9cf0ec789aa tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:49.170514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-900e111c-89b5-40e9-b3e3-e9cf0ec789aa tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:49.178458 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-900e111c-89b5-40e9-b3e3-e9cf0ec789aa tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:49.179019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-900e111c-89b5-40e9-b3e3-e9cf0ec789aa tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:49.179490 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-900e111c-89b5-40e9-b3e3-e9cf0ec789aa tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:49.180246 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-900e111c-89b5-40e9-b3e3-e9cf0ec789aa tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:49.180826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-900e111c-89b5-40e9-b3e3-e9cf0ec789aa tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:49.181335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-900e111c-89b5-40e9-b3e3-e9cf0ec789aa tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:49.249165 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-900e111c-89b5-40e9-b3e3-e9cf0ec789aa tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:49.262694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-900e111c-89b5-40e9-b3e3-e9cf0ec789aa tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.096858 Aug 28 01:02:49.263367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 474/930] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:49 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:02:49.420072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f027d8cc-d86d-4991-b986-5ed21ca707cf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:49.422486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f027d8cc-d86d-4991-b986-5ed21ca707cf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:49.430004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f027d8cc-d86d-4991-b986-5ed21ca707cf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:49.430532 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f027d8cc-d86d-4991-b986-5ed21ca707cf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:49.430962 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f027d8cc-d86d-4991-b986-5ed21ca707cf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:49.431694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f027d8cc-d86d-4991-b986-5ed21ca707cf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:49.432146 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f027d8cc-d86d-4991-b986-5ed21ca707cf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:49.432569 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f027d8cc-d86d-4991-b986-5ed21ca707cf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:49.469740 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f027d8cc-d86d-4991-b986-5ed21ca707cf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054166 Aug 28 01:02:49.470407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 475/931] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:49 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:49.475506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-344bb2fb-867c-4c10-9055-adc36332ea4b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:49.477791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-344bb2fb-867c-4c10-9055-adc36332ea4b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:49.484695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-344bb2fb-867c-4c10-9055-adc36332ea4b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:49.485224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-344bb2fb-867c-4c10-9055-adc36332ea4b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:49.485663 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-344bb2fb-867c-4c10-9055-adc36332ea4b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:49.486455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-344bb2fb-867c-4c10-9055-adc36332ea4b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:49.486908 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-344bb2fb-867c-4c10-9055-adc36332ea4b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:49.487331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-344bb2fb-867c-4c10-9055-adc36332ea4b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:49.532389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-344bb2fb-867c-4c10-9055-adc36332ea4b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060823 Aug 28 01:02:49.533262 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 476/932] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:49 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:50.285336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9fc1b384-7b00-4516-a50e-d3f4354af701 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:50.286034 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9fc1b384-7b00-4516-a50e-d3f4354af701 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:50.294506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9fc1b384-7b00-4516-a50e-d3f4354af701 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:50.295347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9fc1b384-7b00-4516-a50e-d3f4354af701 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:50.296032 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9fc1b384-7b00-4516-a50e-d3f4354af701 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:50.296926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9fc1b384-7b00-4516-a50e-d3f4354af701 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:50.297505 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9fc1b384-7b00-4516-a50e-d3f4354af701 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:50.298035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9fc1b384-7b00-4516-a50e-d3f4354af701 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:50.368441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9fc1b384-7b00-4516-a50e-d3f4354af701 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:50.378264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9fc1b384-7b00-4516-a50e-d3f4354af701 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.097009 Aug 28 01:02:50.378907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 477/933] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:50 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:02:50.483693 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4ce14186-d01b-4b49-8e53-0d262d3660af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:50.486105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4ce14186-d01b-4b49-8e53-0d262d3660af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:50.493129 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ce14186-d01b-4b49-8e53-0d262d3660af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:50.493687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ce14186-d01b-4b49-8e53-0d262d3660af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:50.494163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ce14186-d01b-4b49-8e53-0d262d3660af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:50.494929 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ce14186-d01b-4b49-8e53-0d262d3660af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:50.495467 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ce14186-d01b-4b49-8e53-0d262d3660af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:50.495932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ce14186-d01b-4b49-8e53-0d262d3660af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:50.531185 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4ce14186-d01b-4b49-8e53-0d262d3660af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051880 Aug 28 01:02:50.531964 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 478/934] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:50 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:50.546755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5df215ad-4ac5-4f34-803b-56c040953c70 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:50.549406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5df215ad-4ac5-4f34-803b-56c040953c70 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:50.556301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5df215ad-4ac5-4f34-803b-56c040953c70 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:50.556789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5df215ad-4ac5-4f34-803b-56c040953c70 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:50.557269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5df215ad-4ac5-4f34-803b-56c040953c70 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:50.558006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5df215ad-4ac5-4f34-803b-56c040953c70 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:50.558477 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5df215ad-4ac5-4f34-803b-56c040953c70 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:50.558924 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5df215ad-4ac5-4f34-803b-56c040953c70 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:50.594296 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5df215ad-4ac5-4f34-803b-56c040953c70 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051718 Aug 28 01:02:50.594901 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 479/935] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:50 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:02:51.399894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-acc467cd-a524-4161-8842-f6f1c8a0d036 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:51.400481 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-acc467cd-a524-4161-8842-f6f1c8a0d036 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:51.408610 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-acc467cd-a524-4161-8842-f6f1c8a0d036 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:51.409322 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-acc467cd-a524-4161-8842-f6f1c8a0d036 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:51.409825 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-acc467cd-a524-4161-8842-f6f1c8a0d036 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:51.410550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-acc467cd-a524-4161-8842-f6f1c8a0d036 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:51.411034 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-acc467cd-a524-4161-8842-f6f1c8a0d036 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:51.411452 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-acc467cd-a524-4161-8842-f6f1c8a0d036 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:51.492058 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-acc467cd-a524-4161-8842-f6f1c8a0d036 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:51.500881 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-acc467cd-a524-4161-8842-f6f1c8a0d036 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.104924 Aug 28 01:02:51.501577 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 480/936] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:51 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:02:51.543897 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-85d0512c-64d1-46a7-bee5-772a8d99cd5e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:51.546664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-85d0512c-64d1-46a7-bee5-772a8d99cd5e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:51.553886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85d0512c-64d1-46a7-bee5-772a8d99cd5e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:51.554450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85d0512c-64d1-46a7-bee5-772a8d99cd5e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:51.554972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85d0512c-64d1-46a7-bee5-772a8d99cd5e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:51.555732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85d0512c-64d1-46a7-bee5-772a8d99cd5e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:51.556327 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85d0512c-64d1-46a7-bee5-772a8d99cd5e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:51.556892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85d0512c-64d1-46a7-bee5-772a8d99cd5e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:51.592581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-85d0512c-64d1-46a7-bee5-772a8d99cd5e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052735 Aug 28 01:02:51.593397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 481/937] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:51 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:51.607339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-29bd6df1-89c2-45ff-9b7f-74bc858cc7e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:51.609662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-29bd6df1-89c2-45ff-9b7f-74bc858cc7e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:51.616539 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29bd6df1-89c2-45ff-9b7f-74bc858cc7e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:51.617071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29bd6df1-89c2-45ff-9b7f-74bc858cc7e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:51.617529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29bd6df1-89c2-45ff-9b7f-74bc858cc7e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:51.618254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29bd6df1-89c2-45ff-9b7f-74bc858cc7e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:51.618699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29bd6df1-89c2-45ff-9b7f-74bc858cc7e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:51.619152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29bd6df1-89c2-45ff-9b7f-74bc858cc7e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:51.654378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-29bd6df1-89c2-45ff-9b7f-74bc858cc7e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050924 Aug 28 01:02:51.654874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 482/938] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:51 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:02:52.097379 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=21 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:02:52.125552 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=52 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:02:52.522774 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-45635d88-82d5-426b-996e-12c98a350217 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:52.523763 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-45635d88-82d5-426b-996e-12c98a350217 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:52.531745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45635d88-82d5-426b-996e-12c98a350217 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:52.532245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45635d88-82d5-426b-996e-12c98a350217 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:52.532745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45635d88-82d5-426b-996e-12c98a350217 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:52.533724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45635d88-82d5-426b-996e-12c98a350217 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:52.534195 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45635d88-82d5-426b-996e-12c98a350217 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:52.534624 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45635d88-82d5-426b-996e-12c98a350217 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:52.603143 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-45635d88-82d5-426b-996e-12c98a350217 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:52.612633 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-45635d88-82d5-426b-996e-12c98a350217 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.093774 Aug 28 01:02:52.613278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 483/939] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:52 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:52.618305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-91f2e8ab-4f9b-456f-b377-7c697c613519 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:52.620548 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-91f2e8ab-4f9b-456f-b377-7c697c613519 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:52.628283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91f2e8ab-4f9b-456f-b377-7c697c613519 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:52.628854 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91f2e8ab-4f9b-456f-b377-7c697c613519 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:52.629364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91f2e8ab-4f9b-456f-b377-7c697c613519 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:52.630124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91f2e8ab-4f9b-456f-b377-7c697c613519 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:52.630644 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91f2e8ab-4f9b-456f-b377-7c697c613519 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:52.631103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91f2e8ab-4f9b-456f-b377-7c697c613519 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:52.635962 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:02:52.669899 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-91f2e8ab-4f9b-456f-b377-7c697c613519 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055453 Aug 28 01:02:52.670562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 484/940] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:52 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:52.679030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-64d9beeb-431e-44ce-9deb-05b174e70314 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:52.681377 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-64d9beeb-431e-44ce-9deb-05b174e70314 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:52.688196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64d9beeb-431e-44ce-9deb-05b174e70314 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:52.688945 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64d9beeb-431e-44ce-9deb-05b174e70314 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:52.689470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64d9beeb-431e-44ce-9deb-05b174e70314 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:52.690279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64d9beeb-431e-44ce-9deb-05b174e70314 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:52.690763 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64d9beeb-431e-44ce-9deb-05b174e70314 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:52.691222 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64d9beeb-431e-44ce-9deb-05b174e70314 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:52.726628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-64d9beeb-431e-44ce-9deb-05b174e70314 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054601 Aug 28 01:02:52.727282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 485/941] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:52 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:53.636386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-57ad581e-4533-4321-85d5-7d0b97652abd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:53.637239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-57ad581e-4533-4321-85d5-7d0b97652abd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:53.644401 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-57ad581e-4533-4321-85d5-7d0b97652abd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:53.644918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-57ad581e-4533-4321-85d5-7d0b97652abd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:53.645464 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-57ad581e-4533-4321-85d5-7d0b97652abd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:53.646184 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-57ad581e-4533-4321-85d5-7d0b97652abd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:53.646709 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-57ad581e-4533-4321-85d5-7d0b97652abd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:53.647131 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-57ad581e-4533-4321-85d5-7d0b97652abd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:53.717735 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-57ad581e-4533-4321-85d5-7d0b97652abd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:53.726409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-57ad581e-4533-4321-85d5-7d0b97652abd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.094016 Aug 28 01:02:53.727160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 486/942] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:53 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:53.732621 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c5058ed1-5ac3-48ab-834f-d000a1871021 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:53.735507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c5058ed1-5ac3-48ab-834f-d000a1871021 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:53.742995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5058ed1-5ac3-48ab-834f-d000a1871021 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:53.743637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5058ed1-5ac3-48ab-834f-d000a1871021 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:53.744232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5058ed1-5ac3-48ab-834f-d000a1871021 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:53.745215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5058ed1-5ac3-48ab-834f-d000a1871021 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:53.745758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5058ed1-5ac3-48ab-834f-d000a1871021 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:53.746271 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5058ed1-5ac3-48ab-834f-d000a1871021 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:53.779804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c5058ed1-5ac3-48ab-834f-d000a1871021 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051394 Aug 28 01:02:53.780499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 487/943] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:53 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:53.788950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-33e9fc50-71c7-478e-8d33-7c390cf9d620 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:53.791328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-33e9fc50-71c7-478e-8d33-7c390cf9d620 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:53.798245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-33e9fc50-71c7-478e-8d33-7c390cf9d620 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:53.798925 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-33e9fc50-71c7-478e-8d33-7c390cf9d620 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:53.799396 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-33e9fc50-71c7-478e-8d33-7c390cf9d620 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:53.800171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-33e9fc50-71c7-478e-8d33-7c390cf9d620 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:53.800680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-33e9fc50-71c7-478e-8d33-7c390cf9d620 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:53.801196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-33e9fc50-71c7-478e-8d33-7c390cf9d620 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:53.834875 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-33e9fc50-71c7-478e-8d33-7c390cf9d620 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052891 Aug 28 01:02:53.835630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 488/944] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:53 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:54.752325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-09f4ce65-ff68-4263-9f7b-b238d0f1406d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:54.753389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-09f4ce65-ff68-4263-9f7b-b238d0f1406d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:54.760998 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09f4ce65-ff68-4263-9f7b-b238d0f1406d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:54.761661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09f4ce65-ff68-4263-9f7b-b238d0f1406d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:54.762155 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09f4ce65-ff68-4263-9f7b-b238d0f1406d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:54.762935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09f4ce65-ff68-4263-9f7b-b238d0f1406d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:54.763441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09f4ce65-ff68-4263-9f7b-b238d0f1406d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:54.763915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09f4ce65-ff68-4263-9f7b-b238d0f1406d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:54.840799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-09f4ce65-ff68-4263-9f7b-b238d0f1406d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:54.851073 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-09f4ce65-ff68-4263-9f7b-b238d0f1406d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.102871 Aug 28 01:02:54.851895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 489/945] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:54 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:02:54.861250 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6b19ff4b-d38b-4efd-92d7-c03d9cde1e0a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:54.864815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6b19ff4b-d38b-4efd-92d7-c03d9cde1e0a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:54.875707 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b19ff4b-d38b-4efd-92d7-c03d9cde1e0a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:54.876628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b19ff4b-d38b-4efd-92d7-c03d9cde1e0a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:54.877464 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b19ff4b-d38b-4efd-92d7-c03d9cde1e0a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:54.878531 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b19ff4b-d38b-4efd-92d7-c03d9cde1e0a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:54.879366 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b19ff4b-d38b-4efd-92d7-c03d9cde1e0a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:54.880140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b19ff4b-d38b-4efd-92d7-c03d9cde1e0a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:54.930279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6b19ff4b-d38b-4efd-92d7-c03d9cde1e0a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.076939 Aug 28 01:02:54.931100 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 490/946] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:54 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:54.936948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-caaa08f2-5b7a-45f6-a953-2f557bc902f3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:54.939935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-caaa08f2-5b7a-45f6-a953-2f557bc902f3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:54.947678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-caaa08f2-5b7a-45f6-a953-2f557bc902f3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:54.948246 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-caaa08f2-5b7a-45f6-a953-2f557bc902f3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:54.948717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-caaa08f2-5b7a-45f6-a953-2f557bc902f3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:54.949504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-caaa08f2-5b7a-45f6-a953-2f557bc902f3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:54.950035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-caaa08f2-5b7a-45f6-a953-2f557bc902f3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:54.950483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-caaa08f2-5b7a-45f6-a953-2f557bc902f3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:54.984045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-caaa08f2-5b7a-45f6-a953-2f557bc902f3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051614 Aug 28 01:02:54.984639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 491/947] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:54 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:55.872389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c54b97fc-06da-484f-b8f8-d226eeaa7a9a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:55.873286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c54b97fc-06da-484f-b8f8-d226eeaa7a9a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:55.880204 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c54b97fc-06da-484f-b8f8-d226eeaa7a9a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:55.880699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c54b97fc-06da-484f-b8f8-d226eeaa7a9a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:55.881180 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c54b97fc-06da-484f-b8f8-d226eeaa7a9a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:55.881914 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c54b97fc-06da-484f-b8f8-d226eeaa7a9a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:55.882372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c54b97fc-06da-484f-b8f8-d226eeaa7a9a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:55.882795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c54b97fc-06da-484f-b8f8-d226eeaa7a9a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:55.954973 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c54b97fc-06da-484f-b8f8-d226eeaa7a9a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:55.969802 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c54b97fc-06da-484f-b8f8-d226eeaa7a9a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.100559 Aug 28 01:02:55.971440 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 492/948] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:55 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:55.979786 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-97901cd7-8bc2-48cb-81f6-7e46a67891ea tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:55.983383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-97901cd7-8bc2-48cb-81f6-7e46a67891ea tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:55.995564 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97901cd7-8bc2-48cb-81f6-7e46a67891ea tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:55.996401 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97901cd7-8bc2-48cb-81f6-7e46a67891ea tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:55.997481 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97901cd7-8bc2-48cb-81f6-7e46a67891ea tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:55.998739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97901cd7-8bc2-48cb-81f6-7e46a67891ea tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:55.999595 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97901cd7-8bc2-48cb-81f6-7e46a67891ea tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:56.000323 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97901cd7-8bc2-48cb-81f6-7e46a67891ea tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:56.041748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-97901cd7-8bc2-48cb-81f6-7e46a67891ea tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.068466 Aug 28 01:02:56.042405 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 493/949] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:55 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:56.048297 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a909859e-e375-484b-93f1-577aba057271 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:56.050560 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a909859e-e375-484b-93f1-577aba057271 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:56.057174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a909859e-e375-484b-93f1-577aba057271 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:56.057670 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a909859e-e375-484b-93f1-577aba057271 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:56.058119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a909859e-e375-484b-93f1-577aba057271 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:56.058817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a909859e-e375-484b-93f1-577aba057271 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:56.059276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a909859e-e375-484b-93f1-577aba057271 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:56.059692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a909859e-e375-484b-93f1-577aba057271 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:56.093930 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a909859e-e375-484b-93f1-577aba057271 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050356 Aug 28 01:02:56.094507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 494/950] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:56 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:56.995061 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ea971a70-418a-4540-b075-274149822782 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:56.995764 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ea971a70-418a-4540-b075-274149822782 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:57.002890 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea971a70-418a-4540-b075-274149822782 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:57.003474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea971a70-418a-4540-b075-274149822782 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:57.003959 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea971a70-418a-4540-b075-274149822782 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:57.004784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea971a70-418a-4540-b075-274149822782 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:57.005311 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea971a70-418a-4540-b075-274149822782 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:57.005771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea971a70-418a-4540-b075-274149822782 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:57.078345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ea971a70-418a-4540-b075-274149822782 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:57.092726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ea971a70-418a-4540-b075-274149822782 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.101378 Aug 28 01:02:57.093803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 495/951] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:56 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:57.103092 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7af01069-5555-4c55-aa54-39a07e968997 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:57.106887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7af01069-5555-4c55-aa54-39a07e968997 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:57.110099 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:02:57.119345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7af01069-5555-4c55-aa54-39a07e968997 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:57.120140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7af01069-5555-4c55-aa54-39a07e968997 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:57.120853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7af01069-5555-4c55-aa54-39a07e968997 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:57.122086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7af01069-5555-4c55-aa54-39a07e968997 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:57.123356 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7af01069-5555-4c55-aa54-39a07e968997 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:57.123578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7af01069-5555-4c55-aa54-39a07e968997 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:57.161526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7af01069-5555-4c55-aa54-39a07e968997 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.065797 Aug 28 01:02:57.162246 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 496/952] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:57 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:57.168887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a189620a-f3fa-474b-ae76-e9cb4bcc8d9f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:57.171387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a189620a-f3fa-474b-ae76-e9cb4bcc8d9f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:57.179052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a189620a-f3fa-474b-ae76-e9cb4bcc8d9f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:57.179693 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a189620a-f3fa-474b-ae76-e9cb4bcc8d9f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:57.180154 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a189620a-f3fa-474b-ae76-e9cb4bcc8d9f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:57.180937 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a189620a-f3fa-474b-ae76-e9cb4bcc8d9f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:57.181428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a189620a-f3fa-474b-ae76-e9cb4bcc8d9f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:57.181847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a189620a-f3fa-474b-ae76-e9cb4bcc8d9f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:57.220487 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a189620a-f3fa-474b-ae76-e9cb4bcc8d9f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057009 Aug 28 01:02:57.221106 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 497/953] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:57 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:58.119710 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6e001d2e-43b9-4541-98f2-932f52fd4dbc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:58.120943 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6e001d2e-43b9-4541-98f2-932f52fd4dbc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:58.129837 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e001d2e-43b9-4541-98f2-932f52fd4dbc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:58.130333 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e001d2e-43b9-4541-98f2-932f52fd4dbc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:58.130777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e001d2e-43b9-4541-98f2-932f52fd4dbc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:58.131516 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e001d2e-43b9-4541-98f2-932f52fd4dbc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:58.131977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e001d2e-43b9-4541-98f2-932f52fd4dbc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:58.132404 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e001d2e-43b9-4541-98f2-932f52fd4dbc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:58.213331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6e001d2e-43b9-4541-98f2-932f52fd4dbc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:58.222259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6e001d2e-43b9-4541-98f2-932f52fd4dbc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.109322 Aug 28 01:02:58.222869 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 498/954] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:58 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:02:58.228087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6629dca4-094f-47a9-bd6a-6c86d597eef1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:58.230648 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6629dca4-094f-47a9-bd6a-6c86d597eef1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:58.237740 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6629dca4-094f-47a9-bd6a-6c86d597eef1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:58.238227 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6629dca4-094f-47a9-bd6a-6c86d597eef1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:58.238641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6629dca4-094f-47a9-bd6a-6c86d597eef1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:58.239363 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6629dca4-094f-47a9-bd6a-6c86d597eef1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:58.239828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6629dca4-094f-47a9-bd6a-6c86d597eef1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:58.240246 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6629dca4-094f-47a9-bd6a-6c86d597eef1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:58.285645 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6629dca4-094f-47a9-bd6a-6c86d597eef1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.061093 Aug 28 01:02:58.286652 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 499/955] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:58 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:58.295762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-51160e22-ff41-40df-8a53-8892f355f00a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:58.299614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-51160e22-ff41-40df-8a53-8892f355f00a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:58.310562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-51160e22-ff41-40df-8a53-8892f355f00a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:58.311504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-51160e22-ff41-40df-8a53-8892f355f00a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:58.312279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-51160e22-ff41-40df-8a53-8892f355f00a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:58.313611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-51160e22-ff41-40df-8a53-8892f355f00a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:58.314403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-51160e22-ff41-40df-8a53-8892f355f00a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:58.315156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-51160e22-ff41-40df-8a53-8892f355f00a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:58.363207 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:02:58.373285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-51160e22-ff41-40df-8a53-8892f355f00a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.084627 Aug 28 01:02:58.374132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 500/956] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:58 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:02:59.256589 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c6b42008-1f90-401d-9a0c-88cfd35ce1aa tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:59.257399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c6b42008-1f90-401d-9a0c-88cfd35ce1aa tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:59.265011 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6b42008-1f90-401d-9a0c-88cfd35ce1aa tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:59.265491 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6b42008-1f90-401d-9a0c-88cfd35ce1aa tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:59.265926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6b42008-1f90-401d-9a0c-88cfd35ce1aa tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:59.266651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6b42008-1f90-401d-9a0c-88cfd35ce1aa tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:59.267186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6b42008-1f90-401d-9a0c-88cfd35ce1aa tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:59.267611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6b42008-1f90-401d-9a0c-88cfd35ce1aa tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:59.339572 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c6b42008-1f90-401d-9a0c-88cfd35ce1aa tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:02:59.348538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c6b42008-1f90-401d-9a0c-88cfd35ce1aa tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.095947 Aug 28 01:02:59.349272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 501/957] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:02:59 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:02:59.354543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-76c6eb70-88d7-4aed-a861-195312aa70a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:59.357111 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-76c6eb70-88d7-4aed-a861-195312aa70a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:59.363831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76c6eb70-88d7-4aed-a861-195312aa70a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:59.364315 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76c6eb70-88d7-4aed-a861-195312aa70a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:59.364745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76c6eb70-88d7-4aed-a861-195312aa70a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:59.365517 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76c6eb70-88d7-4aed-a861-195312aa70a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:59.365964 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76c6eb70-88d7-4aed-a861-195312aa70a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:59.366390 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76c6eb70-88d7-4aed-a861-195312aa70a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:59.402405 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-76c6eb70-88d7-4aed-a861-195312aa70a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051970 Aug 28 01:02:59.402996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 502/958] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:02:59 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:02:59.408160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-32a4582b-ce36-48d1-8782-bde18367c4aa tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:02:59.410497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-32a4582b-ce36-48d1-8782-bde18367c4aa tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:02:59.417989 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32a4582b-ce36-48d1-8782-bde18367c4aa tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:59.418825 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32a4582b-ce36-48d1-8782-bde18367c4aa tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:59.419338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32a4582b-ce36-48d1-8782-bde18367c4aa tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:59.420401 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32a4582b-ce36-48d1-8782-bde18367c4aa tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:02:59.421001 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32a4582b-ce36-48d1-8782-bde18367c4aa tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:02:59.421611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32a4582b-ce36-48d1-8782-bde18367c4aa tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:02:59.458223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-32a4582b-ce36-48d1-8782-bde18367c4aa tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054060 Aug 28 01:02:59.458808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 503/959] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:02:59 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:00.382302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ed4a8662-e96c-41ae-984a-4a4e7067b3f6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:00.383215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ed4a8662-e96c-41ae-984a-4a4e7067b3f6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:00.390812 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed4a8662-e96c-41ae-984a-4a4e7067b3f6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:00.391308 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed4a8662-e96c-41ae-984a-4a4e7067b3f6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:00.391742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed4a8662-e96c-41ae-984a-4a4e7067b3f6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:00.392475 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed4a8662-e96c-41ae-984a-4a4e7067b3f6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:00.392933 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed4a8662-e96c-41ae-984a-4a4e7067b3f6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:00.393388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed4a8662-e96c-41ae-984a-4a4e7067b3f6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:00.462913 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ed4a8662-e96c-41ae-984a-4a4e7067b3f6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:00.472316 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ed4a8662-e96c-41ae-984a-4a4e7067b3f6 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.094012 Aug 28 01:03:00.473208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 504/960] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:00 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:00.482400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bde1088c-d8e9-47b4-8f12-b0c2c3a174fd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:00.486299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bde1088c-d8e9-47b4-8f12-b0c2c3a174fd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:00.497851 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bde1088c-d8e9-47b4-8f12-b0c2c3a174fd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:00.498444 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bde1088c-d8e9-47b4-8f12-b0c2c3a174fd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:00.498932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bde1088c-d8e9-47b4-8f12-b0c2c3a174fd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:00.499713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bde1088c-d8e9-47b4-8f12-b0c2c3a174fd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:00.500223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bde1088c-d8e9-47b4-8f12-b0c2c3a174fd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:00.500700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bde1088c-d8e9-47b4-8f12-b0c2c3a174fd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:00.534596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bde1088c-d8e9-47b4-8f12-b0c2c3a174fd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.059889 Aug 28 01:03:00.535188 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 505/961] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:00 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:00.540251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-98c12603-ce71-4a48-a29e-a231e80d8d8a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:00.542734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-98c12603-ce71-4a48-a29e-a231e80d8d8a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:00.549884 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98c12603-ce71-4a48-a29e-a231e80d8d8a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:00.550435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98c12603-ce71-4a48-a29e-a231e80d8d8a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:00.550889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98c12603-ce71-4a48-a29e-a231e80d8d8a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:00.551603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98c12603-ce71-4a48-a29e-a231e80d8d8a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:00.552050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98c12603-ce71-4a48-a29e-a231e80d8d8a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:00.552476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98c12603-ce71-4a48-a29e-a231e80d8d8a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:00.585713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-98c12603-ce71-4a48-a29e-a231e80d8d8a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049410 Aug 28 01:03:00.586310 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 506/962] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:00 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:01.505632 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9148216f-b1f2-48f2-9716-506e477b6732 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:01.506766 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9148216f-b1f2-48f2-9716-506e477b6732 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:01.513853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9148216f-b1f2-48f2-9716-506e477b6732 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:01.514354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9148216f-b1f2-48f2-9716-506e477b6732 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:01.514802 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9148216f-b1f2-48f2-9716-506e477b6732 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:01.515531 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9148216f-b1f2-48f2-9716-506e477b6732 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:01.515991 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9148216f-b1f2-48f2-9716-506e477b6732 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:01.516422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9148216f-b1f2-48f2-9716-506e477b6732 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:01.592899 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9148216f-b1f2-48f2-9716-506e477b6732 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:01.601742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9148216f-b1f2-48f2-9716-506e477b6732 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.099972 Aug 28 01:03:01.602363 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 507/963] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:01 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:01.607468 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0e6823d6-d627-4de1-b80b-7582436ce71c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:01.609786 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0e6823d6-d627-4de1-b80b-7582436ce71c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:01.616625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e6823d6-d627-4de1-b80b-7582436ce71c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:01.617163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e6823d6-d627-4de1-b80b-7582436ce71c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:01.617600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e6823d6-d627-4de1-b80b-7582436ce71c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:01.618324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e6823d6-d627-4de1-b80b-7582436ce71c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:01.618776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e6823d6-d627-4de1-b80b-7582436ce71c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:01.619201 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e6823d6-d627-4de1-b80b-7582436ce71c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:01.652856 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0e6823d6-d627-4de1-b80b-7582436ce71c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049367 Aug 28 01:03:01.653507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 508/964] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:01 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:01.658498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f9d8c165-1fed-4cff-82a1-72a189360083 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:01.660722 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f9d8c165-1fed-4cff-82a1-72a189360083 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:01.667642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f9d8c165-1fed-4cff-82a1-72a189360083 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:01.668143 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f9d8c165-1fed-4cff-82a1-72a189360083 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:01.668583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f9d8c165-1fed-4cff-82a1-72a189360083 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:01.669345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f9d8c165-1fed-4cff-82a1-72a189360083 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:01.669801 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f9d8c165-1fed-4cff-82a1-72a189360083 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:01.670230 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f9d8c165-1fed-4cff-82a1-72a189360083 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:01.703393 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f9d8c165-1fed-4cff-82a1-72a189360083 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048782 Aug 28 01:03:01.703983 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 509/965] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:01 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:02.625004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bb831209-0694-4ec5-8990-00e77b32ff81 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:02.625877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bb831209-0694-4ec5-8990-00e77b32ff81 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:02.633375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bb831209-0694-4ec5-8990-00e77b32ff81 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:02.633868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bb831209-0694-4ec5-8990-00e77b32ff81 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:02.634316 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bb831209-0694-4ec5-8990-00e77b32ff81 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:02.635069 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bb831209-0694-4ec5-8990-00e77b32ff81 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:02.635511 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bb831209-0694-4ec5-8990-00e77b32ff81 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:02.635961 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bb831209-0694-4ec5-8990-00e77b32ff81 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:02.723039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bb831209-0694-4ec5-8990-00e77b32ff81 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:02.733540 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bb831209-0694-4ec5-8990-00e77b32ff81 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.112440 Aug 28 01:03:02.734230 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 510/966] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:02 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:03:02.739835 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0d6ce91b-16d6-4b6a-990d-640e85278224 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:02.742257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0d6ce91b-16d6-4b6a-990d-640e85278224 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:02.750251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0d6ce91b-16d6-4b6a-990d-640e85278224 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:02.750876 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0d6ce91b-16d6-4b6a-990d-640e85278224 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:02.751586 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0d6ce91b-16d6-4b6a-990d-640e85278224 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:02.754382 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0d6ce91b-16d6-4b6a-990d-640e85278224 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:02.754918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0d6ce91b-16d6-4b6a-990d-640e85278224 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:02.755435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0d6ce91b-16d6-4b6a-990d-640e85278224 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:02.801184 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0d6ce91b-16d6-4b6a-990d-640e85278224 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.065692 Aug 28 01:03:02.801783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 511/967] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:02 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:02.807002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-72f81037-5e8c-4608-bb46-e7ea3536187f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:02.811337 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-72f81037-5e8c-4608-bb46-e7ea3536187f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:02.818530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-72f81037-5e8c-4608-bb46-e7ea3536187f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:02.819108 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-72f81037-5e8c-4608-bb46-e7ea3536187f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:02.819585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-72f81037-5e8c-4608-bb46-e7ea3536187f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:02.820344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-72f81037-5e8c-4608-bb46-e7ea3536187f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:02.820801 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-72f81037-5e8c-4608-bb46-e7ea3536187f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:02.821251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-72f81037-5e8c-4608-bb46-e7ea3536187f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:02.855889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-72f81037-5e8c-4608-bb46-e7ea3536187f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052905 Aug 28 01:03:02.856467 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 512/968] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:02 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:03.757859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-db52342b-fe59-40ab-9ab7-5830f15a72de tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:03.758652 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-db52342b-fe59-40ab-9ab7-5830f15a72de tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:03.766922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db52342b-fe59-40ab-9ab7-5830f15a72de tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:03.767545 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db52342b-fe59-40ab-9ab7-5830f15a72de tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:03.768059 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db52342b-fe59-40ab-9ab7-5830f15a72de tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:03.768888 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db52342b-fe59-40ab-9ab7-5830f15a72de tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:03.769438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db52342b-fe59-40ab-9ab7-5830f15a72de tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:03.769956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db52342b-fe59-40ab-9ab7-5830f15a72de tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:03.849442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-db52342b-fe59-40ab-9ab7-5830f15a72de tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:03.858776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-db52342b-fe59-40ab-9ab7-5830f15a72de tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.105409 Aug 28 01:03:03.859653 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 513/969] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:03 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:03.866029 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2f99a6d1-7fa1-48ab-8747-faa61b831119 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:03.868920 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2f99a6d1-7fa1-48ab-8747-faa61b831119 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:03.877137 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f99a6d1-7fa1-48ab-8747-faa61b831119 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:03.877734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f99a6d1-7fa1-48ab-8747-faa61b831119 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:03.878301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f99a6d1-7fa1-48ab-8747-faa61b831119 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:03.879006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f99a6d1-7fa1-48ab-8747-faa61b831119 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:03.879655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f99a6d1-7fa1-48ab-8747-faa61b831119 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:03.880132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f99a6d1-7fa1-48ab-8747-faa61b831119 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:03.916340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2f99a6d1-7fa1-48ab-8747-faa61b831119 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055333 Aug 28 01:03:03.917239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 514/970] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:03 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:03.925227 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fcaf4120-8b80-40c2-9c33-7228f5e0e86d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:03.927784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fcaf4120-8b80-40c2-9c33-7228f5e0e86d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:03.936543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fcaf4120-8b80-40c2-9c33-7228f5e0e86d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:03.937268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fcaf4120-8b80-40c2-9c33-7228f5e0e86d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:03.937939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fcaf4120-8b80-40c2-9c33-7228f5e0e86d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:03.938865 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fcaf4120-8b80-40c2-9c33-7228f5e0e86d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:03.939471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fcaf4120-8b80-40c2-9c33-7228f5e0e86d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:03.939946 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fcaf4120-8b80-40c2-9c33-7228f5e0e86d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:03.975448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fcaf4120-8b80-40c2-9c33-7228f5e0e86d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057244 Aug 28 01:03:03.976018 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 515/971] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:03 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:04.884229 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6c339d9c-764b-4156-904f-7d35f413faee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:04.885087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6c339d9c-764b-4156-904f-7d35f413faee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:04.892926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c339d9c-764b-4156-904f-7d35f413faee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:04.893538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c339d9c-764b-4156-904f-7d35f413faee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:04.894019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c339d9c-764b-4156-904f-7d35f413faee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:04.894822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c339d9c-764b-4156-904f-7d35f413faee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:04.895418 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c339d9c-764b-4156-904f-7d35f413faee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:04.895935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c339d9c-764b-4156-904f-7d35f413faee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:04.964734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6c339d9c-764b-4156-904f-7d35f413faee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:04.973424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6c339d9c-764b-4156-904f-7d35f413faee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.093183 Aug 28 01:03:04.974022 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 516/972] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:04 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:04.978843 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-de3514b2-0455-4d40-86af-fe4c5f7d1807 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:04.981209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-de3514b2-0455-4d40-86af-fe4c5f7d1807 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:04.988049 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de3514b2-0455-4d40-86af-fe4c5f7d1807 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:04.988698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de3514b2-0455-4d40-86af-fe4c5f7d1807 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:04.989256 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de3514b2-0455-4d40-86af-fe4c5f7d1807 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:04.989968 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de3514b2-0455-4d40-86af-fe4c5f7d1807 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:04.990412 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de3514b2-0455-4d40-86af-fe4c5f7d1807 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:04.990827 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de3514b2-0455-4d40-86af-fe4c5f7d1807 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:05.009170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:03:05.024440 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-de3514b2-0455-4d40-86af-fe4c5f7d1807 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049306 Aug 28 01:03:05.025066 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 517/973] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:04 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:05.030088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0effa8f0-8989-4112-bec7-c7fe1f1f475e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:05.032492 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0effa8f0-8989-4112-bec7-c7fe1f1f475e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:05.038977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0effa8f0-8989-4112-bec7-c7fe1f1f475e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:05.039462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0effa8f0-8989-4112-bec7-c7fe1f1f475e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:05.039896 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0effa8f0-8989-4112-bec7-c7fe1f1f475e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:05.040610 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0effa8f0-8989-4112-bec7-c7fe1f1f475e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:05.041089 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0effa8f0-8989-4112-bec7-c7fe1f1f475e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:05.041509 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0effa8f0-8989-4112-bec7-c7fe1f1f475e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:05.075397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0effa8f0-8989-4112-bec7-c7fe1f1f475e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049212 Aug 28 01:03:05.075993 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 518/974] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:05 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:05.999335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-73d26f31-3622-4362-95ef-b33d03c2c494 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:06.000215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-73d26f31-3622-4362-95ef-b33d03c2c494 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:06.008440 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-73d26f31-3622-4362-95ef-b33d03c2c494 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:06.008846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-73d26f31-3622-4362-95ef-b33d03c2c494 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:06.009369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-73d26f31-3622-4362-95ef-b33d03c2c494 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:06.010286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-73d26f31-3622-4362-95ef-b33d03c2c494 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:06.010752 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-73d26f31-3622-4362-95ef-b33d03c2c494 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:06.011283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-73d26f31-3622-4362-95ef-b33d03c2c494 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:06.082967 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-73d26f31-3622-4362-95ef-b33d03c2c494 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:06.099022 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-73d26f31-3622-4362-95ef-b33d03c2c494 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.103185 Aug 28 01:03:06.100071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 519/975] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:05 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:06.108613 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cf405172-2e17-4603-a94e-a30e6a9676b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:06.112228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cf405172-2e17-4603-a94e-a30e6a9676b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:06.123054 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf405172-2e17-4603-a94e-a30e6a9676b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:06.123898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf405172-2e17-4603-a94e-a30e6a9676b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:06.124639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf405172-2e17-4603-a94e-a30e6a9676b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:06.125948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf405172-2e17-4603-a94e-a30e6a9676b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:06.126705 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf405172-2e17-4603-a94e-a30e6a9676b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:06.127440 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf405172-2e17-4603-a94e-a30e6a9676b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:06.168443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cf405172-2e17-4603-a94e-a30e6a9676b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.066761 Aug 28 01:03:06.169197 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 520/976] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:06 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:06.174626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-92d5833a-e558-4300-a5ee-162bba2ae48d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:06.176832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-92d5833a-e558-4300-a5ee-162bba2ae48d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:06.183349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92d5833a-e558-4300-a5ee-162bba2ae48d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:06.183831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92d5833a-e558-4300-a5ee-162bba2ae48d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:06.184257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92d5833a-e558-4300-a5ee-162bba2ae48d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:06.184948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92d5833a-e558-4300-a5ee-162bba2ae48d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:06.185598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92d5833a-e558-4300-a5ee-162bba2ae48d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:06.186055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92d5833a-e558-4300-a5ee-162bba2ae48d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:06.220989 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-92d5833a-e558-4300-a5ee-162bba2ae48d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050566 Aug 28 01:03:06.221617 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 521/977] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:06 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:07.136005 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-85815d79-c9bc-4b7e-a496-8e540c5731cb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:07.136890 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-85815d79-c9bc-4b7e-a496-8e540c5731cb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:07.144657 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85815d79-c9bc-4b7e-a496-8e540c5731cb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:07.145445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85815d79-c9bc-4b7e-a496-8e540c5731cb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:07.146001 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85815d79-c9bc-4b7e-a496-8e540c5731cb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:07.146822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85815d79-c9bc-4b7e-a496-8e540c5731cb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:07.147404 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85815d79-c9bc-4b7e-a496-8e540c5731cb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:07.147937 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85815d79-c9bc-4b7e-a496-8e540c5731cb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:07.225903 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-85815d79-c9bc-4b7e-a496-8e540c5731cb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:07.235290 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-85815d79-c9bc-4b7e-a496-8e540c5731cb tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.108869 Aug 28 01:03:07.235944 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 522/978] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:07 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:07.241458 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b5c5bf54-4e34-4208-94e5-10a115c2c0cb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:07.244101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b5c5bf54-4e34-4208-94e5-10a115c2c0cb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:07.251603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5c5bf54-4e34-4208-94e5-10a115c2c0cb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:07.252112 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5c5bf54-4e34-4208-94e5-10a115c2c0cb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:07.252550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5c5bf54-4e34-4208-94e5-10a115c2c0cb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:07.253339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5c5bf54-4e34-4208-94e5-10a115c2c0cb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:07.253795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5c5bf54-4e34-4208-94e5-10a115c2c0cb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:07.254233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5c5bf54-4e34-4208-94e5-10a115c2c0cb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:07.290699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b5c5bf54-4e34-4208-94e5-10a115c2c0cb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053381 Aug 28 01:03:07.291905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 523/979] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:07 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:07.298515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c22b7d81-7fed-4531-9c3b-a094dfd66c6d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:07.300886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c22b7d81-7fed-4531-9c3b-a094dfd66c6d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:07.307934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c22b7d81-7fed-4531-9c3b-a094dfd66c6d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:07.308438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c22b7d81-7fed-4531-9c3b-a094dfd66c6d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:07.308868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c22b7d81-7fed-4531-9c3b-a094dfd66c6d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:07.309634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c22b7d81-7fed-4531-9c3b-a094dfd66c6d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:07.310070 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c22b7d81-7fed-4531-9c3b-a094dfd66c6d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:07.310514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c22b7d81-7fed-4531-9c3b-a094dfd66c6d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:07.348110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c22b7d81-7fed-4531-9c3b-a094dfd66c6d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054611 Aug 28 01:03:07.348785 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 524/980] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:07 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:08.272201 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c56b74b5-f011-4d10-8674-947ad57b458e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:08.273015 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c56b74b5-f011-4d10-8674-947ad57b458e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:08.281175 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c56b74b5-f011-4d10-8674-947ad57b458e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:08.281672 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c56b74b5-f011-4d10-8674-947ad57b458e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:08.282109 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c56b74b5-f011-4d10-8674-947ad57b458e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:08.282821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c56b74b5-f011-4d10-8674-947ad57b458e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:08.283267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c56b74b5-f011-4d10-8674-947ad57b458e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:08.283680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c56b74b5-f011-4d10-8674-947ad57b458e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:08.365265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c56b74b5-f011-4d10-8674-947ad57b458e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:08.374529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c56b74b5-f011-4d10-8674-947ad57b458e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.106204 Aug 28 01:03:08.375039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 525/981] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:08 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:08.380064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e0ce913e-ac3a-43f1-ad4a-3817f3b6ec66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:08.382379 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e0ce913e-ac3a-43f1-ad4a-3817f3b6ec66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:08.392591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0ce913e-ac3a-43f1-ad4a-3817f3b6ec66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:08.393543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0ce913e-ac3a-43f1-ad4a-3817f3b6ec66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:08.394333 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0ce913e-ac3a-43f1-ad4a-3817f3b6ec66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:08.395657 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0ce913e-ac3a-43f1-ad4a-3817f3b6ec66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:08.396493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0ce913e-ac3a-43f1-ad4a-3817f3b6ec66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:08.397285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0ce913e-ac3a-43f1-ad4a-3817f3b6ec66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:08.458701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e0ce913e-ac3a-43f1-ad4a-3817f3b6ec66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.082138 Aug 28 01:03:08.459783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 526/982] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:08 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:08.468919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9d06f85c-7031-4212-95ff-4a3d0aebe157 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:08.472909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9d06f85c-7031-4212-95ff-4a3d0aebe157 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:08.484551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d06f85c-7031-4212-95ff-4a3d0aebe157 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:08.485567 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d06f85c-7031-4212-95ff-4a3d0aebe157 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:08.486418 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d06f85c-7031-4212-95ff-4a3d0aebe157 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:08.487806 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d06f85c-7031-4212-95ff-4a3d0aebe157 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:08.488591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d06f85c-7031-4212-95ff-4a3d0aebe157 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:08.489355 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d06f85c-7031-4212-95ff-4a3d0aebe157 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:08.535229 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9d06f85c-7031-4212-95ff-4a3d0aebe157 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.073871 Aug 28 01:03:08.535887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 527/983] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:08 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:09.396950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-eb9773b5-ae97-48a7-b7b0-4e5c957d0fcf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:09.397753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-eb9773b5-ae97-48a7-b7b0-4e5c957d0fcf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:09.405392 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb9773b5-ae97-48a7-b7b0-4e5c957d0fcf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:09.405890 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb9773b5-ae97-48a7-b7b0-4e5c957d0fcf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:09.406339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb9773b5-ae97-48a7-b7b0-4e5c957d0fcf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:09.407325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb9773b5-ae97-48a7-b7b0-4e5c957d0fcf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:09.407845 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb9773b5-ae97-48a7-b7b0-4e5c957d0fcf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:09.408425 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb9773b5-ae97-48a7-b7b0-4e5c957d0fcf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:09.477165 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-eb9773b5-ae97-48a7-b7b0-4e5c957d0fcf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:09.489968 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-eb9773b5-ae97-48a7-b7b0-4e5c957d0fcf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.096755 Aug 28 01:03:09.490969 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 528/984] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:09 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:03:09.500161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b20a7328-6175-47c1-a001-31fd9c4c40b8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:09.504064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b20a7328-6175-47c1-a001-31fd9c4c40b8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:09.515834 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b20a7328-6175-47c1-a001-31fd9c4c40b8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:09.516800 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b20a7328-6175-47c1-a001-31fd9c4c40b8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:09.517583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b20a7328-6175-47c1-a001-31fd9c4c40b8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:09.519008 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b20a7328-6175-47c1-a001-31fd9c4c40b8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:09.519817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b20a7328-6175-47c1-a001-31fd9c4c40b8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:09.520545 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b20a7328-6175-47c1-a001-31fd9c4c40b8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:09.581498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b20a7328-6175-47c1-a001-31fd9c4c40b8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.088435 Aug 28 01:03:09.582381 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 529/985] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:09 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:09.591752 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ab93db76-48e7-493c-acbb-9763b3d9b0b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:09.595764 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ab93db76-48e7-493c-acbb-9763b3d9b0b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:09.606988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab93db76-48e7-493c-acbb-9763b3d9b0b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:09.607851 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab93db76-48e7-493c-acbb-9763b3d9b0b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:09.608694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab93db76-48e7-493c-acbb-9763b3d9b0b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:09.610058 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab93db76-48e7-493c-acbb-9763b3d9b0b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:09.611050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab93db76-48e7-493c-acbb-9763b3d9b0b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:09.611839 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab93db76-48e7-493c-acbb-9763b3d9b0b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:09.654214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ab93db76-48e7-493c-acbb-9763b3d9b0b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.069973 Aug 28 01:03:09.655539 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 530/986] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:09 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:10.516003 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8d8b1a2b-f971-4ef4-84ef-d11368811f86 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:10.517347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8d8b1a2b-f971-4ef4-84ef-d11368811f86 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:10.530824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d8b1a2b-f971-4ef4-84ef-d11368811f86 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:10.531719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d8b1a2b-f971-4ef4-84ef-d11368811f86 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:10.532548 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d8b1a2b-f971-4ef4-84ef-d11368811f86 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:10.533919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d8b1a2b-f971-4ef4-84ef-d11368811f86 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:10.534687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d8b1a2b-f971-4ef4-84ef-d11368811f86 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:10.535451 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d8b1a2b-f971-4ef4-84ef-d11368811f86 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:10.633218 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8d8b1a2b-f971-4ef4-84ef-d11368811f86 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:10.642538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8d8b1a2b-f971-4ef4-84ef-d11368811f86 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.133719 Aug 28 01:03:10.643331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 531/987] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:10 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:10.650159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a5e8b788-5f38-4e74-bb40-9a73a26dc681 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:10.652472 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a5e8b788-5f38-4e74-bb40-9a73a26dc681 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:10.659858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5e8b788-5f38-4e74-bb40-9a73a26dc681 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:10.660349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5e8b788-5f38-4e74-bb40-9a73a26dc681 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:10.660781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5e8b788-5f38-4e74-bb40-9a73a26dc681 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:10.661527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5e8b788-5f38-4e74-bb40-9a73a26dc681 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:10.661971 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5e8b788-5f38-4e74-bb40-9a73a26dc681 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:10.662386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5e8b788-5f38-4e74-bb40-9a73a26dc681 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:10.706868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a5e8b788-5f38-4e74-bb40-9a73a26dc681 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.061435 Aug 28 01:03:10.707925 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 532/988] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:10 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:10.717085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5eb9a5ab-f49a-4603-8b06-68ee107aec6e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:10.720682 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5eb9a5ab-f49a-4603-8b06-68ee107aec6e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:10.731577 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5eb9a5ab-f49a-4603-8b06-68ee107aec6e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:10.732408 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5eb9a5ab-f49a-4603-8b06-68ee107aec6e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:10.733197 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5eb9a5ab-f49a-4603-8b06-68ee107aec6e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:10.734583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5eb9a5ab-f49a-4603-8b06-68ee107aec6e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:10.735385 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5eb9a5ab-f49a-4603-8b06-68ee107aec6e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:10.736134 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5eb9a5ab-f49a-4603-8b06-68ee107aec6e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:10.777892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5eb9a5ab-f49a-4603-8b06-68ee107aec6e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067737 Aug 28 01:03:10.778430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 533/989] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:10 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:11.669181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2d1a667f-4094-422a-b293-ed7e92cb59ef tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:11.669971 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2d1a667f-4094-422a-b293-ed7e92cb59ef tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:11.677412 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d1a667f-4094-422a-b293-ed7e92cb59ef tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:11.677926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d1a667f-4094-422a-b293-ed7e92cb59ef tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:11.678363 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d1a667f-4094-422a-b293-ed7e92cb59ef tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:11.679311 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d1a667f-4094-422a-b293-ed7e92cb59ef tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:11.679929 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d1a667f-4094-422a-b293-ed7e92cb59ef tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:11.680515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d1a667f-4094-422a-b293-ed7e92cb59ef tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:11.685279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:03:11.759110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2d1a667f-4094-422a-b293-ed7e92cb59ef tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:11.768090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2d1a667f-4094-422a-b293-ed7e92cb59ef tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.103297 Aug 28 01:03:11.768692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 534/990] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:11 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:11.773699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6c6ab468-118a-4328-976f-e6b7a7e6c90d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:11.780207 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6c6ab468-118a-4328-976f-e6b7a7e6c90d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:11.791755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c6ab468-118a-4328-976f-e6b7a7e6c90d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:11.792262 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c6ab468-118a-4328-976f-e6b7a7e6c90d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:11.792705 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c6ab468-118a-4328-976f-e6b7a7e6c90d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:11.793491 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c6ab468-118a-4328-976f-e6b7a7e6c90d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:11.793958 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c6ab468-118a-4328-976f-e6b7a7e6c90d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:11.794399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c6ab468-118a-4328-976f-e6b7a7e6c90d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:11.833275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6c6ab468-118a-4328-976f-e6b7a7e6c90d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.063229 Aug 28 01:03:11.833888 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 535/991] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:11 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:11.838988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2c59e36a-bdd5-4c95-8920-abed701beeb4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:11.841598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2c59e36a-bdd5-4c95-8920-abed701beeb4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:11.849156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c59e36a-bdd5-4c95-8920-abed701beeb4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:11.849684 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c59e36a-bdd5-4c95-8920-abed701beeb4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:11.850122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c59e36a-bdd5-4c95-8920-abed701beeb4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:11.850860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c59e36a-bdd5-4c95-8920-abed701beeb4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:11.851311 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c59e36a-bdd5-4c95-8920-abed701beeb4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:11.851726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c59e36a-bdd5-4c95-8920-abed701beeb4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:11.890129 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2c59e36a-bdd5-4c95-8920-abed701beeb4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054807 Aug 28 01:03:11.891004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 536/992] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:11 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:12.802394 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-dbd89430-2e87-423e-bb0f-c78acc615766 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:12.803200 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-dbd89430-2e87-423e-bb0f-c78acc615766 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:12.811243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbd89430-2e87-423e-bb0f-c78acc615766 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:12.812115 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbd89430-2e87-423e-bb0f-c78acc615766 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:12.813150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbd89430-2e87-423e-bb0f-c78acc615766 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:12.814336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbd89430-2e87-423e-bb0f-c78acc615766 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:12.815181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbd89430-2e87-423e-bb0f-c78acc615766 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:12.815864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbd89430-2e87-423e-bb0f-c78acc615766 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:12.896064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dbd89430-2e87-423e-bb0f-c78acc615766 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:12.905002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-dbd89430-2e87-423e-bb0f-c78acc615766 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.107745 Aug 28 01:03:12.905607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 537/993] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:12 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:03:12.910608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f40fefb9-56cd-4feb-94d1-6016fbb50c9a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:12.912842 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f40fefb9-56cd-4feb-94d1-6016fbb50c9a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:12.919655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f40fefb9-56cd-4feb-94d1-6016fbb50c9a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:12.920146 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f40fefb9-56cd-4feb-94d1-6016fbb50c9a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:12.920585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f40fefb9-56cd-4feb-94d1-6016fbb50c9a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:12.921357 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f40fefb9-56cd-4feb-94d1-6016fbb50c9a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:12.921809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f40fefb9-56cd-4feb-94d1-6016fbb50c9a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:12.922226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f40fefb9-56cd-4feb-94d1-6016fbb50c9a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:12.957523 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f40fefb9-56cd-4feb-94d1-6016fbb50c9a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050632 Aug 28 01:03:12.957998 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 538/994] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:12 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:12.962945 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2ae10376-2966-4ba6-84d6-99c12dde23ec tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:12.965295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2ae10376-2966-4ba6-84d6-99c12dde23ec tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:12.972561 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ae10376-2966-4ba6-84d6-99c12dde23ec tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:12.973084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ae10376-2966-4ba6-84d6-99c12dde23ec tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:12.973522 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ae10376-2966-4ba6-84d6-99c12dde23ec tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:12.974252 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ae10376-2966-4ba6-84d6-99c12dde23ec tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:12.974691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ae10376-2966-4ba6-84d6-99c12dde23ec tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:12.975122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ae10376-2966-4ba6-84d6-99c12dde23ec tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:13.008285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2ae10376-2966-4ba6-84d6-99c12dde23ec tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049125 Aug 28 01:03:13.008903 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 539/995] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:12 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:13.751909 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.objects.instance [None req-49a755e6-f11f-465d-a9e5-f40e97e328bb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lazy-loading 'services' on Instance uuid 03969ac5-934e-4201-8087-0927eae2bbd2 {{(pid=120524) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:03:13.930696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fecca363-0421-4b4e-80be-f468b1a65f7e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:13.931525 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fecca363-0421-4b4e-80be-f468b1a65f7e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:13.941162 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fecca363-0421-4b4e-80be-f468b1a65f7e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:13.942324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fecca363-0421-4b4e-80be-f468b1a65f7e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:13.943071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fecca363-0421-4b4e-80be-f468b1a65f7e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:13.944317 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fecca363-0421-4b4e-80be-f468b1a65f7e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:13.945167 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fecca363-0421-4b4e-80be-f468b1a65f7e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:13.945969 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fecca363-0421-4b4e-80be-f468b1a65f7e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:14.018936 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fecca363-0421-4b4e-80be-f468b1a65f7e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:14.027814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fecca363-0421-4b4e-80be-f468b1a65f7e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.101933 Aug 28 01:03:14.028412 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 540/996] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:13 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:14.033859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5ce12d90-a1e6-4030-81ed-67eb5de01076 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:14.036310 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5ce12d90-a1e6-4030-81ed-67eb5de01076 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:14.043619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ce12d90-a1e6-4030-81ed-67eb5de01076 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:14.044109 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ce12d90-a1e6-4030-81ed-67eb5de01076 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:14.044597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ce12d90-a1e6-4030-81ed-67eb5de01076 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:14.045533 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ce12d90-a1e6-4030-81ed-67eb5de01076 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:14.045993 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ce12d90-a1e6-4030-81ed-67eb5de01076 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:14.046417 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ce12d90-a1e6-4030-81ed-67eb5de01076 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:14.082275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5ce12d90-a1e6-4030-81ed-67eb5de01076 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052692 Aug 28 01:03:14.082854 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 541/997] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:14 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:14.087851 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-99919815-5f15-4afd-b7f2-9c3e5dd7190f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:14.090138 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-99919815-5f15-4afd-b7f2-9c3e5dd7190f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:14.096813 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-99919815-5f15-4afd-b7f2-9c3e5dd7190f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:14.097330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-99919815-5f15-4afd-b7f2-9c3e5dd7190f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:14.097761 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-99919815-5f15-4afd-b7f2-9c3e5dd7190f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:14.098622 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-99919815-5f15-4afd-b7f2-9c3e5dd7190f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:14.099072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-99919815-5f15-4afd-b7f2-9c3e5dd7190f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:14.099466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-99919815-5f15-4afd-b7f2-9c3e5dd7190f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:14.138231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-99919815-5f15-4afd-b7f2-9c3e5dd7190f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054253 Aug 28 01:03:14.138895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 542/998] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:14 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:15.050352 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-402f8b82-3c13-4a69-bf91-5068156158bf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:15.051118 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-402f8b82-3c13-4a69-bf91-5068156158bf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:15.058496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-402f8b82-3c13-4a69-bf91-5068156158bf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:15.058986 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-402f8b82-3c13-4a69-bf91-5068156158bf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:15.059420 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-402f8b82-3c13-4a69-bf91-5068156158bf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:15.060156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-402f8b82-3c13-4a69-bf91-5068156158bf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:15.060579 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-402f8b82-3c13-4a69-bf91-5068156158bf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:15.061159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-402f8b82-3c13-4a69-bf91-5068156158bf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:15.147044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-402f8b82-3c13-4a69-bf91-5068156158bf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:15.156253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-402f8b82-3c13-4a69-bf91-5068156158bf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.109886 Aug 28 01:03:15.156935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 543/999] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:15 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:03:15.161915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7a0a295e-b6a7-4aef-8aa6-ceeb5f648716 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:15.164109 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7a0a295e-b6a7-4aef-8aa6-ceeb5f648716 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:15.171321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7a0a295e-b6a7-4aef-8aa6-ceeb5f648716 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:15.172044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7a0a295e-b6a7-4aef-8aa6-ceeb5f648716 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:15.172748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7a0a295e-b6a7-4aef-8aa6-ceeb5f648716 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:15.173671 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7a0a295e-b6a7-4aef-8aa6-ceeb5f648716 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:15.174292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7a0a295e-b6a7-4aef-8aa6-ceeb5f648716 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:15.174993 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7a0a295e-b6a7-4aef-8aa6-ceeb5f648716 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:15.211740 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7a0a295e-b6a7-4aef-8aa6-ceeb5f648716 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053586 Aug 28 01:03:15.212469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 544/1000] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:15 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:15.220973 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-42978880-f39d-4dd8-87eb-d9619aefb1cb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:15.223641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-42978880-f39d-4dd8-87eb-d9619aefb1cb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:15.230967 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42978880-f39d-4dd8-87eb-d9619aefb1cb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:15.231457 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42978880-f39d-4dd8-87eb-d9619aefb1cb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:15.231999 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42978880-f39d-4dd8-87eb-d9619aefb1cb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:15.232860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42978880-f39d-4dd8-87eb-d9619aefb1cb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:15.233382 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42978880-f39d-4dd8-87eb-d9619aefb1cb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:15.233810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42978880-f39d-4dd8-87eb-d9619aefb1cb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:15.268325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-42978880-f39d-4dd8-87eb-d9619aefb1cb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053835 Aug 28 01:03:15.268978 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 545/1001] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:15 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:16.189499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-703c8f57-9c14-4073-b353-c581ad1f3ed3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:16.190259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-703c8f57-9c14-4073-b353-c581ad1f3ed3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:16.192933 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:03:16.201731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-703c8f57-9c14-4073-b353-c581ad1f3ed3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:16.202358 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-703c8f57-9c14-4073-b353-c581ad1f3ed3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:16.202894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-703c8f57-9c14-4073-b353-c581ad1f3ed3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:16.203768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-703c8f57-9c14-4073-b353-c581ad1f3ed3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:16.204308 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-703c8f57-9c14-4073-b353-c581ad1f3ed3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:16.204989 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-703c8f57-9c14-4073-b353-c581ad1f3ed3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:16.276281 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-703c8f57-9c14-4073-b353-c581ad1f3ed3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:16.285783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-703c8f57-9c14-4073-b353-c581ad1f3ed3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.100351 Aug 28 01:03:16.286409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 546/1002] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:16 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:03:16.292238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-335e23dc-0608-49c7-a66c-15d98f1d41c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:16.294791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-335e23dc-0608-49c7-a66c-15d98f1d41c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:16.301817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-335e23dc-0608-49c7-a66c-15d98f1d41c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:16.302320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-335e23dc-0608-49c7-a66c-15d98f1d41c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:16.302757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-335e23dc-0608-49c7-a66c-15d98f1d41c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:16.303476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-335e23dc-0608-49c7-a66c-15d98f1d41c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:16.304194 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-335e23dc-0608-49c7-a66c-15d98f1d41c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:16.304661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-335e23dc-0608-49c7-a66c-15d98f1d41c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:16.339248 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-335e23dc-0608-49c7-a66c-15d98f1d41c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051733 Aug 28 01:03:16.339852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 547/1003] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:16 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:16.344868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1dfaa6dd-2e8d-4dee-9d20-62863f6aa801 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:16.347157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1dfaa6dd-2e8d-4dee-9d20-62863f6aa801 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:16.353761 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1dfaa6dd-2e8d-4dee-9d20-62863f6aa801 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:16.354250 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1dfaa6dd-2e8d-4dee-9d20-62863f6aa801 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:16.354684 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1dfaa6dd-2e8d-4dee-9d20-62863f6aa801 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:16.355400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1dfaa6dd-2e8d-4dee-9d20-62863f6aa801 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:16.355857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1dfaa6dd-2e8d-4dee-9d20-62863f6aa801 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:16.356462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1dfaa6dd-2e8d-4dee-9d20-62863f6aa801 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:16.390680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1dfaa6dd-2e8d-4dee-9d20-62863f6aa801 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049690 Aug 28 01:03:16.391493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 548/1004] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:16 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:17.320954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5c9fa828-0b75-4052-927f-867103c87a16 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:17.321971 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5c9fa828-0b75-4052-927f-867103c87a16 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:17.330440 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5c9fa828-0b75-4052-927f-867103c87a16 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:17.331278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5c9fa828-0b75-4052-927f-867103c87a16 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:17.332042 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5c9fa828-0b75-4052-927f-867103c87a16 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:17.333367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5c9fa828-0b75-4052-927f-867103c87a16 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:17.334186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5c9fa828-0b75-4052-927f-867103c87a16 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:17.334939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5c9fa828-0b75-4052-927f-867103c87a16 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:17.410867 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5c9fa828-0b75-4052-927f-867103c87a16 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:17.419673 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5c9fa828-0b75-4052-927f-867103c87a16 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.102736 Aug 28 01:03:17.420246 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 549/1005] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:17 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:17.425303 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cd7b24ac-4239-4c13-969d-b1f8c1ba87f6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:17.428057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cd7b24ac-4239-4c13-969d-b1f8c1ba87f6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:17.434560 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cd7b24ac-4239-4c13-969d-b1f8c1ba87f6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:17.435067 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cd7b24ac-4239-4c13-969d-b1f8c1ba87f6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:17.435496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cd7b24ac-4239-4c13-969d-b1f8c1ba87f6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:17.436219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cd7b24ac-4239-4c13-969d-b1f8c1ba87f6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:17.436674 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cd7b24ac-4239-4c13-969d-b1f8c1ba87f6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:17.437134 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cd7b24ac-4239-4c13-969d-b1f8c1ba87f6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:17.465918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:03:17.472514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cd7b24ac-4239-4c13-969d-b1f8c1ba87f6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051102 Aug 28 01:03:17.473112 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 550/1006] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:17 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:17.478498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-afc1e61f-50a4-41fa-89d7-f9de76bb9df7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:17.481026 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-afc1e61f-50a4-41fa-89d7-f9de76bb9df7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:17.487502 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-afc1e61f-50a4-41fa-89d7-f9de76bb9df7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:17.487992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-afc1e61f-50a4-41fa-89d7-f9de76bb9df7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:17.488426 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-afc1e61f-50a4-41fa-89d7-f9de76bb9df7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:17.489176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-afc1e61f-50a4-41fa-89d7-f9de76bb9df7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:17.489629 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-afc1e61f-50a4-41fa-89d7-f9de76bb9df7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:17.490053 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-afc1e61f-50a4-41fa-89d7-f9de76bb9df7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:17.526730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-afc1e61f-50a4-41fa-89d7-f9de76bb9df7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052323 Aug 28 01:03:17.527324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 551/1007] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:17 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:18.449731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-895f6b23-4e16-4d4a-8d63-82f0d568b73b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:18.450887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-895f6b23-4e16-4d4a-8d63-82f0d568b73b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:18.458488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-895f6b23-4e16-4d4a-8d63-82f0d568b73b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:18.459062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-895f6b23-4e16-4d4a-8d63-82f0d568b73b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:18.459498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-895f6b23-4e16-4d4a-8d63-82f0d568b73b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:18.460198 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-895f6b23-4e16-4d4a-8d63-82f0d568b73b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:18.460647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-895f6b23-4e16-4d4a-8d63-82f0d568b73b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:18.461121 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-895f6b23-4e16-4d4a-8d63-82f0d568b73b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:18.529986 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-895f6b23-4e16-4d4a-8d63-82f0d568b73b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:18.546441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-895f6b23-4e16-4d4a-8d63-82f0d568b73b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.100261 Aug 28 01:03:18.547488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 552/1008] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:18 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:18.555678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b6cad538-aa9d-4695-9af1-9cb7421e8164 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:18.559245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b6cad538-aa9d-4695-9af1-9cb7421e8164 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:18.569055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b6cad538-aa9d-4695-9af1-9cb7421e8164 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:18.569512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b6cad538-aa9d-4695-9af1-9cb7421e8164 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:18.570007 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b6cad538-aa9d-4695-9af1-9cb7421e8164 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:18.570729 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b6cad538-aa9d-4695-9af1-9cb7421e8164 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:18.571199 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b6cad538-aa9d-4695-9af1-9cb7421e8164 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:18.571622 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b6cad538-aa9d-4695-9af1-9cb7421e8164 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:18.605303 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b6cad538-aa9d-4695-9af1-9cb7421e8164 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.056340 Aug 28 01:03:18.605932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 553/1009] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:18 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:18.610925 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-360b5415-df39-431c-bf8f-4464b8909b75 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:18.613172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-360b5415-df39-431c-bf8f-4464b8909b75 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:18.619743 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-360b5415-df39-431c-bf8f-4464b8909b75 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:18.620242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-360b5415-df39-431c-bf8f-4464b8909b75 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:18.620679 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-360b5415-df39-431c-bf8f-4464b8909b75 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:18.621470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-360b5415-df39-431c-bf8f-4464b8909b75 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:18.621923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-360b5415-df39-431c-bf8f-4464b8909b75 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:18.622349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-360b5415-df39-431c-bf8f-4464b8909b75 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:18.656650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-360b5415-df39-431c-bf8f-4464b8909b75 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049542 Aug 28 01:03:18.657288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 554/1010] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:18 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:19.568902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-79d1358b-c9f8-473f-b59f-931941fd52da tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:19.569688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-79d1358b-c9f8-473f-b59f-931941fd52da tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:19.576877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79d1358b-c9f8-473f-b59f-931941fd52da tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:19.577441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79d1358b-c9f8-473f-b59f-931941fd52da tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:19.577842 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79d1358b-c9f8-473f-b59f-931941fd52da tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:19.578653 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79d1358b-c9f8-473f-b59f-931941fd52da tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:19.579096 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79d1358b-c9f8-473f-b59f-931941fd52da tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:19.579522 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79d1358b-c9f8-473f-b59f-931941fd52da tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:19.649621 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-79d1358b-c9f8-473f-b59f-931941fd52da tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:19.658439 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-79d1358b-c9f8-473f-b59f-931941fd52da tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.094118 Aug 28 01:03:19.659033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 555/1011] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:19 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:03:19.663982 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-59df9ebb-b205-4e0e-ae87-2c79ee61a677 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:19.666241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-59df9ebb-b205-4e0e-ae87-2c79ee61a677 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:19.673412 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-59df9ebb-b205-4e0e-ae87-2c79ee61a677 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:19.673904 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-59df9ebb-b205-4e0e-ae87-2c79ee61a677 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:19.674337 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-59df9ebb-b205-4e0e-ae87-2c79ee61a677 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:19.675087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-59df9ebb-b205-4e0e-ae87-2c79ee61a677 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:19.675537 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-59df9ebb-b205-4e0e-ae87-2c79ee61a677 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:19.675963 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-59df9ebb-b205-4e0e-ae87-2c79ee61a677 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:19.710616 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-59df9ebb-b205-4e0e-ae87-2c79ee61a677 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050457 Aug 28 01:03:19.711244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 556/1012] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:19 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:19.716495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d7c9f93e-ccb5-4bff-aada-cc51acb6a6bd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:19.718829 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d7c9f93e-ccb5-4bff-aada-cc51acb6a6bd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:19.725719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c9f93e-ccb5-4bff-aada-cc51acb6a6bd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:19.726224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c9f93e-ccb5-4bff-aada-cc51acb6a6bd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:19.726666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c9f93e-ccb5-4bff-aada-cc51acb6a6bd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:19.727387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c9f93e-ccb5-4bff-aada-cc51acb6a6bd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:19.727850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c9f93e-ccb5-4bff-aada-cc51acb6a6bd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:19.728274 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c9f93e-ccb5-4bff-aada-cc51acb6a6bd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:19.763646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d7c9f93e-ccb5-4bff-aada-cc51acb6a6bd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051140 Aug 28 01:03:19.764279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 557/1013] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:19 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:20.692631 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-15f1fbed-6094-451d-9eb7-1a160e22aab9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:20.693716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-15f1fbed-6094-451d-9eb7-1a160e22aab9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:20.705615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15f1fbed-6094-451d-9eb7-1a160e22aab9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:20.706469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15f1fbed-6094-451d-9eb7-1a160e22aab9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:20.707232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15f1fbed-6094-451d-9eb7-1a160e22aab9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:20.708445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15f1fbed-6094-451d-9eb7-1a160e22aab9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:20.709295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15f1fbed-6094-451d-9eb7-1a160e22aab9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:20.710053 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15f1fbed-6094-451d-9eb7-1a160e22aab9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:20.816144 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-15f1fbed-6094-451d-9eb7-1a160e22aab9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:20.826964 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-15f1fbed-6094-451d-9eb7-1a160e22aab9 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.139593 Aug 28 01:03:20.827898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 558/1014] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:20 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:03:20.834863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c1f171f4-0ae3-4e0a-8465-394e441d1d93 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:20.837779 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c1f171f4-0ae3-4e0a-8465-394e441d1d93 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:20.846101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c1f171f4-0ae3-4e0a-8465-394e441d1d93 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:20.846742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c1f171f4-0ae3-4e0a-8465-394e441d1d93 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:20.847345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c1f171f4-0ae3-4e0a-8465-394e441d1d93 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:20.848265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c1f171f4-0ae3-4e0a-8465-394e441d1d93 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:20.848898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c1f171f4-0ae3-4e0a-8465-394e441d1d93 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:20.849642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c1f171f4-0ae3-4e0a-8465-394e441d1d93 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:20.890333 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c1f171f4-0ae3-4e0a-8465-394e441d1d93 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.060834 Aug 28 01:03:20.891482 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 559/1015] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:20 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:20.899774 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ebf991e0-86fd-402b-baec-91d449a075ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:20.903887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ebf991e0-86fd-402b-baec-91d449a075ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:20.914858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebf991e0-86fd-402b-baec-91d449a075ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:20.915700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebf991e0-86fd-402b-baec-91d449a075ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:20.916501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebf991e0-86fd-402b-baec-91d449a075ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:20.917746 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebf991e0-86fd-402b-baec-91d449a075ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:20.918486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebf991e0-86fd-402b-baec-91d449a075ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:20.919178 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebf991e0-86fd-402b-baec-91d449a075ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:20.974249 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ebf991e0-86fd-402b-baec-91d449a075ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.080692 Aug 28 01:03:20.975215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 560/1016] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:20 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:21.856540 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9d699c35-7e10-4b5f-b786-cb4db8cb5874 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:21.857321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9d699c35-7e10-4b5f-b786-cb4db8cb5874 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:21.864861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d699c35-7e10-4b5f-b786-cb4db8cb5874 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:21.865388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d699c35-7e10-4b5f-b786-cb4db8cb5874 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:21.865820 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d699c35-7e10-4b5f-b786-cb4db8cb5874 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:21.866545 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d699c35-7e10-4b5f-b786-cb4db8cb5874 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:21.866994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d699c35-7e10-4b5f-b786-cb4db8cb5874 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:21.867414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d699c35-7e10-4b5f-b786-cb4db8cb5874 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:21.947337 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9d699c35-7e10-4b5f-b786-cb4db8cb5874 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:21.955926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9d699c35-7e10-4b5f-b786-cb4db8cb5874 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.103487 Aug 28 01:03:21.956512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 561/1017] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:21 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:03:21.961528 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6f2e8d96-e359-4d23-9158-b8c33f07895e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:21.963832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6f2e8d96-e359-4d23-9158-b8c33f07895e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:21.970691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6f2e8d96-e359-4d23-9158-b8c33f07895e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:21.971066 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6f2e8d96-e359-4d23-9158-b8c33f07895e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:21.971498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6f2e8d96-e359-4d23-9158-b8c33f07895e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:21.972216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6f2e8d96-e359-4d23-9158-b8c33f07895e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:21.972665 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6f2e8d96-e359-4d23-9158-b8c33f07895e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:21.973121 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6f2e8d96-e359-4d23-9158-b8c33f07895e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:22.006658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6f2e8d96-e359-4d23-9158-b8c33f07895e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049006 Aug 28 01:03:22.007541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 562/1018] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:21 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:22.012328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e78c50f8-96ce-417a-9929-8dcfbd76acaf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:22.014542 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e78c50f8-96ce-417a-9929-8dcfbd76acaf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:22.021323 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e78c50f8-96ce-417a-9929-8dcfbd76acaf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:22.021808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e78c50f8-96ce-417a-9929-8dcfbd76acaf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:22.022241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e78c50f8-96ce-417a-9929-8dcfbd76acaf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:22.022953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e78c50f8-96ce-417a-9929-8dcfbd76acaf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:22.023405 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e78c50f8-96ce-417a-9929-8dcfbd76acaf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:22.023844 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e78c50f8-96ce-417a-9929-8dcfbd76acaf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:22.058426 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e78c50f8-96ce-417a-9929-8dcfbd76acaf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049961 Aug 28 01:03:22.059013 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 563/1019] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:22 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:22.988396 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-704b1192-9676-45e1-aac3-5a2e64d988ac tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:22.989144 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-704b1192-9676-45e1-aac3-5a2e64d988ac tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:22.996415 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-704b1192-9676-45e1-aac3-5a2e64d988ac tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:22.996931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-704b1192-9676-45e1-aac3-5a2e64d988ac tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:22.997401 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-704b1192-9676-45e1-aac3-5a2e64d988ac tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:22.998128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-704b1192-9676-45e1-aac3-5a2e64d988ac tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:22.998587 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-704b1192-9676-45e1-aac3-5a2e64d988ac tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:22.999016 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-704b1192-9676-45e1-aac3-5a2e64d988ac tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:23.067145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-704b1192-9676-45e1-aac3-5a2e64d988ac tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:23.076040 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-704b1192-9676-45e1-aac3-5a2e64d988ac tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.091796 Aug 28 01:03:23.076708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 564/1020] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:22 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:23.081918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-874303b4-9620-4b1e-9a58-0979f4bf2f33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:23.084134 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-874303b4-9620-4b1e-9a58-0979f4bf2f33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:23.091112 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-874303b4-9620-4b1e-9a58-0979f4bf2f33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:23.091606 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-874303b4-9620-4b1e-9a58-0979f4bf2f33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:23.092040 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-874303b4-9620-4b1e-9a58-0979f4bf2f33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:23.092757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-874303b4-9620-4b1e-9a58-0979f4bf2f33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:23.093243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-874303b4-9620-4b1e-9a58-0979f4bf2f33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:23.093663 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-874303b4-9620-4b1e-9a58-0979f4bf2f33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:23.129298 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-874303b4-9620-4b1e-9a58-0979f4bf2f33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051368 Aug 28 01:03:23.129904 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 565/1021] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:23 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:23.135002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d9655e78-b120-44fd-b25d-3b66212befb9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:23.137229 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d9655e78-b120-44fd-b25d-3b66212befb9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:23.143909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d9655e78-b120-44fd-b25d-3b66212befb9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:23.144399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d9655e78-b120-44fd-b25d-3b66212befb9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:23.144833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d9655e78-b120-44fd-b25d-3b66212befb9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:23.145594 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d9655e78-b120-44fd-b25d-3b66212befb9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:23.146048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d9655e78-b120-44fd-b25d-3b66212befb9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:23.146476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d9655e78-b120-44fd-b25d-3b66212befb9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:23.181109 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d9655e78-b120-44fd-b25d-3b66212befb9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050008 Aug 28 01:03:23.181716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 566/1022] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:23 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:23.771230 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=4 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:03:24.102220 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9590ba34-41c0-46ab-a1f4-e66e8b8cc6ee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:24.102962 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9590ba34-41c0-46ab-a1f4-e66e8b8cc6ee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:24.114590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9590ba34-41c0-46ab-a1f4-e66e8b8cc6ee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:24.115251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9590ba34-41c0-46ab-a1f4-e66e8b8cc6ee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:24.115958 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9590ba34-41c0-46ab-a1f4-e66e8b8cc6ee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:24.117135 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9590ba34-41c0-46ab-a1f4-e66e8b8cc6ee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:24.118025 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9590ba34-41c0-46ab-a1f4-e66e8b8cc6ee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:24.118711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9590ba34-41c0-46ab-a1f4-e66e8b8cc6ee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:24.142692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:03:24.192549 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9590ba34-41c0-46ab-a1f4-e66e8b8cc6ee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:24.201589 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9590ba34-41c0-46ab-a1f4-e66e8b8cc6ee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.106556 Aug 28 01:03:24.202198 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 567/1023] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:24 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:24.207486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1b374507-bb24-41be-a6e2-e3bf46ab0da7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:24.209691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1b374507-bb24-41be-a6e2-e3bf46ab0da7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:24.216311 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b374507-bb24-41be-a6e2-e3bf46ab0da7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:24.216825 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b374507-bb24-41be-a6e2-e3bf46ab0da7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:24.217430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b374507-bb24-41be-a6e2-e3bf46ab0da7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:24.218214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b374507-bb24-41be-a6e2-e3bf46ab0da7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:24.218684 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b374507-bb24-41be-a6e2-e3bf46ab0da7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:24.219158 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b374507-bb24-41be-a6e2-e3bf46ab0da7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:24.253752 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1b374507-bb24-41be-a6e2-e3bf46ab0da7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050246 Aug 28 01:03:24.254406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 568/1024] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:24 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:24.259320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8b0da56d-2f4b-45ee-9428-af9796738e83 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:24.261618 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8b0da56d-2f4b-45ee-9428-af9796738e83 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:24.270022 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b0da56d-2f4b-45ee-9428-af9796738e83 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:24.270718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b0da56d-2f4b-45ee-9428-af9796738e83 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:24.271378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b0da56d-2f4b-45ee-9428-af9796738e83 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:24.272168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b0da56d-2f4b-45ee-9428-af9796738e83 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:24.272712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b0da56d-2f4b-45ee-9428-af9796738e83 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:24.273219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b0da56d-2f4b-45ee-9428-af9796738e83 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:24.309810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8b0da56d-2f4b-45ee-9428-af9796738e83 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054147 Aug 28 01:03:24.310472 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 569/1025] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:24 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:25.236493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3c839fdb-8fe0-43fd-8c8a-b94fdecffbcf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:25.237639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3c839fdb-8fe0-43fd-8c8a-b94fdecffbcf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:25.246112 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3c839fdb-8fe0-43fd-8c8a-b94fdecffbcf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:25.246739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3c839fdb-8fe0-43fd-8c8a-b94fdecffbcf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:25.247272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3c839fdb-8fe0-43fd-8c8a-b94fdecffbcf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:25.248102 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3c839fdb-8fe0-43fd-8c8a-b94fdecffbcf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:25.248661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3c839fdb-8fe0-43fd-8c8a-b94fdecffbcf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:25.249329 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3c839fdb-8fe0-43fd-8c8a-b94fdecffbcf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:25.327690 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3c839fdb-8fe0-43fd-8c8a-b94fdecffbcf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:25.342587 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3c839fdb-8fe0-43fd-8c8a-b94fdecffbcf tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.110813 Aug 28 01:03:25.343245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 570/1026] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:25 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:25.348619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-69eaa9a8-d752-496f-8ac2-126744c7b92b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:25.350928 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-69eaa9a8-d752-496f-8ac2-126744c7b92b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:25.358139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69eaa9a8-d752-496f-8ac2-126744c7b92b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:25.358626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69eaa9a8-d752-496f-8ac2-126744c7b92b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:25.359058 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69eaa9a8-d752-496f-8ac2-126744c7b92b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:25.359841 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69eaa9a8-d752-496f-8ac2-126744c7b92b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:25.360289 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69eaa9a8-d752-496f-8ac2-126744c7b92b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:25.360709 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69eaa9a8-d752-496f-8ac2-126744c7b92b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:25.550293 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-69eaa9a8-d752-496f-8ac2-126744c7b92b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.205701 Aug 28 01:03:25.550872 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 571/1027] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:25 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:25.556081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b7c7f1b1-b192-4541-a82e-bfff5a373df4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:25.558328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b7c7f1b1-b192-4541-a82e-bfff5a373df4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:25.565193 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7c7f1b1-b192-4541-a82e-bfff5a373df4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:25.565861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7c7f1b1-b192-4541-a82e-bfff5a373df4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:25.566340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7c7f1b1-b192-4541-a82e-bfff5a373df4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:25.567585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7c7f1b1-b192-4541-a82e-bfff5a373df4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:25.568086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7c7f1b1-b192-4541-a82e-bfff5a373df4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:25.568534 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7c7f1b1-b192-4541-a82e-bfff5a373df4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:25.604332 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b7c7f1b1-b192-4541-a82e-bfff5a373df4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052267 Aug 28 01:03:25.605040 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 572/1028] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:25 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:26.366948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-089276ae-86f9-49d3-b0f4-4df9dfccfa7f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:26.367830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-089276ae-86f9-49d3-b0f4-4df9dfccfa7f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:26.375069 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-089276ae-86f9-49d3-b0f4-4df9dfccfa7f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:26.375471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-089276ae-86f9-49d3-b0f4-4df9dfccfa7f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:26.375915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-089276ae-86f9-49d3-b0f4-4df9dfccfa7f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:26.376919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-089276ae-86f9-49d3-b0f4-4df9dfccfa7f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:26.377423 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-089276ae-86f9-49d3-b0f4-4df9dfccfa7f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:26.377853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-089276ae-86f9-49d3-b0f4-4df9dfccfa7f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:26.468390 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-089276ae-86f9-49d3-b0f4-4df9dfccfa7f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:26.477520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-089276ae-86f9-49d3-b0f4-4df9dfccfa7f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.115013 Aug 28 01:03:26.478147 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 573/1029] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:26 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:26.565941 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cfe58e45-8591-4e27-b75d-63513ef58689 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:26.569905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cfe58e45-8591-4e27-b75d-63513ef58689 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:26.580378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cfe58e45-8591-4e27-b75d-63513ef58689 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:26.581268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cfe58e45-8591-4e27-b75d-63513ef58689 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:26.582036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cfe58e45-8591-4e27-b75d-63513ef58689 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:26.583301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cfe58e45-8591-4e27-b75d-63513ef58689 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:26.584121 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cfe58e45-8591-4e27-b75d-63513ef58689 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:26.584876 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cfe58e45-8591-4e27-b75d-63513ef58689 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:26.642015 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cfe58e45-8591-4e27-b75d-63513ef58689 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.082240 Aug 28 01:03:26.643047 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 574/1030] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:26 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 28 01:03:26.653597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b820094f-f31e-4499-9e0f-17ca702ec654 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:26.657280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b820094f-f31e-4499-9e0f-17ca702ec654 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:26.667170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b820094f-f31e-4499-9e0f-17ca702ec654 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:26.668016 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b820094f-f31e-4499-9e0f-17ca702ec654 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:26.668755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b820094f-f31e-4499-9e0f-17ca702ec654 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:26.669941 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b820094f-f31e-4499-9e0f-17ca702ec654 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:26.670422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b820094f-f31e-4499-9e0f-17ca702ec654 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:26.670850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b820094f-f31e-4499-9e0f-17ca702ec654 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:26.710044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b820094f-f31e-4499-9e0f-17ca702ec654 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.065340 Aug 28 01:03:26.711357 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 575/1031] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:26 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:27.499435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fafeb3af-5f94-485d-89c9-dba9e410ebe7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:27.500233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fafeb3af-5f94-485d-89c9-dba9e410ebe7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:27.507468 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fafeb3af-5f94-485d-89c9-dba9e410ebe7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:27.507963 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fafeb3af-5f94-485d-89c9-dba9e410ebe7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:27.508402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fafeb3af-5f94-485d-89c9-dba9e410ebe7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:27.509175 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fafeb3af-5f94-485d-89c9-dba9e410ebe7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:27.509616 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fafeb3af-5f94-485d-89c9-dba9e410ebe7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:27.510040 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fafeb3af-5f94-485d-89c9-dba9e410ebe7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:27.577774 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fafeb3af-5f94-485d-89c9-dba9e410ebe7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:27.587082 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fafeb3af-5f94-485d-89c9-dba9e410ebe7 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.091544 Aug 28 01:03:27.587675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 576/1032] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:27 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:03:27.656284 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-592936dd-a620-42a1-965e-1972ba7835e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:27.658933 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-592936dd-a620-42a1-965e-1972ba7835e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:27.665989 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-592936dd-a620-42a1-965e-1972ba7835e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:27.666647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-592936dd-a620-42a1-965e-1972ba7835e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:27.667132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-592936dd-a620-42a1-965e-1972ba7835e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:27.667973 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-592936dd-a620-42a1-965e-1972ba7835e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:27.668474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-592936dd-a620-42a1-965e-1972ba7835e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:27.669136 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-592936dd-a620-42a1-965e-1972ba7835e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:27.706256 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-592936dd-a620-42a1-965e-1972ba7835e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054252 Aug 28 01:03:27.706941 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 577/1033] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:27 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Aug 28 01:03:27.726828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a2ffee96-d7b3-47e6-b7c2-a15ba2a4bd61 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:27.729272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a2ffee96-d7b3-47e6-b7c2-a15ba2a4bd61 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:27.736237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2ffee96-d7b3-47e6-b7c2-a15ba2a4bd61 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:27.736803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2ffee96-d7b3-47e6-b7c2-a15ba2a4bd61 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:27.737312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2ffee96-d7b3-47e6-b7c2-a15ba2a4bd61 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:27.738075 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2ffee96-d7b3-47e6-b7c2-a15ba2a4bd61 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:27.738616 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2ffee96-d7b3-47e6-b7c2-a15ba2a4bd61 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:27.739093 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2ffee96-d7b3-47e6-b7c2-a15ba2a4bd61 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:27.772651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a2ffee96-d7b3-47e6-b7c2-a15ba2a4bd61 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049843 Aug 28 01:03:27.773283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 578/1034] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:27 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:03:28.620186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-615f5cc0-4047-4841-bb1a-46a0f1628b85 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:28.620983 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-615f5cc0-4047-4841-bb1a-46a0f1628b85 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:28.629377 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-615f5cc0-4047-4841-bb1a-46a0f1628b85 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:28.629873 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-615f5cc0-4047-4841-bb1a-46a0f1628b85 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:28.630300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-615f5cc0-4047-4841-bb1a-46a0f1628b85 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:28.630995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-615f5cc0-4047-4841-bb1a-46a0f1628b85 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:28.631440 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-615f5cc0-4047-4841-bb1a-46a0f1628b85 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:28.631856 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-615f5cc0-4047-4841-bb1a-46a0f1628b85 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:28.703864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-615f5cc0-4047-4841-bb1a-46a0f1628b85 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:28.717917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-615f5cc0-4047-4841-bb1a-46a0f1628b85 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.101209 Aug 28 01:03:28.718996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 579/1035] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:28 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:03:28.727226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5c2cb49d-3b2a-484c-96e9-05bc15241183 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:28.729460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5c2cb49d-3b2a-484c-96e9-05bc15241183 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:28.736139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5c2cb49d-3b2a-484c-96e9-05bc15241183 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:28.736591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5c2cb49d-3b2a-484c-96e9-05bc15241183 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:28.737048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5c2cb49d-3b2a-484c-96e9-05bc15241183 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:28.737768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5c2cb49d-3b2a-484c-96e9-05bc15241183 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:28.738214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5c2cb49d-3b2a-484c-96e9-05bc15241183 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:28.738625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5c2cb49d-3b2a-484c-96e9-05bc15241183 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:28.771881 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5c2cb49d-3b2a-484c-96e9-05bc15241183 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051228 Aug 28 01:03:28.772615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 580/1036] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:28 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:28.785832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6994a40b-ee5b-42c5-b012-83ff44a60a86 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:28.788083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6994a40b-ee5b-42c5-b012-83ff44a60a86 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:28.796276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6994a40b-ee5b-42c5-b012-83ff44a60a86 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:28.797152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6994a40b-ee5b-42c5-b012-83ff44a60a86 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:28.797934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6994a40b-ee5b-42c5-b012-83ff44a60a86 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:28.799466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6994a40b-ee5b-42c5-b012-83ff44a60a86 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:28.800375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6994a40b-ee5b-42c5-b012-83ff44a60a86 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:28.801125 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6994a40b-ee5b-42c5-b012-83ff44a60a86 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:28.837511 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6994a40b-ee5b-42c5-b012-83ff44a60a86 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056031 Aug 28 01:03:28.838289 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 581/1037] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:28 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:29.740058 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-08ff2113-00e3-400a-9124-8c1d83d2330d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:29.740900 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-08ff2113-00e3-400a-9124-8c1d83d2330d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:29.748285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08ff2113-00e3-400a-9124-8c1d83d2330d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:29.748767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08ff2113-00e3-400a-9124-8c1d83d2330d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:29.749236 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08ff2113-00e3-400a-9124-8c1d83d2330d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:29.750032 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08ff2113-00e3-400a-9124-8c1d83d2330d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:29.750484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08ff2113-00e3-400a-9124-8c1d83d2330d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:29.750907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08ff2113-00e3-400a-9124-8c1d83d2330d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:29.819214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-08ff2113-00e3-400a-9124-8c1d83d2330d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:29.829158 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-08ff2113-00e3-400a-9124-8c1d83d2330d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.092908 Aug 28 01:03:29.829794 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 582/1038] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:29 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:03:29.836452 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-415f9eb7-b7b1-4a2b-9a5e-2f6b77e6067c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:29.839097 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-415f9eb7-b7b1-4a2b-9a5e-2f6b77e6067c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:29.848083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-415f9eb7-b7b1-4a2b-9a5e-2f6b77e6067c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:29.848721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-415f9eb7-b7b1-4a2b-9a5e-2f6b77e6067c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:29.849418 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-415f9eb7-b7b1-4a2b-9a5e-2f6b77e6067c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:29.850187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-415f9eb7-b7b1-4a2b-9a5e-2f6b77e6067c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:29.850690 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-415f9eb7-b7b1-4a2b-9a5e-2f6b77e6067c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:29.851172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-415f9eb7-b7b1-4a2b-9a5e-2f6b77e6067c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:29.886023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-415f9eb7-b7b1-4a2b-9a5e-2f6b77e6067c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054685 Aug 28 01:03:29.886591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 583/1039] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:29 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:29.893647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-10a905c6-a0ad-4e7d-a94c-61a8ea982773 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:29.896191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-10a905c6-a0ad-4e7d-a94c-61a8ea982773 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:29.904181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10a905c6-a0ad-4e7d-a94c-61a8ea982773 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:29.904683 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10a905c6-a0ad-4e7d-a94c-61a8ea982773 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:29.905177 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10a905c6-a0ad-4e7d-a94c-61a8ea982773 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:29.906025 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10a905c6-a0ad-4e7d-a94c-61a8ea982773 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:29.906513 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10a905c6-a0ad-4e7d-a94c-61a8ea982773 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:29.906916 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10a905c6-a0ad-4e7d-a94c-61a8ea982773 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:29.911468 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:03:29.943304 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-10a905c6-a0ad-4e7d-a94c-61a8ea982773 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055548 Aug 28 01:03:29.943940 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 584/1040] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:29 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:30.852756 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-aa83025c-ded3-4c17-81ef-606546454445 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:30.853646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-aa83025c-ded3-4c17-81ef-606546454445 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:30.860910 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa83025c-ded3-4c17-81ef-606546454445 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:30.861444 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa83025c-ded3-4c17-81ef-606546454445 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:30.861882 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa83025c-ded3-4c17-81ef-606546454445 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:30.862605 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa83025c-ded3-4c17-81ef-606546454445 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:30.863060 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa83025c-ded3-4c17-81ef-606546454445 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:30.863492 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa83025c-ded3-4c17-81ef-606546454445 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:30.934594 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-aa83025c-ded3-4c17-81ef-606546454445 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:30.943568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-aa83025c-ded3-4c17-81ef-606546454445 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.094995 Aug 28 01:03:30.944163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 585/1041] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:30 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:03:30.949419 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-893572d6-22c2-4526-b241-651b31fea709 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:30.951654 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-893572d6-22c2-4526-b241-651b31fea709 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:30.958630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-893572d6-22c2-4526-b241-651b31fea709 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:30.959118 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-893572d6-22c2-4526-b241-651b31fea709 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:30.959546 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-893572d6-22c2-4526-b241-651b31fea709 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:30.960269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-893572d6-22c2-4526-b241-651b31fea709 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:30.960717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-893572d6-22c2-4526-b241-651b31fea709 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:30.961168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-893572d6-22c2-4526-b241-651b31fea709 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:30.994609 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-893572d6-22c2-4526-b241-651b31fea709 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049334 Aug 28 01:03:30.995193 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 586/1042] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:30 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:31.000294 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-69c70159-1e0b-4140-ae7c-c02cda7ecfae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:31.002634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-69c70159-1e0b-4140-ae7c-c02cda7ecfae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:31.009057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69c70159-1e0b-4140-ae7c-c02cda7ecfae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:31.009548 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69c70159-1e0b-4140-ae7c-c02cda7ecfae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:31.009972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69c70159-1e0b-4140-ae7c-c02cda7ecfae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:31.010671 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69c70159-1e0b-4140-ae7c-c02cda7ecfae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:31.011108 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69c70159-1e0b-4140-ae7c-c02cda7ecfae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:31.011517 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69c70159-1e0b-4140-ae7c-c02cda7ecfae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:31.045334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-69c70159-1e0b-4140-ae7c-c02cda7ecfae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048899 Aug 28 01:03:31.045920 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 587/1043] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:30 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:31.969631 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-07dd9af6-6c27-46eb-9896-b6aa27ae989c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:31.970354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-07dd9af6-6c27-46eb-9896-b6aa27ae989c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:31.978233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07dd9af6-6c27-46eb-9896-b6aa27ae989c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:31.978782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07dd9af6-6c27-46eb-9896-b6aa27ae989c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:31.979240 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07dd9af6-6c27-46eb-9896-b6aa27ae989c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:31.980076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07dd9af6-6c27-46eb-9896-b6aa27ae989c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:31.980622 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07dd9af6-6c27-46eb-9896-b6aa27ae989c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:31.981285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07dd9af6-6c27-46eb-9896-b6aa27ae989c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:32.057467 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-07dd9af6-6c27-46eb-9896-b6aa27ae989c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:32.066898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-07dd9af6-6c27-46eb-9896-b6aa27ae989c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.101156 Aug 28 01:03:32.067501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 588/1044] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:31 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:03:32.072932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c61c1ca2-f479-434e-aa47-82c249eaa1ec tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:32.075555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c61c1ca2-f479-434e-aa47-82c249eaa1ec tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:32.083209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c61c1ca2-f479-434e-aa47-82c249eaa1ec tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:32.083720 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c61c1ca2-f479-434e-aa47-82c249eaa1ec tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:32.084255 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c61c1ca2-f479-434e-aa47-82c249eaa1ec tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:32.085057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c61c1ca2-f479-434e-aa47-82c249eaa1ec tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:32.085571 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c61c1ca2-f479-434e-aa47-82c249eaa1ec tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:32.086171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c61c1ca2-f479-434e-aa47-82c249eaa1ec tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:32.129592 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c61c1ca2-f479-434e-aa47-82c249eaa1ec tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.060862 Aug 28 01:03:32.130589 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 589/1045] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:32 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:32.136540 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6d1be437-b633-453f-b402-284ca545ba64 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:32.139461 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6d1be437-b633-453f-b402-284ca545ba64 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:32.148286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d1be437-b633-453f-b402-284ca545ba64 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:32.148847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d1be437-b633-453f-b402-284ca545ba64 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:32.149396 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d1be437-b633-453f-b402-284ca545ba64 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:32.150442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d1be437-b633-453f-b402-284ca545ba64 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:32.151005 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d1be437-b633-453f-b402-284ca545ba64 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:32.151639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d1be437-b633-453f-b402-284ca545ba64 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:32.189357 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6d1be437-b633-453f-b402-284ca545ba64 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057207 Aug 28 01:03:32.189986 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 590/1046] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:32 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:33.091343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-127f8ad9-2366-4a41-8fa3-473a4fedc4c1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:33.092355 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-127f8ad9-2366-4a41-8fa3-473a4fedc4c1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:33.100225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-127f8ad9-2366-4a41-8fa3-473a4fedc4c1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:33.100870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-127f8ad9-2366-4a41-8fa3-473a4fedc4c1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:33.101433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-127f8ad9-2366-4a41-8fa3-473a4fedc4c1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:33.102397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-127f8ad9-2366-4a41-8fa3-473a4fedc4c1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:33.102912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-127f8ad9-2366-4a41-8fa3-473a4fedc4c1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:33.103622 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-127f8ad9-2366-4a41-8fa3-473a4fedc4c1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:33.183361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-127f8ad9-2366-4a41-8fa3-473a4fedc4c1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:33.199937 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-127f8ad9-2366-4a41-8fa3-473a4fedc4c1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.112320 Aug 28 01:03:33.201258 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 591/1047] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:33 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:33.210035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-75a28845-537b-40ef-9256-433db2099f86 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:33.211790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-75a28845-537b-40ef-9256-433db2099f86 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:33.218460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75a28845-537b-40ef-9256-433db2099f86 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:33.219061 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75a28845-537b-40ef-9256-433db2099f86 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:33.219500 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75a28845-537b-40ef-9256-433db2099f86 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:33.220239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75a28845-537b-40ef-9256-433db2099f86 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:33.220677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75a28845-537b-40ef-9256-433db2099f86 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:33.221135 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75a28845-537b-40ef-9256-433db2099f86 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:33.255312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-75a28845-537b-40ef-9256-433db2099f86 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052469 Aug 28 01:03:33.256176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 592/1048] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:33 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:33.261008 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f06605af-38c4-4eab-8e34-81427c833344 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:33.263411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f06605af-38c4-4eab-8e34-81427c833344 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:33.270072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f06605af-38c4-4eab-8e34-81427c833344 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:33.270543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f06605af-38c4-4eab-8e34-81427c833344 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:33.270978 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f06605af-38c4-4eab-8e34-81427c833344 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:33.271694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f06605af-38c4-4eab-8e34-81427c833344 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:33.272127 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f06605af-38c4-4eab-8e34-81427c833344 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:33.272557 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f06605af-38c4-4eab-8e34-81427c833344 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:33.306879 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f06605af-38c4-4eab-8e34-81427c833344 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049723 Aug 28 01:03:33.307808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 593/1049] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:33 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:34.234654 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7fc5e00e-a0b6-44aa-b3ed-79a381be99e2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:34.235405 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7fc5e00e-a0b6-44aa-b3ed-79a381be99e2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:34.242702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fc5e00e-a0b6-44aa-b3ed-79a381be99e2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:34.243225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fc5e00e-a0b6-44aa-b3ed-79a381be99e2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:34.243639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fc5e00e-a0b6-44aa-b3ed-79a381be99e2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:34.244406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fc5e00e-a0b6-44aa-b3ed-79a381be99e2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:34.244870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fc5e00e-a0b6-44aa-b3ed-79a381be99e2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:34.245335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fc5e00e-a0b6-44aa-b3ed-79a381be99e2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:34.316765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7fc5e00e-a0b6-44aa-b3ed-79a381be99e2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:34.325369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7fc5e00e-a0b6-44aa-b3ed-79a381be99e2 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.094861 Aug 28 01:03:34.325966 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 594/1050] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:34 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:03:34.331398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a23b1bc4-5f59-4b7f-91db-d0ef51540de2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:34.333958 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a23b1bc4-5f59-4b7f-91db-d0ef51540de2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:34.340948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a23b1bc4-5f59-4b7f-91db-d0ef51540de2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:34.341474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a23b1bc4-5f59-4b7f-91db-d0ef51540de2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:34.341904 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a23b1bc4-5f59-4b7f-91db-d0ef51540de2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:34.342634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a23b1bc4-5f59-4b7f-91db-d0ef51540de2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:34.343090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a23b1bc4-5f59-4b7f-91db-d0ef51540de2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:34.343544 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a23b1bc4-5f59-4b7f-91db-d0ef51540de2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:34.378530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a23b1bc4-5f59-4b7f-91db-d0ef51540de2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051393 Aug 28 01:03:34.379104 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 595/1051] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:34 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:34.384274 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6ca9a6bc-a28e-4e66-b6fc-6fe62d9bb098 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:34.386579 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6ca9a6bc-a28e-4e66-b6fc-6fe62d9bb098 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:34.388637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:03:34.394706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ca9a6bc-a28e-4e66-b6fc-6fe62d9bb098 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:34.395232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ca9a6bc-a28e-4e66-b6fc-6fe62d9bb098 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:34.395676 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ca9a6bc-a28e-4e66-b6fc-6fe62d9bb098 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:34.396447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ca9a6bc-a28e-4e66-b6fc-6fe62d9bb098 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:34.396905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ca9a6bc-a28e-4e66-b6fc-6fe62d9bb098 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:34.397351 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ca9a6bc-a28e-4e66-b6fc-6fe62d9bb098 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:34.431647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6ca9a6bc-a28e-4e66-b6fc-6fe62d9bb098 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051374 Aug 28 01:03:34.432237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 596/1052] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:34 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:35.348380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0d7ba3c8-136b-400d-8381-019b19bf563b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:35.349285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0d7ba3c8-136b-400d-8381-019b19bf563b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:35.357879 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0d7ba3c8-136b-400d-8381-019b19bf563b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:35.358565 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0d7ba3c8-136b-400d-8381-019b19bf563b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:35.359060 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0d7ba3c8-136b-400d-8381-019b19bf563b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:35.359895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0d7ba3c8-136b-400d-8381-019b19bf563b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:35.360354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0d7ba3c8-136b-400d-8381-019b19bf563b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:35.360799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0d7ba3c8-136b-400d-8381-019b19bf563b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:35.445044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0d7ba3c8-136b-400d-8381-019b19bf563b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:35.454622 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0d7ba3c8-136b-400d-8381-019b19bf563b tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.111031 Aug 28 01:03:35.455252 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 597/1053] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:35 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:35.460625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-85a399db-8d12-4ea5-8ad8-df3296055e6c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:35.462903 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-85a399db-8d12-4ea5-8ad8-df3296055e6c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:35.470280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85a399db-8d12-4ea5-8ad8-df3296055e6c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:35.470777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85a399db-8d12-4ea5-8ad8-df3296055e6c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:35.471212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85a399db-8d12-4ea5-8ad8-df3296055e6c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:35.471939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85a399db-8d12-4ea5-8ad8-df3296055e6c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:35.472391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85a399db-8d12-4ea5-8ad8-df3296055e6c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:35.472819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85a399db-8d12-4ea5-8ad8-df3296055e6c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:35.507410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-85a399db-8d12-4ea5-8ad8-df3296055e6c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050965 Aug 28 01:03:35.508002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 598/1054] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:35 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:35.513151 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-970abf0b-115e-487f-951c-7fb036e59bd4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:35.515447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-970abf0b-115e-487f-951c-7fb036e59bd4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:35.522258 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-970abf0b-115e-487f-951c-7fb036e59bd4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:35.522762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-970abf0b-115e-487f-951c-7fb036e59bd4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:35.523319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-970abf0b-115e-487f-951c-7fb036e59bd4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:35.523974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-970abf0b-115e-487f-951c-7fb036e59bd4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:35.524432 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-970abf0b-115e-487f-951c-7fb036e59bd4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:35.524863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-970abf0b-115e-487f-951c-7fb036e59bd4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:35.558192 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-970abf0b-115e-487f-951c-7fb036e59bd4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049006 Aug 28 01:03:35.558759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 599/1055] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:35 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:36.486517 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-16b4aa22-8066-4c28-8b15-d6aa7027b147 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:36.487436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-16b4aa22-8066-4c28-8b15-d6aa7027b147 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:36.494806 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16b4aa22-8066-4c28-8b15-d6aa7027b147 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:36.495387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16b4aa22-8066-4c28-8b15-d6aa7027b147 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:36.495867 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16b4aa22-8066-4c28-8b15-d6aa7027b147 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:36.496658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16b4aa22-8066-4c28-8b15-d6aa7027b147 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:36.497234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16b4aa22-8066-4c28-8b15-d6aa7027b147 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:36.497871 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16b4aa22-8066-4c28-8b15-d6aa7027b147 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:36.570355 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-16b4aa22-8066-4c28-8b15-d6aa7027b147 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:36.574488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:03:36.583416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-16b4aa22-8066-4c28-8b15-d6aa7027b147 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.100589 Aug 28 01:03:36.584374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 600/1056] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:36 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:36.590350 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-246ce6f8-112e-4f78-93f2-bfbf1300ece2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:36.593091 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-246ce6f8-112e-4f78-93f2-bfbf1300ece2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:36.602493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-246ce6f8-112e-4f78-93f2-bfbf1300ece2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:36.602980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-246ce6f8-112e-4f78-93f2-bfbf1300ece2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:36.603436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-246ce6f8-112e-4f78-93f2-bfbf1300ece2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:36.604138 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-246ce6f8-112e-4f78-93f2-bfbf1300ece2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:36.604595 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-246ce6f8-112e-4f78-93f2-bfbf1300ece2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:36.605085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-246ce6f8-112e-4f78-93f2-bfbf1300ece2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:36.639094 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-246ce6f8-112e-4f78-93f2-bfbf1300ece2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053345 Aug 28 01:03:36.639664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 601/1057] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:36 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:36.644738 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ca8fa573-cbd7-41cb-80ef-1f1491f6fa45 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:36.649748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ca8fa573-cbd7-41cb-80ef-1f1491f6fa45 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:36.657057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ca8fa573-cbd7-41cb-80ef-1f1491f6fa45 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:36.657544 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ca8fa573-cbd7-41cb-80ef-1f1491f6fa45 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:36.657973 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ca8fa573-cbd7-41cb-80ef-1f1491f6fa45 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:36.658691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ca8fa573-cbd7-41cb-80ef-1f1491f6fa45 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:36.659154 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ca8fa573-cbd7-41cb-80ef-1f1491f6fa45 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:36.659571 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ca8fa573-cbd7-41cb-80ef-1f1491f6fa45 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:36.694582 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ca8fa573-cbd7-41cb-80ef-1f1491f6fa45 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053551 Aug 28 01:03:36.695298 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 602/1058] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:36 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:37.608827 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7633df9a-cd6a-41df-ba86-8c4391ad2ea0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:37.609691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7633df9a-cd6a-41df-ba86-8c4391ad2ea0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:37.617614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7633df9a-cd6a-41df-ba86-8c4391ad2ea0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:37.618100 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7633df9a-cd6a-41df-ba86-8c4391ad2ea0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:37.618548 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7633df9a-cd6a-41df-ba86-8c4391ad2ea0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:37.619294 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7633df9a-cd6a-41df-ba86-8c4391ad2ea0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:37.620025 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7633df9a-cd6a-41df-ba86-8c4391ad2ea0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:37.620649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7633df9a-cd6a-41df-ba86-8c4391ad2ea0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:37.694648 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7633df9a-cd6a-41df-ba86-8c4391ad2ea0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:37.704044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7633df9a-cd6a-41df-ba86-8c4391ad2ea0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.101186 Aug 28 01:03:37.704700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 603/1059] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:37 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:37.710174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-98761399-13dd-4ee3-890d-0b2063764700 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:37.712397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-98761399-13dd-4ee3-890d-0b2063764700 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:37.719361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98761399-13dd-4ee3-890d-0b2063764700 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:37.719882 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98761399-13dd-4ee3-890d-0b2063764700 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:37.720330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98761399-13dd-4ee3-890d-0b2063764700 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:37.721138 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98761399-13dd-4ee3-890d-0b2063764700 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:37.721641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98761399-13dd-4ee3-890d-0b2063764700 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:37.722090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98761399-13dd-4ee3-890d-0b2063764700 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:37.759172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-98761399-13dd-4ee3-890d-0b2063764700 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052969 Aug 28 01:03:37.759964 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 604/1060] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:37 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:37.767148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e2c1ca26-3261-4eb6-ac3b-9408e6361be4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:37.770146 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e2c1ca26-3261-4eb6-ac3b-9408e6361be4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:37.778501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2c1ca26-3261-4eb6-ac3b-9408e6361be4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:37.779006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2c1ca26-3261-4eb6-ac3b-9408e6361be4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:37.779484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2c1ca26-3261-4eb6-ac3b-9408e6361be4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:37.780260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2c1ca26-3261-4eb6-ac3b-9408e6361be4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:37.780720 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2c1ca26-3261-4eb6-ac3b-9408e6361be4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:37.781345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2c1ca26-3261-4eb6-ac3b-9408e6361be4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:37.821602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e2c1ca26-3261-4eb6-ac3b-9408e6361be4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060103 Aug 28 01:03:37.822352 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 605/1061] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:37 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:38.737219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7c63d5dc-44c1-4240-8148-4e767e852c73 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:38.737969 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7c63d5dc-44c1-4240-8148-4e767e852c73 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:38.747191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c63d5dc-44c1-4240-8148-4e767e852c73 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:38.748032 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c63d5dc-44c1-4240-8148-4e767e852c73 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:38.748770 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c63d5dc-44c1-4240-8148-4e767e852c73 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:38.750201 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c63d5dc-44c1-4240-8148-4e767e852c73 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:38.751010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c63d5dc-44c1-4240-8148-4e767e852c73 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:38.751809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c63d5dc-44c1-4240-8148-4e767e852c73 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:38.840737 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7c63d5dc-44c1-4240-8148-4e767e852c73 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:38.852734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7c63d5dc-44c1-4240-8148-4e767e852c73 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.119366 Aug 28 01:03:38.853734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 606/1062] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:38 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:38.862251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f76d6503-434b-43ff-a941-4a4b6f8fbae3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:38.865738 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f76d6503-434b-43ff-a941-4a4b6f8fbae3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:38.876662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f76d6503-434b-43ff-a941-4a4b6f8fbae3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:38.877537 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f76d6503-434b-43ff-a941-4a4b6f8fbae3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:38.878281 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f76d6503-434b-43ff-a941-4a4b6f8fbae3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:38.879494 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f76d6503-434b-43ff-a941-4a4b6f8fbae3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:38.880227 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f76d6503-434b-43ff-a941-4a4b6f8fbae3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:38.880992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f76d6503-434b-43ff-a941-4a4b6f8fbae3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:38.919011 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f76d6503-434b-43ff-a941-4a4b6f8fbae3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.063897 Aug 28 01:03:38.919586 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 607/1063] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:38 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:38.924848 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fc4a5e16-9125-4688-8023-9f02c53ba80a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:38.927128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fc4a5e16-9125-4688-8023-9f02c53ba80a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:38.934352 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc4a5e16-9125-4688-8023-9f02c53ba80a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:38.934998 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc4a5e16-9125-4688-8023-9f02c53ba80a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:38.935543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc4a5e16-9125-4688-8023-9f02c53ba80a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:38.936435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc4a5e16-9125-4688-8023-9f02c53ba80a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:38.937094 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc4a5e16-9125-4688-8023-9f02c53ba80a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:38.937581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc4a5e16-9125-4688-8023-9f02c53ba80a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:38.973683 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fc4a5e16-9125-4688-8023-9f02c53ba80a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052778 Aug 28 01:03:38.974182 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 608/1064] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:38 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:39.874061 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4a3e6959-bd96-4a06-8e2f-9e2f0f00457c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:39.874847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4a3e6959-bd96-4a06-8e2f-9e2f0f00457c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:39.882119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a3e6959-bd96-4a06-8e2f-9e2f0f00457c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:39.882616 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a3e6959-bd96-4a06-8e2f-9e2f0f00457c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:39.883051 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a3e6959-bd96-4a06-8e2f-9e2f0f00457c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:39.883785 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a3e6959-bd96-4a06-8e2f-9e2f0f00457c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:39.884242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a3e6959-bd96-4a06-8e2f-9e2f0f00457c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:39.884670 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a3e6959-bd96-4a06-8e2f-9e2f0f00457c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:39.955537 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4a3e6959-bd96-4a06-8e2f-9e2f0f00457c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:39.963819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4a3e6959-bd96-4a06-8e2f-9e2f0f00457c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.093644 Aug 28 01:03:39.964413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 609/1065] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:39 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:39.969650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a87d7c5f-31be-4de1-a21b-67581c42296a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:39.971900 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a87d7c5f-31be-4de1-a21b-67581c42296a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:39.978912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a87d7c5f-31be-4de1-a21b-67581c42296a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:39.979496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a87d7c5f-31be-4de1-a21b-67581c42296a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:39.980093 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a87d7c5f-31be-4de1-a21b-67581c42296a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:39.981309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a87d7c5f-31be-4de1-a21b-67581c42296a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:39.981526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a87d7c5f-31be-4de1-a21b-67581c42296a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:39.982014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a87d7c5f-31be-4de1-a21b-67581c42296a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:40.017135 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a87d7c5f-31be-4de1-a21b-67581c42296a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051566 Aug 28 01:03:40.017741 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 610/1066] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:39 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:40.023439 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-284895c8-d6f7-4acc-ac27-3b4211b2c8b4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:40.027301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-284895c8-d6f7-4acc-ac27-3b4211b2c8b4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:40.038787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-284895c8-d6f7-4acc-ac27-3b4211b2c8b4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:40.040135 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-284895c8-d6f7-4acc-ac27-3b4211b2c8b4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:40.040135 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-284895c8-d6f7-4acc-ac27-3b4211b2c8b4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:40.041042 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-284895c8-d6f7-4acc-ac27-3b4211b2c8b4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:40.041511 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-284895c8-d6f7-4acc-ac27-3b4211b2c8b4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:40.041936 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-284895c8-d6f7-4acc-ac27-3b4211b2c8b4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:40.097743 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-284895c8-d6f7-4acc-ac27-3b4211b2c8b4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.078449 Aug 28 01:03:40.098773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 611/1067] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:40 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:40.987208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-dc93e7d4-0cb7-4735-a8bf-758585925598 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:40.988114 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-dc93e7d4-0cb7-4735-a8bf-758585925598 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:40.995656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc93e7d4-0cb7-4735-a8bf-758585925598 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:40.996145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc93e7d4-0cb7-4735-a8bf-758585925598 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:40.996630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc93e7d4-0cb7-4735-a8bf-758585925598 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:40.997392 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc93e7d4-0cb7-4735-a8bf-758585925598 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:40.997858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc93e7d4-0cb7-4735-a8bf-758585925598 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:40.998287 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc93e7d4-0cb7-4735-a8bf-758585925598 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:41.066736 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dc93e7d4-0cb7-4735-a8bf-758585925598 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:41.076520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-dc93e7d4-0cb7-4735-a8bf-758585925598 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.093531 Aug 28 01:03:41.077346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 612/1068] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:40 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:03:41.082918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fe357e3e-d29d-4b1f-9bb1-eff8d3599178 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:41.085412 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fe357e3e-d29d-4b1f-9bb1-eff8d3599178 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:41.092561 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe357e3e-d29d-4b1f-9bb1-eff8d3599178 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:41.093110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe357e3e-d29d-4b1f-9bb1-eff8d3599178 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:41.093554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe357e3e-d29d-4b1f-9bb1-eff8d3599178 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:41.094270 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe357e3e-d29d-4b1f-9bb1-eff8d3599178 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:41.094723 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe357e3e-d29d-4b1f-9bb1-eff8d3599178 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:41.095152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe357e3e-d29d-4b1f-9bb1-eff8d3599178 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:41.131561 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fe357e3e-d29d-4b1f-9bb1-eff8d3599178 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052885 Aug 28 01:03:41.132420 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 613/1069] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:41 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:41.137955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a2a3a0ed-7826-41d5-916b-133d263b015b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:41.140422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a2a3a0ed-7826-41d5-916b-133d263b015b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:41.147283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2a3a0ed-7826-41d5-916b-133d263b015b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:41.147767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2a3a0ed-7826-41d5-916b-133d263b015b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:41.148194 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2a3a0ed-7826-41d5-916b-133d263b015b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:41.148952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2a3a0ed-7826-41d5-916b-133d263b015b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:41.149447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2a3a0ed-7826-41d5-916b-133d263b015b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:41.149863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2a3a0ed-7826-41d5-916b-133d263b015b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:41.185758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a2a3a0ed-7826-41d5-916b-133d263b015b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052134 Aug 28 01:03:41.186358 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 614/1070] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:03:41 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:03:42.099450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-328b1be7-abdc-4352-8a6c-8f68dea4fe20 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:42.100450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-328b1be7-abdc-4352-8a6c-8f68dea4fe20 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:42.107606 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-328b1be7-abdc-4352-8a6c-8f68dea4fe20 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:42.108244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-328b1be7-abdc-4352-8a6c-8f68dea4fe20 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:42.108938 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-328b1be7-abdc-4352-8a6c-8f68dea4fe20 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:42.109823 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-328b1be7-abdc-4352-8a6c-8f68dea4fe20 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:42.110348 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-328b1be7-abdc-4352-8a6c-8f68dea4fe20 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:42.110782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-328b1be7-abdc-4352-8a6c-8f68dea4fe20 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:42.187695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-328b1be7-abdc-4352-8a6c-8f68dea4fe20 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:03:42.201834 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-328b1be7-abdc-4352-8a6c-8f68dea4fe20 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.105838 Aug 28 01:03:42.202901 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 615/1071] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:03:42 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:03:42.212299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6bb9a85e-062b-42ca-8367-ba432817ddc1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:42.216322 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6bb9a85e-062b-42ca-8367-ba432817ddc1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:42.227952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bb9a85e-062b-42ca-8367-ba432817ddc1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:42.228831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bb9a85e-062b-42ca-8367-ba432817ddc1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:42.229747 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bb9a85e-062b-42ca-8367-ba432817ddc1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:42.231067 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bb9a85e-062b-42ca-8367-ba432817ddc1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:42.231849 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bb9a85e-062b-42ca-8367-ba432817ddc1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:42.232617 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bb9a85e-062b-42ca-8367-ba432817ddc1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:42.291907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6bb9a85e-062b-42ca-8367-ba432817ddc1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.087017 Aug 28 01:03:42.292940 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 616/1072] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:03:42 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:03:42.302113 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8cad3f8e-8414-4697-9561-e81a8e19a626 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:03:42.303006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8cad3f8e-8414-4697-9561-e81a8e19a626 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:03:42.314298 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8cad3f8e-8414-4697-9561-e81a8e19a626 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:42.315174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8cad3f8e-8414-4697-9561-e81a8e19a626 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:42.315921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8cad3f8e-8414-4697-9561-e81a8e19a626 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:42.317633 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8cad3f8e-8414-4697-9561-e81a8e19a626 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:03:42.318411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8cad3f8e-8414-4697-9561-e81a8e19a626 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:03:42.319164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8cad3f8e-8414-4697-9561-e81a8e19a626 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:03:42.354489 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:03:52.309263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=43 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:04:42.185861 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-49a755e6-f11f-465d-a9e5-f40e97e328bb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c" status: 202 len: 0 microversion: 2.1 time: 120.097550 Aug 28 01:04:42.187232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 457/1073] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Aug 28 01:02:42 2022] DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c => generated 0 bytes in 120100 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 01:04:42.199298 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c979f0d4-20c7-4592-ac58-ab0f50690324 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:04:42.203395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c979f0d4-20c7-4592-ac58-ab0f50690324 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:04:42.216374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c979f0d4-20c7-4592-ac58-ab0f50690324 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:04:42.217297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c979f0d4-20c7-4592-ac58-ab0f50690324 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:04:42.218088 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c979f0d4-20c7-4592-ac58-ab0f50690324 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:04:42.219394 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c979f0d4-20c7-4592-ac58-ab0f50690324 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:04:42.220805 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c979f0d4-20c7-4592-ac58-ab0f50690324 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:04:42.222126 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c979f0d4-20c7-4592-ac58-ab0f50690324 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:04:42.279641 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c979f0d4-20c7-4592-ac58-ab0f50690324 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.090280 Aug 28 01:04:42.280733 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 458/1074] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:04:42 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:04:42.286329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2088c7f0-045e-4e43-88b8-60f13251e25a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:04:42.286962 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2088c7f0-045e-4e43-88b8-60f13251e25a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:04:42.293572 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2088c7f0-045e-4e43-88b8-60f13251e25a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:04:42.294111 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2088c7f0-045e-4e43-88b8-60f13251e25a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:04:42.294548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2088c7f0-045e-4e43-88b8-60f13251e25a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:04:42.295413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2088c7f0-045e-4e43-88b8-60f13251e25a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:04:42.295735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2088c7f0-045e-4e43-88b8-60f13251e25a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:04:42.296158 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2088c7f0-045e-4e43-88b8-60f13251e25a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:04:42.375905 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2088c7f0-045e-4e43-88b8-60f13251e25a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:04:42.390170 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2088c7f0-045e-4e43-88b8-60f13251e25a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.108026 Aug 28 01:04:42.391224 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 459/1075] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:04:42 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:04:42.400473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-78e238c0-8196-49ae-9bee-c77a3887bc29 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:04:42.405551 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-78e238c0-8196-49ae-9bee-c77a3887bc29 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:04:42.416444 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78e238c0-8196-49ae-9bee-c77a3887bc29 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:04:42.417323 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78e238c0-8196-49ae-9bee-c77a3887bc29 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:04:42.417649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.objects.instance [None req-8cad3f8e-8414-4697-9561-e81a8e19a626 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lazy-loading 'services' on Instance uuid 03969ac5-934e-4201-8087-0927eae2bbd2 {{(pid=120525) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:04:42.418370 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78e238c0-8196-49ae-9bee-c77a3887bc29 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:04:42.419356 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78e238c0-8196-49ae-9bee-c77a3887bc29 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:04:42.420183 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78e238c0-8196-49ae-9bee-c77a3887bc29 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:04:42.420920 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78e238c0-8196-49ae-9bee-c77a3887bc29 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:04:42.482322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-78e238c0-8196-49ae-9bee-c77a3887bc29 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.088844 Aug 28 01:04:42.483698 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 460/1076] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:04:42 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:04:42.492671 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8523ed49-d319-4b1a-8734-517714a9d405 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:04:42.493557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8523ed49-d319-4b1a-8734-517714a9d405 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:04:42.504520 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8523ed49-d319-4b1a-8734-517714a9d405 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:04:42.505429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8523ed49-d319-4b1a-8734-517714a9d405 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:04:42.506191 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8523ed49-d319-4b1a-8734-517714a9d405 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:04:42.507442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8523ed49-d319-4b1a-8734-517714a9d405 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:04:42.508312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8523ed49-d319-4b1a-8734-517714a9d405 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:04:42.509181 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8523ed49-d319-4b1a-8734-517714a9d405 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:04:52.424426 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=4 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:04:52.499614 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=8 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:04:52.536515 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=23,nova_cell1:INSERT=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:05:14.031228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-8cad3f8e-8414-4697-9561-e81a8e19a626 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d0ac8c36-ae02-4ee8-8daa-57369a30360b) Aug 28 01:05:14.036186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8cad3f8e-8414-4697-9561-e81a8e19a626 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Request method failure captured: Aug 28 01:05:14.036186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c HTTP/1.1 Aug 28 01:05:14.036186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:05:14.036186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:05:14.036186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:05:14.036186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:05:14.036186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:05:14.036186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:05:14.036186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:05:14.036186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:05:14.036186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCrzWcuiPK_0ynCrragXxXd_fRw7lHCmbNVMAqr4NhyopxGAZIiQrt0m3y3NawnXLWqyOVM4YbBc-yLRrJ4cXICU0MsT01bsu6VzxaCxGd6JRCChv8LlCV7kFwwyy7Ffwd9JpYuFLguuT-xhEcVlfMIZxgSYB22XTBbtvAeHM_AtK1Xg Aug 28 01:05:14.036186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:05:14.036186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:05:14.036186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:05:14.036186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:05:14.036186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:05:14.036186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:05:14.036186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 01:05:14.036186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:05:14.036186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:05:14.036186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:05:14.039225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_6f3acee4af62483ab1f2cbf3a47b09e1", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/6f3acee4af62483ab1f2cbf3a47b09e1"}]}] Aug 28 01:05:14.039225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:05:14.039225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 01:05:14.039225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:05:14.039225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 01:05:14.039225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:05:14.039225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:05:14.039225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: 1ab2fcb3759045c7a076e5c8222a421d Aug 28 01:05:14.039225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 01:05:14.039225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 01:05:14.040713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d0ac8c36-ae02-4ee8-8daa-57369a30360b) Aug 28 01:05:14.040713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'server_id': '03969ac5-934e-4201-8087-0927eae2bbd2', 'id': '6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c'} Aug 28 01:05:14.040713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:05:14.040713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:05:14.040713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:05:14.040713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:05:14.040713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:05:14.040713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:05:14.040713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:05:14.040713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:05:14.040713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:05:14.040713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:05:14.040713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:05:14.040713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:05:14.040713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:05:14.040713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:05:14.040713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:05:14.043157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:05:14.043157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:05:14.043157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:05:14.043157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d0ac8c36-ae02-4ee8-8daa-57369a30360b) Aug 28 01:05:14.043157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:05:14.043157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:05:14.043157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:05:14.043157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:05:14.043157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5192, in _detach_volume Aug 28 01:05:14.043157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 28 01:05:14.043157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 28 01:05:14.043157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 28 01:05:14.043157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 28 01:05:14.043157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 28 01:05:14.043157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:05:14.043157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:05:14.043157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:05:14.043157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:05:14.043157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:05:14.045716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:05:14.045716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:05:14.045716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:05:14.045716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:05:14.045716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:05:14.045716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:05:14.045716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:05:14.045716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:05:14.045716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:05:14.045716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:05:14.045716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:05:14.045716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d0ac8c36-ae02-4ee8-8daa-57369a30360b) Aug 28 01:05:14.045716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:05:14.045716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:05:14.045716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:05:14.045716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:05:14.045716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 28 01:05:14.047948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 28 01:05:14.047948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Aug 28 01:05:14.047948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:05:14.047948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Aug 28 01:05:14.047948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 28 01:05:14.047948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5214, in detach_volume Aug 28 01:05:14.047948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 28 01:05:14.047948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Aug 28 01:05:14.047948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:05:14.047948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Aug 28 01:05:14.047948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 28 01:05:14.047948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d0ac8c36-ae02-4ee8-8daa-57369a30360b) Aug 28 01:05:14.047948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:05:14.047948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:05:14.047948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:05:14.047948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:05:14.047948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:05:14.047948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:05:14.053695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:05:14.053695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:05:14.053695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:05:14.053695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:05:14.053695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 28 01:05:14.053695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 28 01:05:14.053695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d0ac8c36-ae02-4ee8-8daa-57369a30360b) Aug 28 01:05:14.053695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:05:14.053695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:05:14.053695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:05:14.053695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:05:14.053695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:05:14.053695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:05:14.053695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:05:14.053695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:05:14.053695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d0ac8c36-ae02-4ee8-8daa-57369a30360b) Aug 28 01:05:14.053695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:05:14.056145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8cad3f8e-8414-4697-9561-e81a8e19a626 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d0ac8c36-ae02-4ee8-8daa-57369a30360b) {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:05:14.056145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8cad3f8e-8414-4697-9561-e81a8e19a626 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c" status: 400 len: 269 microversion: 2.1 time: 91.744061 Aug 28 01:05:14.056145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 617/1077] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Aug 28 01:03:42 2022] DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c => generated 269 bytes in 91746 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 28 01:05:14.056145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4c6797b7-b2a4-4815-ba7d-16041f899729 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:14.057030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4c6797b7-b2a4-4815-ba7d-16041f899729 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:14.062195 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c6797b7-b2a4-4815-ba7d-16041f899729 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:14.062816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c6797b7-b2a4-4815-ba7d-16041f899729 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:14.063326 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c6797b7-b2a4-4815-ba7d-16041f899729 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:14.064557 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c6797b7-b2a4-4815-ba7d-16041f899729 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:14.065501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c6797b7-b2a4-4815-ba7d-16041f899729 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:14.066355 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c6797b7-b2a4-4815-ba7d-16041f899729 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:14.107501 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.objects.instance [None req-8523ed49-d319-4b1a-8734-517714a9d405 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lazy-loading 'services' on Instance uuid 03969ac5-934e-4201-8087-0927eae2bbd2 {{(pid=120524) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:05:14.111066 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4c6797b7-b2a4-4815-ba7d-16041f899729 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.069638 Aug 28 01:05:14.111948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 618/1078] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:14 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:05:14.122312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-163e8691-03f4-43bf-b469-74066e784d38 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:14.123335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-163e8691-03f4-43bf-b469-74066e784d38 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:14.133832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-163e8691-03f4-43bf-b469-74066e784d38 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:14.134589 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-163e8691-03f4-43bf-b469-74066e784d38 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:14.135866 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-163e8691-03f4-43bf-b469-74066e784d38 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:14.137024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-163e8691-03f4-43bf-b469-74066e784d38 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:14.137777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-163e8691-03f4-43bf-b469-74066e784d38 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:14.138409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-163e8691-03f4-43bf-b469-74066e784d38 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:14.223250 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-163e8691-03f4-43bf-b469-74066e784d38 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:14.237725 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-163e8691-03f4-43bf-b469-74066e784d38 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.124061 Aug 28 01:05:14.238929 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 619/1079] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:14 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:05:14.248009 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d66d04e6-9d99-4550-9f43-2de9893f3c1c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:14.251569 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d66d04e6-9d99-4550-9f43-2de9893f3c1c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:14.261063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d66d04e6-9d99-4550-9f43-2de9893f3c1c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:14.261879 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d66d04e6-9d99-4550-9f43-2de9893f3c1c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:14.262550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d66d04e6-9d99-4550-9f43-2de9893f3c1c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:14.263726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d66d04e6-9d99-4550-9f43-2de9893f3c1c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:14.264799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d66d04e6-9d99-4550-9f43-2de9893f3c1c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:14.265512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d66d04e6-9d99-4550-9f43-2de9893f3c1c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:14.305525 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d66d04e6-9d99-4550-9f43-2de9893f3c1c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.064736 Aug 28 01:05:14.306257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 620/1080] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:14 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:15.124239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-406df695-562f-47ad-84fa-2bf35a78e62a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:15.126584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-406df695-562f-47ad-84fa-2bf35a78e62a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:15.133755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-406df695-562f-47ad-84fa-2bf35a78e62a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:15.134321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-406df695-562f-47ad-84fa-2bf35a78e62a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:15.134701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-406df695-562f-47ad-84fa-2bf35a78e62a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:15.135446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-406df695-562f-47ad-84fa-2bf35a78e62a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:15.135879 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-406df695-562f-47ad-84fa-2bf35a78e62a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:15.136308 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-406df695-562f-47ad-84fa-2bf35a78e62a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:15.172635 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-406df695-562f-47ad-84fa-2bf35a78e62a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051865 Aug 28 01:05:15.173318 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 621/1081] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:15 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:05:15.260431 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-43b015d1-4c4e-4488-aca0-8db2a6fd66a0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:15.261171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-43b015d1-4c4e-4488-aca0-8db2a6fd66a0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:15.268358 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-43b015d1-4c4e-4488-aca0-8db2a6fd66a0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:15.268856 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-43b015d1-4c4e-4488-aca0-8db2a6fd66a0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:15.269327 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-43b015d1-4c4e-4488-aca0-8db2a6fd66a0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:15.270066 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-43b015d1-4c4e-4488-aca0-8db2a6fd66a0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:15.270518 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-43b015d1-4c4e-4488-aca0-8db2a6fd66a0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:15.270945 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-43b015d1-4c4e-4488-aca0-8db2a6fd66a0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:15.339441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-43b015d1-4c4e-4488-aca0-8db2a6fd66a0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:15.348679 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-43b015d1-4c4e-4488-aca0-8db2a6fd66a0 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.092257 Aug 28 01:05:15.349334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 622/1082] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:15 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:05:15.354436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8a2cb4fb-fb23-4467-89a0-f2fd4e8fb614 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:15.356677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8a2cb4fb-fb23-4467-89a0-f2fd4e8fb614 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:15.363680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8a2cb4fb-fb23-4467-89a0-f2fd4e8fb614 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:15.364163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8a2cb4fb-fb23-4467-89a0-f2fd4e8fb614 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:15.364607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8a2cb4fb-fb23-4467-89a0-f2fd4e8fb614 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:15.365340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8a2cb4fb-fb23-4467-89a0-f2fd4e8fb614 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:15.365815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8a2cb4fb-fb23-4467-89a0-f2fd4e8fb614 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:15.366234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8a2cb4fb-fb23-4467-89a0-f2fd4e8fb614 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:15.399840 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8a2cb4fb-fb23-4467-89a0-f2fd4e8fb614 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049378 Aug 28 01:05:15.400435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 623/1083] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:15 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:16.187187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-286f9ddb-8e62-44f3-a33d-4f0c41687a05 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:16.189705 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-286f9ddb-8e62-44f3-a33d-4f0c41687a05 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:16.197469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-286f9ddb-8e62-44f3-a33d-4f0c41687a05 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:16.197958 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-286f9ddb-8e62-44f3-a33d-4f0c41687a05 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:16.198383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-286f9ddb-8e62-44f3-a33d-4f0c41687a05 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:16.199092 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-286f9ddb-8e62-44f3-a33d-4f0c41687a05 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:16.199534 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-286f9ddb-8e62-44f3-a33d-4f0c41687a05 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:16.199951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-286f9ddb-8e62-44f3-a33d-4f0c41687a05 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:16.234562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-286f9ddb-8e62-44f3-a33d-4f0c41687a05 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052716 Aug 28 01:05:16.235204 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 624/1084] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:16 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:05:16.380602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-dc7d31c9-4ee6-49a0-9c0c-0badfcac1691 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:16.381275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-dc7d31c9-4ee6-49a0-9c0c-0badfcac1691 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:16.388268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc7d31c9-4ee6-49a0-9c0c-0badfcac1691 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:16.388782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc7d31c9-4ee6-49a0-9c0c-0badfcac1691 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:16.389232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc7d31c9-4ee6-49a0-9c0c-0badfcac1691 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:16.389956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc7d31c9-4ee6-49a0-9c0c-0badfcac1691 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:16.390428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc7d31c9-4ee6-49a0-9c0c-0badfcac1691 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:16.390847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc7d31c9-4ee6-49a0-9c0c-0badfcac1691 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:16.460823 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dc7d31c9-4ee6-49a0-9c0c-0badfcac1691 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:16.470237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-dc7d31c9-4ee6-49a0-9c0c-0badfcac1691 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.093646 Aug 28 01:05:16.470976 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 625/1085] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:16 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:05:16.476183 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6b8cc693-1ff9-499e-8d7e-daa75bbafa0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:16.478537 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6b8cc693-1ff9-499e-8d7e-daa75bbafa0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:16.485014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b8cc693-1ff9-499e-8d7e-daa75bbafa0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:16.485484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b8cc693-1ff9-499e-8d7e-daa75bbafa0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:16.485909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b8cc693-1ff9-499e-8d7e-daa75bbafa0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:16.486607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b8cc693-1ff9-499e-8d7e-daa75bbafa0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:16.487047 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b8cc693-1ff9-499e-8d7e-daa75bbafa0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:16.487438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b8cc693-1ff9-499e-8d7e-daa75bbafa0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:16.520672 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6b8cc693-1ff9-499e-8d7e-daa75bbafa0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048499 Aug 28 01:05:16.521282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 626/1086] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:16 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:17.249464 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1f42dda9-d6a8-457f-ab5d-3a5410c51dc9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:17.251685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1f42dda9-d6a8-457f-ab5d-3a5410c51dc9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:17.258968 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1f42dda9-d6a8-457f-ab5d-3a5410c51dc9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:17.259471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1f42dda9-d6a8-457f-ab5d-3a5410c51dc9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:17.259910 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1f42dda9-d6a8-457f-ab5d-3a5410c51dc9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:17.260658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1f42dda9-d6a8-457f-ab5d-3a5410c51dc9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:17.261160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1f42dda9-d6a8-457f-ab5d-3a5410c51dc9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:17.261656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1f42dda9-d6a8-457f-ab5d-3a5410c51dc9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:17.295456 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1f42dda9-d6a8-457f-ab5d-3a5410c51dc9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049980 Aug 28 01:05:17.296033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 627/1087] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:17 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:05:17.491278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4f88d846-9e12-42f4-9a4d-d246f9433523 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:17.492076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4f88d846-9e12-42f4-9a4d-d246f9433523 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:17.498910 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f88d846-9e12-42f4-9a4d-d246f9433523 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:17.499401 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f88d846-9e12-42f4-9a4d-d246f9433523 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:17.499834 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f88d846-9e12-42f4-9a4d-d246f9433523 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:17.500535 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f88d846-9e12-42f4-9a4d-d246f9433523 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:17.500992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f88d846-9e12-42f4-9a4d-d246f9433523 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:17.501434 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f88d846-9e12-42f4-9a4d-d246f9433523 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:17.567256 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4f88d846-9e12-42f4-9a4d-d246f9433523 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:17.575484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4f88d846-9e12-42f4-9a4d-d246f9433523 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.088130 Aug 28 01:05:17.576030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 628/1088] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:17 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:05:17.581002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fc80d6f9-02d5-4862-bb20-952247cf412f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:17.583433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fc80d6f9-02d5-4862-bb20-952247cf412f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:17.590253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc80d6f9-02d5-4862-bb20-952247cf412f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:17.590726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc80d6f9-02d5-4862-bb20-952247cf412f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:17.591181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc80d6f9-02d5-4862-bb20-952247cf412f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:17.591903 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc80d6f9-02d5-4862-bb20-952247cf412f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:17.592359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc80d6f9-02d5-4862-bb20-952247cf412f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:17.592786 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc80d6f9-02d5-4862-bb20-952247cf412f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:17.626273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fc80d6f9-02d5-4862-bb20-952247cf412f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049117 Aug 28 01:05:17.626849 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 629/1089] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:17 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:18.308520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-683b4125-9ca4-48bd-8704-4572c533a56f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:18.310726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-683b4125-9ca4-48bd-8704-4572c533a56f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:18.317917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-683b4125-9ca4-48bd-8704-4572c533a56f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:18.318424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-683b4125-9ca4-48bd-8704-4572c533a56f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:18.318844 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-683b4125-9ca4-48bd-8704-4572c533a56f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:18.319554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-683b4125-9ca4-48bd-8704-4572c533a56f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:18.320013 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-683b4125-9ca4-48bd-8704-4572c533a56f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:18.320436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-683b4125-9ca4-48bd-8704-4572c533a56f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:18.354059 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-683b4125-9ca4-48bd-8704-4572c533a56f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049658 Aug 28 01:05:18.354649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 630/1090] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:18 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:05:18.596072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8444a2c3-f0ff-4c01-aa31-a1dd7a57a79c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:18.596587 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8444a2c3-f0ff-4c01-aa31-a1dd7a57a79c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:18.603439 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8444a2c3-f0ff-4c01-aa31-a1dd7a57a79c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:18.603973 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8444a2c3-f0ff-4c01-aa31-a1dd7a57a79c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:18.604531 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8444a2c3-f0ff-4c01-aa31-a1dd7a57a79c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:18.605442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8444a2c3-f0ff-4c01-aa31-a1dd7a57a79c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:18.605886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8444a2c3-f0ff-4c01-aa31-a1dd7a57a79c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:18.606304 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8444a2c3-f0ff-4c01-aa31-a1dd7a57a79c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:18.672907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8444a2c3-f0ff-4c01-aa31-a1dd7a57a79c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:18.681760 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8444a2c3-f0ff-4c01-aa31-a1dd7a57a79c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.089563 Aug 28 01:05:18.682347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 631/1091] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:18 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:05:18.687652 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e0b8c9c0-d5af-4ca6-81c9-6f05613ed4b2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:18.690017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e0b8c9c0-d5af-4ca6-81c9-6f05613ed4b2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:18.696892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0b8c9c0-d5af-4ca6-81c9-6f05613ed4b2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:18.697412 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0b8c9c0-d5af-4ca6-81c9-6f05613ed4b2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:18.697825 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0b8c9c0-d5af-4ca6-81c9-6f05613ed4b2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:18.698557 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0b8c9c0-d5af-4ca6-81c9-6f05613ed4b2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:18.699024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0b8c9c0-d5af-4ca6-81c9-6f05613ed4b2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:18.699479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0b8c9c0-d5af-4ca6-81c9-6f05613ed4b2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:18.732296 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e0b8c9c0-d5af-4ca6-81c9-6f05613ed4b2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048851 Aug 28 01:05:18.732870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 632/1092] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:18 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:19.367453 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-223bb128-adb3-4d9c-a160-8103e621e401 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:19.369735 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-223bb128-adb3-4d9c-a160-8103e621e401 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:19.377526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-223bb128-adb3-4d9c-a160-8103e621e401 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:19.377994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-223bb128-adb3-4d9c-a160-8103e621e401 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:19.378427 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-223bb128-adb3-4d9c-a160-8103e621e401 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:19.379160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-223bb128-adb3-4d9c-a160-8103e621e401 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:19.379597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-223bb128-adb3-4d9c-a160-8103e621e401 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:19.380022 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-223bb128-adb3-4d9c-a160-8103e621e401 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:19.413336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-223bb128-adb3-4d9c-a160-8103e621e401 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049881 Aug 28 01:05:19.413922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 633/1093] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:19 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:05:19.704126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c8a7902c-0619-4505-9cd9-2fd5e1e523d8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:19.704791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c8a7902c-0619-4505-9cd9-2fd5e1e523d8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:19.712438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c8a7902c-0619-4505-9cd9-2fd5e1e523d8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:19.712919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c8a7902c-0619-4505-9cd9-2fd5e1e523d8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:19.713375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c8a7902c-0619-4505-9cd9-2fd5e1e523d8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:19.714081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c8a7902c-0619-4505-9cd9-2fd5e1e523d8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:19.714523 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c8a7902c-0619-4505-9cd9-2fd5e1e523d8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:19.715052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c8a7902c-0619-4505-9cd9-2fd5e1e523d8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:19.735102 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:05:19.783382 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c8a7902c-0619-4505-9cd9-2fd5e1e523d8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:19.792023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c8a7902c-0619-4505-9cd9-2fd5e1e523d8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.092139 Aug 28 01:05:19.792626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 634/1094] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:19 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:05:19.797787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5a096c98-3236-49fb-b89b-b18ee9126e06 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:19.800032 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5a096c98-3236-49fb-b89b-b18ee9126e06 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:19.807055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5a096c98-3236-49fb-b89b-b18ee9126e06 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:19.807627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5a096c98-3236-49fb-b89b-b18ee9126e06 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:19.808069 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5a096c98-3236-49fb-b89b-b18ee9126e06 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:19.808931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5a096c98-3236-49fb-b89b-b18ee9126e06 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:19.809418 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5a096c98-3236-49fb-b89b-b18ee9126e06 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:19.809840 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5a096c98-3236-49fb-b89b-b18ee9126e06 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:19.842710 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5a096c98-3236-49fb-b89b-b18ee9126e06 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048923 Aug 28 01:05:19.843295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 635/1095] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:19 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:20.426281 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b19df2eb-d9d8-4677-a8dd-1e192617a213 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:20.428645 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b19df2eb-d9d8-4677-a8dd-1e192617a213 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:20.435466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b19df2eb-d9d8-4677-a8dd-1e192617a213 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:20.435958 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b19df2eb-d9d8-4677-a8dd-1e192617a213 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:20.436379 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b19df2eb-d9d8-4677-a8dd-1e192617a213 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:20.437166 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b19df2eb-d9d8-4677-a8dd-1e192617a213 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:20.437618 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b19df2eb-d9d8-4677-a8dd-1e192617a213 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:20.438053 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b19df2eb-d9d8-4677-a8dd-1e192617a213 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:20.473590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b19df2eb-d9d8-4677-a8dd-1e192617a213 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051213 Aug 28 01:05:20.474387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 636/1096] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:20 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:05:20.816570 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0a9199fc-8395-4058-b66f-84d82b22a764 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:20.817224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0a9199fc-8395-4058-b66f-84d82b22a764 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:20.824994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a9199fc-8395-4058-b66f-84d82b22a764 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:20.825503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a9199fc-8395-4058-b66f-84d82b22a764 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:20.825929 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a9199fc-8395-4058-b66f-84d82b22a764 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:20.826638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a9199fc-8395-4058-b66f-84d82b22a764 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:20.827084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a9199fc-8395-4058-b66f-84d82b22a764 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:20.827497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a9199fc-8395-4058-b66f-84d82b22a764 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:20.904496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0a9199fc-8395-4058-b66f-84d82b22a764 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:20.920856 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0a9199fc-8395-4058-b66f-84d82b22a764 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.108765 Aug 28 01:05:20.922894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 637/1097] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:20 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:05:20.933079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f1c43057-8b8a-4b4d-9814-f276ffc038fd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:20.937371 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f1c43057-8b8a-4b4d-9814-f276ffc038fd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:20.952470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f1c43057-8b8a-4b4d-9814-f276ffc038fd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:20.953678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f1c43057-8b8a-4b4d-9814-f276ffc038fd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:20.954443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f1c43057-8b8a-4b4d-9814-f276ffc038fd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:20.955552 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f1c43057-8b8a-4b4d-9814-f276ffc038fd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:20.956941 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f1c43057-8b8a-4b4d-9814-f276ffc038fd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:20.957268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f1c43057-8b8a-4b4d-9814-f276ffc038fd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:21.008166 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f1c43057-8b8a-4b4d-9814-f276ffc038fd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.083666 Aug 28 01:05:21.009140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 638/1098] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:20 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:21.486894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3bf3d580-2a16-4eba-95f5-0e63cbb6e929 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:21.489127 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3bf3d580-2a16-4eba-95f5-0e63cbb6e929 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:21.495481 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bf3d580-2a16-4eba-95f5-0e63cbb6e929 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:21.496110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bf3d580-2a16-4eba-95f5-0e63cbb6e929 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:21.496547 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bf3d580-2a16-4eba-95f5-0e63cbb6e929 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:21.497277 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bf3d580-2a16-4eba-95f5-0e63cbb6e929 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:21.497727 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bf3d580-2a16-4eba-95f5-0e63cbb6e929 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:21.498142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bf3d580-2a16-4eba-95f5-0e63cbb6e929 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:21.529640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3bf3d580-2a16-4eba-95f5-0e63cbb6e929 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046664 Aug 28 01:05:21.530487 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 639/1099] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:21 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:05:21.954295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4f9e3c52-66db-433c-bf69-72803a5614f3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:21.954903 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4f9e3c52-66db-433c-bf69-72803a5614f3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:21.961935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f9e3c52-66db-433c-bf69-72803a5614f3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:21.962407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f9e3c52-66db-433c-bf69-72803a5614f3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:21.962831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f9e3c52-66db-433c-bf69-72803a5614f3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:21.963532 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f9e3c52-66db-433c-bf69-72803a5614f3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:21.963972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f9e3c52-66db-433c-bf69-72803a5614f3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:21.964384 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f9e3c52-66db-433c-bf69-72803a5614f3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:22.030045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4f9e3c52-66db-433c-bf69-72803a5614f3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:22.038506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4f9e3c52-66db-433c-bf69-72803a5614f3 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.088024 Aug 28 01:05:22.039109 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 640/1100] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:21 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:05:22.044191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-24f4842b-4618-4c34-b3da-45e18e4315e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:22.046462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-24f4842b-4618-4c34-b3da-45e18e4315e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:22.054312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24f4842b-4618-4c34-b3da-45e18e4315e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:22.055111 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24f4842b-4618-4c34-b3da-45e18e4315e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:22.055750 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24f4842b-4618-4c34-b3da-45e18e4315e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:22.056728 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24f4842b-4618-4c34-b3da-45e18e4315e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:22.057553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24f4842b-4618-4c34-b3da-45e18e4315e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:22.058129 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24f4842b-4618-4c34-b3da-45e18e4315e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:22.092283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-24f4842b-4618-4c34-b3da-45e18e4315e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052055 Aug 28 01:05:22.092860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 641/1101] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:22 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:22.542385 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6623a3fa-f082-4166-9608-ebe427c60dbc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:22.544662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6623a3fa-f082-4166-9608-ebe427c60dbc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:22.551308 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6623a3fa-f082-4166-9608-ebe427c60dbc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:22.551789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6623a3fa-f082-4166-9608-ebe427c60dbc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:22.552262 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6623a3fa-f082-4166-9608-ebe427c60dbc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:22.553061 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6623a3fa-f082-4166-9608-ebe427c60dbc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:22.553505 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6623a3fa-f082-4166-9608-ebe427c60dbc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:22.553926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6623a3fa-f082-4166-9608-ebe427c60dbc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:22.588194 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6623a3fa-f082-4166-9608-ebe427c60dbc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049874 Aug 28 01:05:22.588732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 642/1102] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:22 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:05:23.060537 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4f632c1c-b830-4aa5-9afb-05ff9938823d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:23.061195 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4f632c1c-b830-4aa5-9afb-05ff9938823d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:23.068791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f632c1c-b830-4aa5-9afb-05ff9938823d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:23.069389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f632c1c-b830-4aa5-9afb-05ff9938823d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:23.069820 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f632c1c-b830-4aa5-9afb-05ff9938823d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:23.070624 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f632c1c-b830-4aa5-9afb-05ff9938823d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:23.071073 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f632c1c-b830-4aa5-9afb-05ff9938823d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:23.071490 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f632c1c-b830-4aa5-9afb-05ff9938823d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:23.142415 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4f632c1c-b830-4aa5-9afb-05ff9938823d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:23.151147 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4f632c1c-b830-4aa5-9afb-05ff9938823d tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.094572 Aug 28 01:05:23.151726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 643/1103] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:23 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:05:23.156701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7704ba02-ae0b-4cf9-ac73-911f29daa035 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:23.159089 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7704ba02-ae0b-4cf9-ac73-911f29daa035 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:23.166420 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7704ba02-ae0b-4cf9-ac73-911f29daa035 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:23.166912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7704ba02-ae0b-4cf9-ac73-911f29daa035 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:23.167347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7704ba02-ae0b-4cf9-ac73-911f29daa035 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:23.168183 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7704ba02-ae0b-4cf9-ac73-911f29daa035 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:23.168643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7704ba02-ae0b-4cf9-ac73-911f29daa035 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:23.169096 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7704ba02-ae0b-4cf9-ac73-911f29daa035 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:23.203339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7704ba02-ae0b-4cf9-ac73-911f29daa035 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050491 Aug 28 01:05:23.203926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 644/1104] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:23 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:23.600415 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7709747f-4ca8-4131-97b6-16090fda0029 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:23.602849 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7709747f-4ca8-4131-97b6-16090fda0029 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:23.609528 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7709747f-4ca8-4131-97b6-16090fda0029 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:23.610021 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7709747f-4ca8-4131-97b6-16090fda0029 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:23.610465 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7709747f-4ca8-4131-97b6-16090fda0029 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:23.611176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7709747f-4ca8-4131-97b6-16090fda0029 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:23.611658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7709747f-4ca8-4131-97b6-16090fda0029 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:23.612085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7709747f-4ca8-4131-97b6-16090fda0029 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:23.646243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7709747f-4ca8-4131-97b6-16090fda0029 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049992 Aug 28 01:05:23.646875 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 645/1105] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:23 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:05:24.134810 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=4 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:05:24.175965 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5f693e03-ebe6-4b6a-b14b-5eae352c0260 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:24.176495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5f693e03-ebe6-4b6a-b14b-5eae352c0260 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:24.183868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f693e03-ebe6-4b6a-b14b-5eae352c0260 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:24.184338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f693e03-ebe6-4b6a-b14b-5eae352c0260 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:24.184792 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f693e03-ebe6-4b6a-b14b-5eae352c0260 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:24.185556 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f693e03-ebe6-4b6a-b14b-5eae352c0260 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:24.186002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f693e03-ebe6-4b6a-b14b-5eae352c0260 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:24.186417 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f693e03-ebe6-4b6a-b14b-5eae352c0260 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:24.250150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5f693e03-ebe6-4b6a-b14b-5eae352c0260 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:24.258283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5f693e03-ebe6-4b6a-b14b-5eae352c0260 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.086432 Aug 28 01:05:24.258858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 646/1106] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:24 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:05:24.264764 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ec1d7cb0-43e9-45bf-bcaf-55ef6afa4f55 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:24.266946 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ec1d7cb0-43e9-45bf-bcaf-55ef6afa4f55 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:24.273450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec1d7cb0-43e9-45bf-bcaf-55ef6afa4f55 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:24.274009 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec1d7cb0-43e9-45bf-bcaf-55ef6afa4f55 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:24.274533 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec1d7cb0-43e9-45bf-bcaf-55ef6afa4f55 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:24.275238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec1d7cb0-43e9-45bf-bcaf-55ef6afa4f55 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:24.275681 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec1d7cb0-43e9-45bf-bcaf-55ef6afa4f55 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:24.276096 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec1d7cb0-43e9-45bf-bcaf-55ef6afa4f55 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:24.306955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ec1d7cb0-43e9-45bf-bcaf-55ef6afa4f55 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046933 Aug 28 01:05:24.307595 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 647/1107] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:24 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:24.660444 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1204d81a-8bbf-4c53-9353-424e8e4dcf72 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:24.663914 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1204d81a-8bbf-4c53-9353-424e8e4dcf72 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:24.670637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1204d81a-8bbf-4c53-9353-424e8e4dcf72 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:24.671123 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1204d81a-8bbf-4c53-9353-424e8e4dcf72 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:24.671551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1204d81a-8bbf-4c53-9353-424e8e4dcf72 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:24.672261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1204d81a-8bbf-4c53-9353-424e8e4dcf72 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:24.672699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1204d81a-8bbf-4c53-9353-424e8e4dcf72 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:24.673161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1204d81a-8bbf-4c53-9353-424e8e4dcf72 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:24.706492 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1204d81a-8bbf-4c53-9353-424e8e4dcf72 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050745 Aug 28 01:05:24.707072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 648/1108] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:24 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:05:25.279887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b667dea4-b4e1-48dd-814a-b36b7179f428 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:25.280487 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b667dea4-b4e1-48dd-814a-b36b7179f428 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:25.287383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b667dea4-b4e1-48dd-814a-b36b7179f428 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:25.287864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b667dea4-b4e1-48dd-814a-b36b7179f428 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:25.288288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b667dea4-b4e1-48dd-814a-b36b7179f428 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:25.288998 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b667dea4-b4e1-48dd-814a-b36b7179f428 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:25.289452 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b667dea4-b4e1-48dd-814a-b36b7179f428 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:25.289851 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b667dea4-b4e1-48dd-814a-b36b7179f428 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:25.359202 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b667dea4-b4e1-48dd-814a-b36b7179f428 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:25.375553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b667dea4-b4e1-48dd-814a-b36b7179f428 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.099438 Aug 28 01:05:25.375985 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 649/1109] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:25 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:05:25.383243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a60b299f-f05b-482d-9e32-35d94a7ef3ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:25.387207 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a60b299f-f05b-482d-9e32-35d94a7ef3ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:25.394012 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a60b299f-f05b-482d-9e32-35d94a7ef3ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:25.394510 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a60b299f-f05b-482d-9e32-35d94a7ef3ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:25.394941 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a60b299f-f05b-482d-9e32-35d94a7ef3ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:25.395684 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a60b299f-f05b-482d-9e32-35d94a7ef3ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:25.396128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a60b299f-f05b-482d-9e32-35d94a7ef3ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:25.396548 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a60b299f-f05b-482d-9e32-35d94a7ef3ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:25.427223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a60b299f-f05b-482d-9e32-35d94a7ef3ae tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050140 Aug 28 01:05:25.427794 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 650/1110] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:25 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:25.721140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-14df0371-e571-4e28-8fd3-20fb934dd10d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:25.724651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-14df0371-e571-4e28-8fd3-20fb934dd10d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:25.733660 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14df0371-e571-4e28-8fd3-20fb934dd10d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:25.734487 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14df0371-e571-4e28-8fd3-20fb934dd10d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:25.735273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14df0371-e571-4e28-8fd3-20fb934dd10d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:25.736456 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14df0371-e571-4e28-8fd3-20fb934dd10d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:25.737077 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14df0371-e571-4e28-8fd3-20fb934dd10d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:25.737500 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14df0371-e571-4e28-8fd3-20fb934dd10d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:25.742379 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:05:25.794045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-14df0371-e571-4e28-8fd3-20fb934dd10d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.078213 Aug 28 01:05:25.794922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 651/1111] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:25 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:05:26.404159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e23552bf-ccd0-45c4-969e-92b41f883928 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:26.404767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e23552bf-ccd0-45c4-969e-92b41f883928 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:26.412164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e23552bf-ccd0-45c4-969e-92b41f883928 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:26.412665 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e23552bf-ccd0-45c4-969e-92b41f883928 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:26.413149 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e23552bf-ccd0-45c4-969e-92b41f883928 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:26.413893 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e23552bf-ccd0-45c4-969e-92b41f883928 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:26.414338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e23552bf-ccd0-45c4-969e-92b41f883928 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:26.414753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e23552bf-ccd0-45c4-969e-92b41f883928 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:26.483643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e23552bf-ccd0-45c4-969e-92b41f883928 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:26.492273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e23552bf-ccd0-45c4-969e-92b41f883928 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.092364 Aug 28 01:05:26.492840 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 652/1112] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:26 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:05:26.497911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b7eef7fa-7dee-4821-b818-a483818372e2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:26.500078 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b7eef7fa-7dee-4821-b818-a483818372e2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:26.506872 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7eef7fa-7dee-4821-b818-a483818372e2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:26.507376 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7eef7fa-7dee-4821-b818-a483818372e2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:26.507804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7eef7fa-7dee-4821-b818-a483818372e2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:26.508530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7eef7fa-7dee-4821-b818-a483818372e2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:26.509017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7eef7fa-7dee-4821-b818-a483818372e2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:26.509432 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7eef7fa-7dee-4821-b818-a483818372e2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:26.544629 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b7eef7fa-7dee-4821-b818-a483818372e2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050594 Aug 28 01:05:26.545349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 653/1113] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:26 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:26.810151 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a29a5c53-4653-420b-a820-e59ddfac33ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:26.812403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a29a5c53-4653-420b-a820-e59ddfac33ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:26.819848 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a29a5c53-4653-420b-a820-e59ddfac33ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:26.820342 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a29a5c53-4653-420b-a820-e59ddfac33ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:26.820781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a29a5c53-4653-420b-a820-e59ddfac33ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:26.821593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a29a5c53-4653-420b-a820-e59ddfac33ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:26.821993 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a29a5c53-4653-420b-a820-e59ddfac33ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:26.822421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a29a5c53-4653-420b-a820-e59ddfac33ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:26.856344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a29a5c53-4653-420b-a820-e59ddfac33ff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050306 Aug 28 01:05:26.856935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 654/1114] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:26 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:05:27.516392 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e412a3fc-17d1-4422-99e1-3567515ee167 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:27.517007 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e412a3fc-17d1-4422-99e1-3567515ee167 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:27.524650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e412a3fc-17d1-4422-99e1-3567515ee167 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:27.525201 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e412a3fc-17d1-4422-99e1-3567515ee167 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:27.525621 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e412a3fc-17d1-4422-99e1-3567515ee167 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:27.526361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e412a3fc-17d1-4422-99e1-3567515ee167 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:27.526821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e412a3fc-17d1-4422-99e1-3567515ee167 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:27.527268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e412a3fc-17d1-4422-99e1-3567515ee167 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:27.597593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e412a3fc-17d1-4422-99e1-3567515ee167 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:27.606753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e412a3fc-17d1-4422-99e1-3567515ee167 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.094387 Aug 28 01:05:27.607381 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 655/1115] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:27 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:05:27.614157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4f8b37d1-661d-4931-9653-fcc9e85aefb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:27.616563 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4f8b37d1-661d-4931-9653-fcc9e85aefb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:27.626836 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f8b37d1-661d-4931-9653-fcc9e85aefb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:27.627693 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f8b37d1-661d-4931-9653-fcc9e85aefb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:27.628414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f8b37d1-661d-4931-9653-fcc9e85aefb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:27.629696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f8b37d1-661d-4931-9653-fcc9e85aefb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:27.630499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f8b37d1-661d-4931-9653-fcc9e85aefb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:27.631253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f8b37d1-661d-4931-9653-fcc9e85aefb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:27.678444 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4f8b37d1-661d-4931-9653-fcc9e85aefb0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.069841 Aug 28 01:05:27.679025 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 656/1116] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:27 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:27.868277 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-94460409-f903-442b-be92-4afe98ca4369 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:27.870698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-94460409-f903-442b-be92-4afe98ca4369 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:27.877715 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-94460409-f903-442b-be92-4afe98ca4369 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:27.878236 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-94460409-f903-442b-be92-4afe98ca4369 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:27.878648 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-94460409-f903-442b-be92-4afe98ca4369 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:27.879372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-94460409-f903-442b-be92-4afe98ca4369 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:27.879832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-94460409-f903-442b-be92-4afe98ca4369 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:27.880255 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-94460409-f903-442b-be92-4afe98ca4369 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:27.913897 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-94460409-f903-442b-be92-4afe98ca4369 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049672 Aug 28 01:05:27.914489 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 657/1117] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:27 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:05:28.630504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b9dff22a-481e-4116-afa2-9f757259a4ff tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:28.631286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b9dff22a-481e-4116-afa2-9f757259a4ff tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:28.639579 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9dff22a-481e-4116-afa2-9f757259a4ff tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:28.640044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9dff22a-481e-4116-afa2-9f757259a4ff tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:28.640501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9dff22a-481e-4116-afa2-9f757259a4ff tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:28.641428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9dff22a-481e-4116-afa2-9f757259a4ff tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:28.641885 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9dff22a-481e-4116-afa2-9f757259a4ff tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:28.642284 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9dff22a-481e-4116-afa2-9f757259a4ff tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:28.711798 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b9dff22a-481e-4116-afa2-9f757259a4ff tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:28.720753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b9dff22a-481e-4116-afa2-9f757259a4ff tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.094603 Aug 28 01:05:28.721555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 658/1118] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:28 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:05:28.726758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-eacf6e3e-1206-4263-85fe-655fb5565463 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:28.729395 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-eacf6e3e-1206-4263-85fe-655fb5565463 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:28.737563 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eacf6e3e-1206-4263-85fe-655fb5565463 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:28.738212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eacf6e3e-1206-4263-85fe-655fb5565463 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:28.738767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eacf6e3e-1206-4263-85fe-655fb5565463 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:28.739583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eacf6e3e-1206-4263-85fe-655fb5565463 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:28.740180 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eacf6e3e-1206-4263-85fe-655fb5565463 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:28.740717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eacf6e3e-1206-4263-85fe-655fb5565463 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:28.776164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-eacf6e3e-1206-4263-85fe-655fb5565463 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053389 Aug 28 01:05:28.776754 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 659/1119] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:28 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:28.931909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7fc72186-e552-4ab5-a80f-2f42e39c561b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:28.935733 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7fc72186-e552-4ab5-a80f-2f42e39c561b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:28.947476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fc72186-e552-4ab5-a80f-2f42e39c561b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:28.948339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fc72186-e552-4ab5-a80f-2f42e39c561b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:28.949162 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fc72186-e552-4ab5-a80f-2f42e39c561b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:28.950533 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fc72186-e552-4ab5-a80f-2f42e39c561b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:28.951328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fc72186-e552-4ab5-a80f-2f42e39c561b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:28.952091 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fc72186-e552-4ab5-a80f-2f42e39c561b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:28.992529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7fc72186-e552-4ab5-a80f-2f42e39c561b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.068803 Aug 28 01:05:28.993169 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 660/1120] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:28 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:05:29.744141 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9ce1fbf0-5988-48cb-85a2-d84bb929f73a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:29.744772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9ce1fbf0-5988-48cb-85a2-d84bb929f73a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:29.752990 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9ce1fbf0-5988-48cb-85a2-d84bb929f73a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:29.753621 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9ce1fbf0-5988-48cb-85a2-d84bb929f73a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:29.754046 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9ce1fbf0-5988-48cb-85a2-d84bb929f73a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:29.754760 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9ce1fbf0-5988-48cb-85a2-d84bb929f73a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:29.755199 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9ce1fbf0-5988-48cb-85a2-d84bb929f73a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:29.755614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9ce1fbf0-5988-48cb-85a2-d84bb929f73a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:29.828185 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9ce1fbf0-5988-48cb-85a2-d84bb929f73a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:29.841299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9ce1fbf0-5988-48cb-85a2-d84bb929f73a tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.100674 Aug 28 01:05:29.842307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 661/1121] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:29 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:05:29.851230 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2ef6de1e-fffb-4865-9b81-a03c7408db25 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:29.855020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2ef6de1e-fffb-4865-9b81-a03c7408db25 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:29.866017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ef6de1e-fffb-4865-9b81-a03c7408db25 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:29.866854 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ef6de1e-fffb-4865-9b81-a03c7408db25 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:29.867614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ef6de1e-fffb-4865-9b81-a03c7408db25 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:29.868908 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ef6de1e-fffb-4865-9b81-a03c7408db25 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:29.869789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ef6de1e-fffb-4865-9b81-a03c7408db25 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:29.870519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ef6de1e-fffb-4865-9b81-a03c7408db25 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:29.909643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2ef6de1e-fffb-4865-9b81-a03c7408db25 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.065767 Aug 28 01:05:29.910252 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 662/1122] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:29 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:30.006532 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-381170fd-b2a7-47e6-996f-9881f6e63599 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:30.008688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-381170fd-b2a7-47e6-996f-9881f6e63599 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:30.016052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-381170fd-b2a7-47e6-996f-9881f6e63599 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:30.016528 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-381170fd-b2a7-47e6-996f-9881f6e63599 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:30.017208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-381170fd-b2a7-47e6-996f-9881f6e63599 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:30.017976 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-381170fd-b2a7-47e6-996f-9881f6e63599 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:30.018508 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-381170fd-b2a7-47e6-996f-9881f6e63599 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:30.018950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-381170fd-b2a7-47e6-996f-9881f6e63599 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:30.054787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-381170fd-b2a7-47e6-996f-9881f6e63599 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052533 Aug 28 01:05:30.055372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 663/1123] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:30 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:05:30.864083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6a052c41-d737-47bd-bce5-1759831b6cf5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:30.864728 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6a052c41-d737-47bd-bce5-1759831b6cf5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:30.872539 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6a052c41-d737-47bd-bce5-1759831b6cf5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:30.873133 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6a052c41-d737-47bd-bce5-1759831b6cf5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:30.873562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6a052c41-d737-47bd-bce5-1759831b6cf5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:30.874314 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6a052c41-d737-47bd-bce5-1759831b6cf5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:30.874760 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6a052c41-d737-47bd-bce5-1759831b6cf5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:30.875185 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6a052c41-d737-47bd-bce5-1759831b6cf5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:30.944352 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6a052c41-d737-47bd-bce5-1759831b6cf5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:30.958645 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6a052c41-d737-47bd-bce5-1759831b6cf5 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.098231 Aug 28 01:05:30.959721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 664/1124] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:30 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:05:30.968771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0e7f400a-43a8-4e9e-bfa7-b15c7ddb4940 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:30.972515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0e7f400a-43a8-4e9e-bfa7-b15c7ddb4940 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:30.983845 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e7f400a-43a8-4e9e-bfa7-b15c7ddb4940 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:30.984710 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e7f400a-43a8-4e9e-bfa7-b15c7ddb4940 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:30.985493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e7f400a-43a8-4e9e-bfa7-b15c7ddb4940 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:30.986736 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e7f400a-43a8-4e9e-bfa7-b15c7ddb4940 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:30.987529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e7f400a-43a8-4e9e-bfa7-b15c7ddb4940 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:30.988284 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e7f400a-43a8-4e9e-bfa7-b15c7ddb4940 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:31.048124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0e7f400a-43a8-4e9e-bfa7-b15c7ddb4940 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.086328 Aug 28 01:05:31.048992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 665/1125] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:30 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:31.069031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4732e5d0-a455-4a53-b39c-6df906465a3c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:31.071410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4732e5d0-a455-4a53-b39c-6df906465a3c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:31.078705 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4732e5d0-a455-4a53-b39c-6df906465a3c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:31.079215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4732e5d0-a455-4a53-b39c-6df906465a3c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:31.079657 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4732e5d0-a455-4a53-b39c-6df906465a3c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:31.080423 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4732e5d0-a455-4a53-b39c-6df906465a3c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:31.080887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4732e5d0-a455-4a53-b39c-6df906465a3c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:31.081350 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4732e5d0-a455-4a53-b39c-6df906465a3c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:31.117796 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4732e5d0-a455-4a53-b39c-6df906465a3c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053421 Aug 28 01:05:31.118752 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 666/1126] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:31 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:05:31.981264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9f8a95e0-fee5-4ff9-a949-4240a3dabcfd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:31.982029 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9f8a95e0-fee5-4ff9-a949-4240a3dabcfd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:31.985943 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:05:31.989772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9f8a95e0-fee5-4ff9-a949-4240a3dabcfd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:31.990286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9f8a95e0-fee5-4ff9-a949-4240a3dabcfd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:31.990699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9f8a95e0-fee5-4ff9-a949-4240a3dabcfd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:31.991415 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9f8a95e0-fee5-4ff9-a949-4240a3dabcfd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:31.991857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9f8a95e0-fee5-4ff9-a949-4240a3dabcfd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:31.992275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9f8a95e0-fee5-4ff9-a949-4240a3dabcfd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:32.061895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9f8a95e0-fee5-4ff9-a949-4240a3dabcfd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:32.067124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:05:32.073397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9f8a95e0-fee5-4ff9-a949-4240a3dabcfd tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.096342 Aug 28 01:05:32.074453 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 667/1127] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:31 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:05:32.083438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ed59271b-e2c7-47b0-9ffe-d61453db212d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:32.087329 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ed59271b-e2c7-47b0-9ffe-d61453db212d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:32.099239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed59271b-e2c7-47b0-9ffe-d61453db212d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:32.100067 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed59271b-e2c7-47b0-9ffe-d61453db212d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:32.100647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed59271b-e2c7-47b0-9ffe-d61453db212d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:32.101675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed59271b-e2c7-47b0-9ffe-d61453db212d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:32.102335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed59271b-e2c7-47b0-9ffe-d61453db212d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:32.102897 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed59271b-e2c7-47b0-9ffe-d61453db212d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:32.144287 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ed59271b-e2c7-47b0-9ffe-d61453db212d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.068358 Aug 28 01:05:32.144880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 668/1128] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:32 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:32.153437 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d58fd7e5-9b7a-474e-99b5-2c833ffac9b8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:32.155787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d58fd7e5-9b7a-474e-99b5-2c833ffac9b8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:32.163297 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d58fd7e5-9b7a-474e-99b5-2c833ffac9b8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:32.163786 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d58fd7e5-9b7a-474e-99b5-2c833ffac9b8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:32.164245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d58fd7e5-9b7a-474e-99b5-2c833ffac9b8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:32.165004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d58fd7e5-9b7a-474e-99b5-2c833ffac9b8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:32.165444 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d58fd7e5-9b7a-474e-99b5-2c833ffac9b8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:32.165855 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d58fd7e5-9b7a-474e-99b5-2c833ffac9b8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:32.201664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d58fd7e5-9b7a-474e-99b5-2c833ffac9b8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055616 Aug 28 01:05:32.202249 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 669/1129] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:32 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:05:33.108564 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4f8501ed-f20c-4a76-afcd-5d02125e3f76 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:33.109531 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4f8501ed-f20c-4a76-afcd-5d02125e3f76 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:33.117102 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f8501ed-f20c-4a76-afcd-5d02125e3f76 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:33.117608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f8501ed-f20c-4a76-afcd-5d02125e3f76 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:33.118051 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f8501ed-f20c-4a76-afcd-5d02125e3f76 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:33.118794 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f8501ed-f20c-4a76-afcd-5d02125e3f76 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:33.119257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f8501ed-f20c-4a76-afcd-5d02125e3f76 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:33.119685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f8501ed-f20c-4a76-afcd-5d02125e3f76 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:33.203329 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4f8501ed-f20c-4a76-afcd-5d02125e3f76 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:33.217545 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4f8501ed-f20c-4a76-afcd-5d02125e3f76 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.112957 Aug 28 01:05:33.218610 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 670/1130] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:33 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:05:33.227818 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4c1a5684-4354-45c6-983d-b2a74d6cafd1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:33.231629 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4c1a5684-4354-45c6-983d-b2a74d6cafd1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:33.242904 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c1a5684-4354-45c6-983d-b2a74d6cafd1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:33.243778 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c1a5684-4354-45c6-983d-b2a74d6cafd1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:33.244571 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c1a5684-4354-45c6-983d-b2a74d6cafd1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:33.246024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c1a5684-4354-45c6-983d-b2a74d6cafd1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:33.246875 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c1a5684-4354-45c6-983d-b2a74d6cafd1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:33.247658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c1a5684-4354-45c6-983d-b2a74d6cafd1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:33.292951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4c1a5684-4354-45c6-983d-b2a74d6cafd1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.072775 Aug 28 01:05:33.293569 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 671/1131] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:33 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:33.305737 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-78de1b0e-c7ed-47df-b9d8-fbf2821807d5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:33.308789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-78de1b0e-c7ed-47df-b9d8-fbf2821807d5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:33.315935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-78de1b0e-c7ed-47df-b9d8-fbf2821807d5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:33.316470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-78de1b0e-c7ed-47df-b9d8-fbf2821807d5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:33.316949 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-78de1b0e-c7ed-47df-b9d8-fbf2821807d5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:33.317759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-78de1b0e-c7ed-47df-b9d8-fbf2821807d5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:33.318269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-78de1b0e-c7ed-47df-b9d8-fbf2821807d5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:33.318709 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-78de1b0e-c7ed-47df-b9d8-fbf2821807d5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:33.355316 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-78de1b0e-c7ed-47df-b9d8-fbf2821807d5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060601 Aug 28 01:05:33.355942 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 672/1132] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:33 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:05:34.239721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b2bf13a1-8eaf-4af2-bef4-4680320cef47 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:34.240507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b2bf13a1-8eaf-4af2-bef4-4680320cef47 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:34.248385 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2bf13a1-8eaf-4af2-bef4-4680320cef47 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:34.248893 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2bf13a1-8eaf-4af2-bef4-4680320cef47 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:34.249555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2bf13a1-8eaf-4af2-bef4-4680320cef47 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:34.250349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2bf13a1-8eaf-4af2-bef4-4680320cef47 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:34.250799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2bf13a1-8eaf-4af2-bef4-4680320cef47 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:34.251260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2bf13a1-8eaf-4af2-bef4-4680320cef47 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:34.326037 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b2bf13a1-8eaf-4af2-bef4-4680320cef47 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:34.336146 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b2bf13a1-8eaf-4af2-bef4-4680320cef47 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.100408 Aug 28 01:05:34.336769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 673/1133] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:34 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:05:34.342052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3e489516-79c1-46c5-8b1c-84e71cd6eb95 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:34.344291 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3e489516-79c1-46c5-8b1c-84e71cd6eb95 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:34.351076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e489516-79c1-46c5-8b1c-84e71cd6eb95 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:34.351580 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e489516-79c1-46c5-8b1c-84e71cd6eb95 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:34.352048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e489516-79c1-46c5-8b1c-84e71cd6eb95 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:34.352724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e489516-79c1-46c5-8b1c-84e71cd6eb95 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:34.353227 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e489516-79c1-46c5-8b1c-84e71cd6eb95 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:34.353653 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e489516-79c1-46c5-8b1c-84e71cd6eb95 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:34.386657 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3e489516-79c1-46c5-8b1c-84e71cd6eb95 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048676 Aug 28 01:05:34.387262 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 674/1134] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:34 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:34.459733 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7e638dc5-8043-426c-b1ea-5c410bfe2092 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:34.462128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7e638dc5-8043-426c-b1ea-5c410bfe2092 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:34.469962 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e638dc5-8043-426c-b1ea-5c410bfe2092 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:34.470585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e638dc5-8043-426c-b1ea-5c410bfe2092 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:34.471137 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e638dc5-8043-426c-b1ea-5c410bfe2092 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:34.471981 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e638dc5-8043-426c-b1ea-5c410bfe2092 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:34.472541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e638dc5-8043-426c-b1ea-5c410bfe2092 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:34.473110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e638dc5-8043-426c-b1ea-5c410bfe2092 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:34.504862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7e638dc5-8043-426c-b1ea-5c410bfe2092 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.116506 Aug 28 01:05:34.505501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 675/1135] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:34 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:05:35.364999 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6ede446b-7a9e-4f91-a919-42acc18c77c8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:35.365699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6ede446b-7a9e-4f91-a919-42acc18c77c8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:35.373372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ede446b-7a9e-4f91-a919-42acc18c77c8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:35.373777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ede446b-7a9e-4f91-a919-42acc18c77c8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:35.374208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ede446b-7a9e-4f91-a919-42acc18c77c8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:35.374932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ede446b-7a9e-4f91-a919-42acc18c77c8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:35.375380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ede446b-7a9e-4f91-a919-42acc18c77c8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:35.375804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ede446b-7a9e-4f91-a919-42acc18c77c8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:35.446942 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6ede446b-7a9e-4f91-a919-42acc18c77c8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:35.455295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6ede446b-7a9e-4f91-a919-42acc18c77c8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.094171 Aug 28 01:05:35.455882 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 676/1136] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:35 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:05:35.460753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b827aff3-3d8e-4e2a-ad9c-48eaa8190023 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:35.463009 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b827aff3-3d8e-4e2a-ad9c-48eaa8190023 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:35.469825 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b827aff3-3d8e-4e2a-ad9c-48eaa8190023 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:35.470321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b827aff3-3d8e-4e2a-ad9c-48eaa8190023 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:35.470758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b827aff3-3d8e-4e2a-ad9c-48eaa8190023 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:35.471479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b827aff3-3d8e-4e2a-ad9c-48eaa8190023 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:35.471931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b827aff3-3d8e-4e2a-ad9c-48eaa8190023 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:35.472367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b827aff3-3d8e-4e2a-ad9c-48eaa8190023 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:35.504631 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b827aff3-3d8e-4e2a-ad9c-48eaa8190023 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047594 Aug 28 01:05:35.505464 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 677/1137] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:35 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:35.517646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-81e33c92-e053-423f-bdc6-1273110523b7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:35.519859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-81e33c92-e053-423f-bdc6-1273110523b7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:35.526490 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81e33c92-e053-423f-bdc6-1273110523b7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:35.526981 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81e33c92-e053-423f-bdc6-1273110523b7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:35.527517 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81e33c92-e053-423f-bdc6-1273110523b7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:35.528230 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81e33c92-e053-423f-bdc6-1273110523b7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:35.528683 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81e33c92-e053-423f-bdc6-1273110523b7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:35.529153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81e33c92-e053-423f-bdc6-1273110523b7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:35.562365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-81e33c92-e053-423f-bdc6-1273110523b7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048681 Aug 28 01:05:35.562942 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 678/1138] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:35 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:05:36.476880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a16beee9-bb89-4c5f-a35a-8b16295a5e41 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:36.477611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a16beee9-bb89-4c5f-a35a-8b16295a5e41 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:36.484514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a16beee9-bb89-4c5f-a35a-8b16295a5e41 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:36.485016 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a16beee9-bb89-4c5f-a35a-8b16295a5e41 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:36.485443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a16beee9-bb89-4c5f-a35a-8b16295a5e41 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:36.486154 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a16beee9-bb89-4c5f-a35a-8b16295a5e41 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:36.486612 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a16beee9-bb89-4c5f-a35a-8b16295a5e41 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:36.487038 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a16beee9-bb89-4c5f-a35a-8b16295a5e41 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:36.559091 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a16beee9-bb89-4c5f-a35a-8b16295a5e41 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:36.567632 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a16beee9-bb89-4c5f-a35a-8b16295a5e41 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.094808 Aug 28 01:05:36.568171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 679/1139] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:36 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:05:36.577394 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-66694c44-2acd-405f-aec7-a6ed416cb7e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:36.577394 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-66694c44-2acd-405f-aec7-a6ed416cb7e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:36.582889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66694c44-2acd-405f-aec7-a6ed416cb7e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:36.583568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66694c44-2acd-405f-aec7-a6ed416cb7e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:36.584309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66694c44-2acd-405f-aec7-a6ed416cb7e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:36.585616 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66694c44-2acd-405f-aec7-a6ed416cb7e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:36.586258 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66694c44-2acd-405f-aec7-a6ed416cb7e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:36.586755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66694c44-2acd-405f-aec7-a6ed416cb7e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:36.620857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-66694c44-2acd-405f-aec7-a6ed416cb7e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051371 Aug 28 01:05:36.621472 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 680/1140] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:36 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:36.626115 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9eb6da6b-10dc-44ba-b009-d187ae487a4e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:36.628259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9eb6da6b-10dc-44ba-b009-d187ae487a4e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:36.634394 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9eb6da6b-10dc-44ba-b009-d187ae487a4e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:36.634880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9eb6da6b-10dc-44ba-b009-d187ae487a4e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:36.635359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9eb6da6b-10dc-44ba-b009-d187ae487a4e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:36.636065 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9eb6da6b-10dc-44ba-b009-d187ae487a4e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:36.636515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9eb6da6b-10dc-44ba-b009-d187ae487a4e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:36.636942 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9eb6da6b-10dc-44ba-b009-d187ae487a4e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:36.669014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9eb6da6b-10dc-44ba-b009-d187ae487a4e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046335 Aug 28 01:05:36.669589 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 681/1141] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:36 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:05:37.593212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-64d0e21b-6250-4598-b6c7-8599eff0b504 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:37.593936 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-64d0e21b-6250-4598-b6c7-8599eff0b504 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:37.600585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64d0e21b-6250-4598-b6c7-8599eff0b504 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:37.601086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64d0e21b-6250-4598-b6c7-8599eff0b504 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:37.601511 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64d0e21b-6250-4598-b6c7-8599eff0b504 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:37.602193 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64d0e21b-6250-4598-b6c7-8599eff0b504 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:37.602635 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64d0e21b-6250-4598-b6c7-8599eff0b504 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:37.603051 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64d0e21b-6250-4598-b6c7-8599eff0b504 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:37.672593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-64d0e21b-6250-4598-b6c7-8599eff0b504 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:37.684805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-64d0e21b-6250-4598-b6c7-8599eff0b504 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.095226 Aug 28 01:05:37.685515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 682/1142] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:37 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:05:37.690067 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7eca9fe7-caea-4905-8c27-f41d34294af8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:37.692503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7eca9fe7-caea-4905-8c27-f41d34294af8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:37.699024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7eca9fe7-caea-4905-8c27-f41d34294af8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:37.699494 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7eca9fe7-caea-4905-8c27-f41d34294af8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:37.699921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7eca9fe7-caea-4905-8c27-f41d34294af8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:37.700597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7eca9fe7-caea-4905-8c27-f41d34294af8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:37.701083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7eca9fe7-caea-4905-8c27-f41d34294af8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:37.701499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7eca9fe7-caea-4905-8c27-f41d34294af8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:37.732338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7eca9fe7-caea-4905-8c27-f41d34294af8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045754 Aug 28 01:05:37.732867 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 683/1143] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:37 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:37.737427 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f2249334-9423-45fd-8a31-075b85a71afd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:37.739513 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f2249334-9423-45fd-8a31-075b85a71afd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:37.745452 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f2249334-9423-45fd-8a31-075b85a71afd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:37.745919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f2249334-9423-45fd-8a31-075b85a71afd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:37.746345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f2249334-9423-45fd-8a31-075b85a71afd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:37.747020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f2249334-9423-45fd-8a31-075b85a71afd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:37.747478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f2249334-9423-45fd-8a31-075b85a71afd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:37.747934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f2249334-9423-45fd-8a31-075b85a71afd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:37.764304 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:05:37.779187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f2249334-9423-45fd-8a31-075b85a71afd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045186 Aug 28 01:05:37.779777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 684/1144] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:37 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:05:38.709218 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3af40020-1cc1-466d-b5da-5954f5a81c04 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:38.709921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3af40020-1cc1-466d-b5da-5954f5a81c04 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:38.717286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3af40020-1cc1-466d-b5da-5954f5a81c04 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:38.717810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3af40020-1cc1-466d-b5da-5954f5a81c04 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:38.718276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3af40020-1cc1-466d-b5da-5954f5a81c04 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:38.718979 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3af40020-1cc1-466d-b5da-5954f5a81c04 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:38.719431 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3af40020-1cc1-466d-b5da-5954f5a81c04 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:38.719848 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3af40020-1cc1-466d-b5da-5954f5a81c04 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:38.791874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3af40020-1cc1-466d-b5da-5954f5a81c04 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:38.800236 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3af40020-1cc1-466d-b5da-5954f5a81c04 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.095033 Aug 28 01:05:38.800800 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 685/1145] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:38 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:05:38.805663 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8e1944a5-13f8-4334-b911-2489638b847e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:38.807828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8e1944a5-13f8-4334-b911-2489638b847e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:38.813708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e1944a5-13f8-4334-b911-2489638b847e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:38.814199 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e1944a5-13f8-4334-b911-2489638b847e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:38.814660 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e1944a5-13f8-4334-b911-2489638b847e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:38.815343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e1944a5-13f8-4334-b911-2489638b847e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:38.816134 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e1944a5-13f8-4334-b911-2489638b847e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:38.816550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e1944a5-13f8-4334-b911-2489638b847e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:38.845798 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8e1944a5-13f8-4334-b911-2489638b847e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043949 Aug 28 01:05:38.846346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 686/1146] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:38 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:38.851005 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f99bf586-82ae-4374-8762-9db6e188f7e1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:38.852953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f99bf586-82ae-4374-8762-9db6e188f7e1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:38.858910 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f99bf586-82ae-4374-8762-9db6e188f7e1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:38.859379 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f99bf586-82ae-4374-8762-9db6e188f7e1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:38.859792 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f99bf586-82ae-4374-8762-9db6e188f7e1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:38.860468 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f99bf586-82ae-4374-8762-9db6e188f7e1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:38.860913 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f99bf586-82ae-4374-8762-9db6e188f7e1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:38.861355 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f99bf586-82ae-4374-8762-9db6e188f7e1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:38.890694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f99bf586-82ae-4374-8762-9db6e188f7e1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043333 Aug 28 01:05:38.891245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 687/1147] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:38 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:05:39.822126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-aef0a981-64d4-440b-97a7-5c89c9be5e4c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:39.822812 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-aef0a981-64d4-440b-97a7-5c89c9be5e4c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:39.830398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aef0a981-64d4-440b-97a7-5c89c9be5e4c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:39.830848 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aef0a981-64d4-440b-97a7-5c89c9be5e4c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:39.831282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aef0a981-64d4-440b-97a7-5c89c9be5e4c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:39.831988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aef0a981-64d4-440b-97a7-5c89c9be5e4c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:39.832495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aef0a981-64d4-440b-97a7-5c89c9be5e4c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:39.832924 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aef0a981-64d4-440b-97a7-5c89c9be5e4c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:39.899240 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-aef0a981-64d4-440b-97a7-5c89c9be5e4c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:39.907628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-aef0a981-64d4-440b-97a7-5c89c9be5e4c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.089509 Aug 28 01:05:39.908203 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 688/1148] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:39 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:05:39.913008 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a4d4b755-fd3b-4ec9-9a51-21939224c92a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:39.915150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a4d4b755-fd3b-4ec9-9a51-21939224c92a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:39.921446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4d4b755-fd3b-4ec9-9a51-21939224c92a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:39.921941 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4d4b755-fd3b-4ec9-9a51-21939224c92a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:39.922365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4d4b755-fd3b-4ec9-9a51-21939224c92a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:39.923074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4d4b755-fd3b-4ec9-9a51-21939224c92a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:39.923515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4d4b755-fd3b-4ec9-9a51-21939224c92a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:39.923922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4d4b755-fd3b-4ec9-9a51-21939224c92a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:39.955048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a4d4b755-fd3b-4ec9-9a51-21939224c92a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045840 Aug 28 01:05:39.955616 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 689/1149] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:39 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:39.960129 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0907474d-6dbc-4424-a912-ea1e050d3bd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:39.962357 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0907474d-6dbc-4424-a912-ea1e050d3bd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:39.967970 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0907474d-6dbc-4424-a912-ea1e050d3bd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:39.968447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0907474d-6dbc-4424-a912-ea1e050d3bd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:39.969054 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0907474d-6dbc-4424-a912-ea1e050d3bd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:39.969705 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0907474d-6dbc-4424-a912-ea1e050d3bd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:39.970145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0907474d-6dbc-4424-a912-ea1e050d3bd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:39.970529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0907474d-6dbc-4424-a912-ea1e050d3bd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:40.000201 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0907474d-6dbc-4424-a912-ea1e050d3bd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043440 Aug 28 01:05:40.000742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 690/1150] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:39 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:05:40.929264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-da58ba8c-827b-4605-9506-41183e1fa849 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:40.929969 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-da58ba8c-827b-4605-9506-41183e1fa849 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:40.936782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da58ba8c-827b-4605-9506-41183e1fa849 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:40.937289 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da58ba8c-827b-4605-9506-41183e1fa849 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:40.937735 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da58ba8c-827b-4605-9506-41183e1fa849 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:40.938434 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da58ba8c-827b-4605-9506-41183e1fa849 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:40.938888 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da58ba8c-827b-4605-9506-41183e1fa849 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:40.939311 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da58ba8c-827b-4605-9506-41183e1fa849 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:41.002066 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-da58ba8c-827b-4605-9506-41183e1fa849 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:41.013146 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-da58ba8c-827b-4605-9506-41183e1fa849 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.087625 Aug 28 01:05:41.013888 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 691/1151] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:40 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:05:41.019368 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-56848da1-8445-45c1-bb2b-e106e595c765 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:41.021529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-56848da1-8445-45c1-bb2b-e106e595c765 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:41.027721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56848da1-8445-45c1-bb2b-e106e595c765 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:41.028150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56848da1-8445-45c1-bb2b-e106e595c765 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:41.028579 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56848da1-8445-45c1-bb2b-e106e595c765 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:41.029387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56848da1-8445-45c1-bb2b-e106e595c765 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:41.029752 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56848da1-8445-45c1-bb2b-e106e595c765 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:41.030177 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56848da1-8445-45c1-bb2b-e106e595c765 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:41.060989 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-56848da1-8445-45c1-bb2b-e106e595c765 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045862 Aug 28 01:05:41.061583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 692/1152] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:41 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:41.066557 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-17a73182-9b0a-439c-a81c-53428e207b3b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:41.068664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-17a73182-9b0a-439c-a81c-53428e207b3b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:41.074791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-17a73182-9b0a-439c-a81c-53428e207b3b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:41.075270 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-17a73182-9b0a-439c-a81c-53428e207b3b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:41.075704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-17a73182-9b0a-439c-a81c-53428e207b3b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:41.076362 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-17a73182-9b0a-439c-a81c-53428e207b3b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:41.076822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-17a73182-9b0a-439c-a81c-53428e207b3b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:41.077393 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-17a73182-9b0a-439c-a81c-53428e207b3b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:41.107708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-17a73182-9b0a-439c-a81c-53428e207b3b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045029 Aug 28 01:05:41.108259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 693/1153] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:41 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:05:42.036760 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-374a126a-ddcf-451d-913c-7127a7516a42 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:42.037660 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-374a126a-ddcf-451d-913c-7127a7516a42 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:42.044677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-374a126a-ddcf-451d-913c-7127a7516a42 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:42.045393 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-374a126a-ddcf-451d-913c-7127a7516a42 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:42.045846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-374a126a-ddcf-451d-913c-7127a7516a42 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:42.046559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-374a126a-ddcf-451d-913c-7127a7516a42 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:42.047072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-374a126a-ddcf-451d-913c-7127a7516a42 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:42.047636 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-374a126a-ddcf-451d-913c-7127a7516a42 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:42.122844 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-374a126a-ddcf-451d-913c-7127a7516a42 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:05:42.131885 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-374a126a-ddcf-451d-913c-7127a7516a42 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.100216 Aug 28 01:05:42.132473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 694/1154] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:05:42 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:05:42.137628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-92b18542-fbd9-4645-9011-b3dacea638c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:42.139914 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-92b18542-fbd9-4645-9011-b3dacea638c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:42.146601 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92b18542-fbd9-4645-9011-b3dacea638c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:42.147101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92b18542-fbd9-4645-9011-b3dacea638c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:42.147553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92b18542-fbd9-4645-9011-b3dacea638c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:42.148284 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92b18542-fbd9-4645-9011-b3dacea638c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:42.148839 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92b18542-fbd9-4645-9011-b3dacea638c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:42.149241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92b18542-fbd9-4645-9011-b3dacea638c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:42.180735 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-92b18542-fbd9-4645-9011-b3dacea638c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047125 Aug 28 01:05:42.181527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 695/1155] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:05:42 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:05:42.187449 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b9360d2f-6d11-434b-a18d-b01b2ead1dee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:42.190057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b9360d2f-6d11-434b-a18d-b01b2ead1dee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:42.199223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9360d2f-6d11-434b-a18d-b01b2ead1dee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:42.200528 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9360d2f-6d11-434b-a18d-b01b2ead1dee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:42.200977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9360d2f-6d11-434b-a18d-b01b2ead1dee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:42.201793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9360d2f-6d11-434b-a18d-b01b2ead1dee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:42.202239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9360d2f-6d11-434b-a18d-b01b2ead1dee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:42.202664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9360d2f-6d11-434b-a18d-b01b2ead1dee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:42.234345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b9360d2f-6d11-434b-a18d-b01b2ead1dee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051541 Aug 28 01:05:42.234923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 696/1156] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:05:42 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:05:42.280546 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c95ded82-e4d4-4118-814d-3eb5a34175e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:05:42.281242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c95ded82-e4d4-4118-814d-3eb5a34175e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:05:42.288644 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c95ded82-e4d4-4118-814d-3eb5a34175e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:42.289137 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c95ded82-e4d4-4118-814d-3eb5a34175e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:42.289579 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c95ded82-e4d4-4118-814d-3eb5a34175e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:42.290280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c95ded82-e4d4-4118-814d-3eb5a34175e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:05:42.290730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c95ded82-e4d4-4118-814d-3eb5a34175e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:05:42.291131 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c95ded82-e4d4-4118-814d-3eb5a34175e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:05:52.286152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=60 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:05:52.310570 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=78 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:06:42.609820 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-8523ed49-d319-4b1a-8734-517714a9d405 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f42e1ea7-80b3-4260-a736-af29d99a7706) Aug 28 01:06:42.612413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8523ed49-d319-4b1a-8734-517714a9d405 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Request method failure captured: Aug 28 01:06:42.612413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c HTTP/1.1 Aug 28 01:06:42.612413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:06:42.612413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:06:42.612413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:06:42.612413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:06:42.612413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:06:42.612413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:06:42.612413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:06:42.612413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:06:42.612413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCrzWcuiPK_0ynCrragXxXd_fRw7lHCmbNVMAqr4NhyopxGAZIiQrt0m3y3NawnXLWqyOVM4YbBc-yLRrJ4cXICU0MsT01bsu6VzxaCxGd6JRCChv8LlCV7kFwwyy7Ffwd9JpYuFLguuT-xhEcVlfMIZxgSYB22XTBbtvAeHM_AtK1Xg Aug 28 01:06:42.612413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:06:42.612413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:06:42.612413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:06:42.612413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:06:42.612413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:06:42.612413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:06:42.612413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 01:06:42.612413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:06:42.612413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:06:42.612413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:06:42.614924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_6f3acee4af62483ab1f2cbf3a47b09e1", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/6f3acee4af62483ab1f2cbf3a47b09e1"}]}] Aug 28 01:06:42.614924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:06:42.614924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 01:06:42.614924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:06:42.614924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 01:06:42.614924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:06:42.614924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:06:42.614924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: 1ab2fcb3759045c7a076e5c8222a421d Aug 28 01:06:42.614924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 01:06:42.614924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:06:42.615872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f42e1ea7-80b3-4260-a736-af29d99a7706) Aug 28 01:06:42.615872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'server_id': '03969ac5-934e-4201-8087-0927eae2bbd2', 'id': '6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c'} Aug 28 01:06:42.615872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:06:42.615872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:06:42.615872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:06:42.615872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:06:42.615872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:06:42.615872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:06:42.615872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:06:42.615872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:06:42.615872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:06:42.615872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:06:42.615872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:06:42.615872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:06:42.615872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:06:42.615872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:06:42.615872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:06:42.618189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:06:42.618189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:06:42.618189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:06:42.618189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f42e1ea7-80b3-4260-a736-af29d99a7706) Aug 28 01:06:42.618189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:06:42.618189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:06:42.618189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:06:42.618189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:06:42.618189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5192, in _detach_volume Aug 28 01:06:42.618189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 28 01:06:42.618189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 28 01:06:42.618189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 28 01:06:42.618189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 28 01:06:42.618189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 28 01:06:42.618189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:06:42.618189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:06:42.618189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:06:42.618189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:06:42.618189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:06:42.620767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:06:42.620767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:06:42.620767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:06:42.620767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:06:42.620767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:06:42.620767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:06:42.620767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:06:42.620767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:06:42.620767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:06:42.620767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:06:42.620767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:06:42.620767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f42e1ea7-80b3-4260-a736-af29d99a7706) Aug 28 01:06:42.620767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:06:42.620767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:06:42.620767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:06:42.620767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:06:42.620767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 28 01:06:42.622554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 28 01:06:42.622554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Aug 28 01:06:42.622554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:06:42.622554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Aug 28 01:06:42.622554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 28 01:06:42.622554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5214, in detach_volume Aug 28 01:06:42.622554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 28 01:06:42.622554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Aug 28 01:06:42.622554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:06:42.622554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Aug 28 01:06:42.622554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 28 01:06:42.622554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f42e1ea7-80b3-4260-a736-af29d99a7706) Aug 28 01:06:42.622554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:06:42.622554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:06:42.622554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:06:42.622554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:06:42.622554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:06:42.622554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:06:42.625558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:06:42.625558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:06:42.625558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:06:42.625558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:06:42.625558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 28 01:06:42.625558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 28 01:06:42.625558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f42e1ea7-80b3-4260-a736-af29d99a7706) Aug 28 01:06:42.625558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:06:42.625558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:06:42.625558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:06:42.625558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:06:42.625558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:06:42.625558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:06:42.625558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:06:42.625558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:06:42.625558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f42e1ea7-80b3-4260-a736-af29d99a7706) Aug 28 01:06:42.625558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:06:42.628214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8523ed49-d319-4b1a-8734-517714a9d405 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f42e1ea7-80b3-4260-a736-af29d99a7706) {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:06:42.628214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8523ed49-d319-4b1a-8734-517714a9d405 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c" status: 400 len: 269 microversion: 2.1 time: 120.128762 Aug 28 01:06:42.628214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 461/1157] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Aug 28 01:04:42 2022] DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c => generated 269 bytes in 120131 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 28 01:06:42.628214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ee743d39-ac5a-4ac5-a899-fab2e98428ae tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:06:42.628909 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ee743d39-ac5a-4ac5-a899-fab2e98428ae tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:06:42.629371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ee743d39-ac5a-4ac5-a899-fab2e98428ae tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:06:42.630282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ee743d39-ac5a-4ac5-a899-fab2e98428ae tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:06:42.630694 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ee743d39-ac5a-4ac5-a899-fab2e98428ae tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:06:42.631528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ee743d39-ac5a-4ac5-a899-fab2e98428ae tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:06:42.631984 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ee743d39-ac5a-4ac5-a899-fab2e98428ae tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:06:42.632409 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ee743d39-ac5a-4ac5-a899-fab2e98428ae tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:06:42.705384 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ee743d39-ac5a-4ac5-a899-fab2e98428ae tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:06:42.713535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ee743d39-ac5a-4ac5-a899-fab2e98428ae tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.097586 Aug 28 01:06:42.714139 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 462/1158] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:06:42 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:06:42.797516 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-118b803b-c51b-4af2-b6ae-db94bf1459b7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:06:42.799780 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-118b803b-c51b-4af2-b6ae-db94bf1459b7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:06:42.806643 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-118b803b-c51b-4af2-b6ae-db94bf1459b7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:06:42.807133 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-118b803b-c51b-4af2-b6ae-db94bf1459b7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:06:42.807510 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-118b803b-c51b-4af2-b6ae-db94bf1459b7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:06:42.808216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-118b803b-c51b-4af2-b6ae-db94bf1459b7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:06:42.808666 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-118b803b-c51b-4af2-b6ae-db94bf1459b7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:06:42.809125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-118b803b-c51b-4af2-b6ae-db94bf1459b7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:06:42.816219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.objects.instance [None req-c95ded82-e4d4-4118-814d-3eb5a34175e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lazy-loading 'services' on Instance uuid 03969ac5-934e-4201-8087-0927eae2bbd2 {{(pid=120525) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:06:42.845833 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-118b803b-c51b-4af2-b6ae-db94bf1459b7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.130588 Aug 28 01:06:42.847175 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 463/1159] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:06:42 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:06:42.854581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e239585d-7f84-4f37-b173-33a11a1b841c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:06:42.856750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e239585d-7f84-4f37-b173-33a11a1b841c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:06:42.865995 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e239585d-7f84-4f37-b173-33a11a1b841c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:06:42.866492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e239585d-7f84-4f37-b173-33a11a1b841c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:06:42.866923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e239585d-7f84-4f37-b173-33a11a1b841c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:06:42.867611 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e239585d-7f84-4f37-b173-33a11a1b841c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:06:42.868064 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e239585d-7f84-4f37-b173-33a11a1b841c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:06:42.868503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e239585d-7f84-4f37-b173-33a11a1b841c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:06:42.901640 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e239585d-7f84-4f37-b173-33a11a1b841c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053787 Aug 28 01:06:42.902213 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 464/1160] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:06:42 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:06:42.907844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e21e3e30-547b-4dca-a045-fd6a11bbb1e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:06:42.908333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e21e3e30-547b-4dca-a045-fd6a11bbb1e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:06:42.915333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e21e3e30-547b-4dca-a045-fd6a11bbb1e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:06:42.915877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e21e3e30-547b-4dca-a045-fd6a11bbb1e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:06:42.916399 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e21e3e30-547b-4dca-a045-fd6a11bbb1e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:06:42.917258 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e21e3e30-547b-4dca-a045-fd6a11bbb1e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:06:42.917755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e21e3e30-547b-4dca-a045-fd6a11bbb1e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:06:42.918170 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e21e3e30-547b-4dca-a045-fd6a11bbb1e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:06:52.822226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=4 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:06:52.912546 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=8 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:06:52.937258 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=22 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:07:14.333207 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-c95ded82-e4d4-4118-814d-3eb5a34175e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-11edb8a0-5291-4e1e-882b-173f308d488e) Aug 28 01:07:14.335391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c95ded82-e4d4-4118-814d-3eb5a34175e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Request method failure captured: Aug 28 01:07:14.335391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c HTTP/1.1 Aug 28 01:07:14.335391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:07:14.335391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:07:14.335391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:07:14.335391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:07:14.335391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:07:14.335391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:07:14.335391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:07:14.335391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:07:14.335391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCrzWcuiPK_0ynCrragXxXd_fRw7lHCmbNVMAqr4NhyopxGAZIiQrt0m3y3NawnXLWqyOVM4YbBc-yLRrJ4cXICU0MsT01bsu6VzxaCxGd6JRCChv8LlCV7kFwwyy7Ffwd9JpYuFLguuT-xhEcVlfMIZxgSYB22XTBbtvAeHM_AtK1Xg Aug 28 01:07:14.335391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:07:14.335391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:07:14.335391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:07:14.335391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:07:14.335391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:07:14.335391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:07:14.335391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 01:07:14.335391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:07:14.335391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:07:14.335391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:07:14.337024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_6f3acee4af62483ab1f2cbf3a47b09e1", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/6f3acee4af62483ab1f2cbf3a47b09e1"}]}] Aug 28 01:07:14.337024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:07:14.337024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 01:07:14.337024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:07:14.337024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 01:07:14.337024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:07:14.337024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:07:14.337024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: 1ab2fcb3759045c7a076e5c8222a421d Aug 28 01:07:14.337024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 01:07:14.337024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 01:07:14.338179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-11edb8a0-5291-4e1e-882b-173f308d488e) Aug 28 01:07:14.338179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'server_id': '03969ac5-934e-4201-8087-0927eae2bbd2', 'id': '6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c'} Aug 28 01:07:14.338179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:07:14.338179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:07:14.338179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:07:14.338179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:07:14.338179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:07:14.338179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:07:14.338179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:07:14.338179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:07:14.338179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:07:14.338179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:07:14.338179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:07:14.338179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:07:14.338179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:07:14.338179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:07:14.338179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:07:14.339591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:07:14.339591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:07:14.339591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:07:14.339591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-11edb8a0-5291-4e1e-882b-173f308d488e) Aug 28 01:07:14.339591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:07:14.339591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:07:14.339591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:07:14.339591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:07:14.339591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5192, in _detach_volume Aug 28 01:07:14.339591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 28 01:07:14.339591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 28 01:07:14.339591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 28 01:07:14.339591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 28 01:07:14.339591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 28 01:07:14.339591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:07:14.339591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:07:14.339591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:07:14.339591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:07:14.339591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:07:14.341174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:07:14.341174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:07:14.341174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:07:14.341174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:07:14.341174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:07:14.341174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:07:14.341174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:07:14.341174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:07:14.341174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:07:14.341174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:07:14.341174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:07:14.341174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-11edb8a0-5291-4e1e-882b-173f308d488e) Aug 28 01:07:14.341174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:07:14.341174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:07:14.341174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:07:14.341174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:07:14.341174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 28 01:07:14.342433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 28 01:07:14.342433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Aug 28 01:07:14.342433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:07:14.342433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Aug 28 01:07:14.342433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 28 01:07:14.342433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5214, in detach_volume Aug 28 01:07:14.342433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 28 01:07:14.342433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Aug 28 01:07:14.342433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:07:14.342433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Aug 28 01:07:14.342433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 28 01:07:14.342433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-11edb8a0-5291-4e1e-882b-173f308d488e) Aug 28 01:07:14.342433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:07:14.342433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:07:14.342433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:07:14.342433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:07:14.342433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:07:14.342433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:07:14.343769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:07:14.343769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:07:14.343769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:07:14.343769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:07:14.343769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 28 01:07:14.343769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 28 01:07:14.343769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-11edb8a0-5291-4e1e-882b-173f308d488e) Aug 28 01:07:14.343769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:07:14.343769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:07:14.343769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:07:14.343769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:07:14.343769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:07:14.343769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:07:14.343769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:07:14.343769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:07:14.343769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-11edb8a0-5291-4e1e-882b-173f308d488e) Aug 28 01:07:14.343769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:07:14.345101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c95ded82-e4d4-4118-814d-3eb5a34175e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-11edb8a0-5291-4e1e-882b-173f308d488e) {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:07:14.345101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c95ded82-e4d4-4118-814d-3eb5a34175e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c" status: 400 len: 269 microversion: 2.1 time: 92.063406 Aug 28 01:07:14.345101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 697/1161] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Aug 28 01:05:42 2022] DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c => generated 269 bytes in 92065 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Aug 28 01:07:14.345479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d7303693-afb5-4523-866b-8f4b403f692d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:07:14.346314 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d7303693-afb5-4523-866b-8f4b403f692d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:07:14.354909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7303693-afb5-4523-866b-8f4b403f692d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:07:14.355437 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7303693-afb5-4523-866b-8f4b403f692d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:07:14.355858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7303693-afb5-4523-866b-8f4b403f692d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:07:14.356714 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7303693-afb5-4523-866b-8f4b403f692d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:07:14.357439 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7303693-afb5-4523-866b-8f4b403f692d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:07:14.358305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7303693-afb5-4523-866b-8f4b403f692d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:07:14.429215 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.objects.instance [None req-e21e3e30-547b-4dca-a045-fd6a11bbb1e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lazy-loading 'services' on Instance uuid 03969ac5-934e-4201-8087-0927eae2bbd2 {{(pid=120524) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:07:24.351929 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=2 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:07:24.382542 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=4 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:07:24.433779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=4 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:08:42.966069 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.objects.instance [None req-d7303693-afb5-4523-866b-8f4b403f692d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lazy-loading 'services' on Instance uuid 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc {{(pid=120525) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:08:43.003323 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-e21e3e30-547b-4dca-a045-fd6a11bbb1e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff91d476-c963-4305-8763-95ca2ff620fc) Aug 28 01:08:43.008280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e21e3e30-547b-4dca-a045-fd6a11bbb1e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Request method failure captured: Aug 28 01:08:43.008280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c HTTP/1.1 Aug 28 01:08:43.008280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:08:43.008280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:08:43.008280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:08:43.008280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:08:43.008280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:08:43.008280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:08:43.008280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:08:43.008280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:08:43.008280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCrzWcuiPK_0ynCrragXxXd_fRw7lHCmbNVMAqr4NhyopxGAZIiQrt0m3y3NawnXLWqyOVM4YbBc-yLRrJ4cXICU0MsT01bsu6VzxaCxGd6JRCChv8LlCV7kFwwyy7Ffwd9JpYuFLguuT-xhEcVlfMIZxgSYB22XTBbtvAeHM_AtK1Xg Aug 28 01:08:43.008280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:08:43.008280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:08:43.008280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:08:43.008280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:08:43.008280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:08:43.008280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:08:43.008280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 01:08:43.008280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:08:43.008280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:08:43.008280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:08:43.010091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_6f3acee4af62483ab1f2cbf3a47b09e1", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/6f3acee4af62483ab1f2cbf3a47b09e1"}]}] Aug 28 01:08:43.010091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:08:43.010091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 01:08:43.010091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:08:43.010091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 01:08:43.010091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:08:43.010091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:08:43.010091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: 1ab2fcb3759045c7a076e5c8222a421d Aug 28 01:08:43.010091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 01:08:43.010091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:08:43.011275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff91d476-c963-4305-8763-95ca2ff620fc) Aug 28 01:08:43.011275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'server_id': '03969ac5-934e-4201-8087-0927eae2bbd2', 'id': '6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c'} Aug 28 01:08:43.011275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:08:43.011275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:08:43.011275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:08:43.011275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:08:43.011275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:08:43.011275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:08:43.011275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:08:43.011275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:08:43.011275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:08:43.011275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:08:43.011275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:08:43.011275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:08:43.011275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:08:43.011275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:08:43.011275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:08:43.013882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:08:43.013882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:08:43.013882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:08:43.013882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff91d476-c963-4305-8763-95ca2ff620fc) Aug 28 01:08:43.013882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:08:43.013882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:08:43.013882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:08:43.013882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:08:43.013882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5192, in _detach_volume Aug 28 01:08:43.013882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 28 01:08:43.013882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 28 01:08:43.013882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 28 01:08:43.013882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 28 01:08:43.013882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 28 01:08:43.013882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:08:43.013882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:08:43.013882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:08:43.013882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:08:43.013882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:08:43.016852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:08:43.016852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:08:43.016852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:08:43.016852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:08:43.016852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:08:43.016852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:08:43.016852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:08:43.016852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:08:43.016852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:08:43.016852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:08:43.016852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:08:43.016852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff91d476-c963-4305-8763-95ca2ff620fc) Aug 28 01:08:43.016852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:08:43.016852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:08:43.016852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:08:43.016852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:08:43.016852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 28 01:08:43.019802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 28 01:08:43.019802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Aug 28 01:08:43.019802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:08:43.019802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Aug 28 01:08:43.019802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 28 01:08:43.019802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5214, in detach_volume Aug 28 01:08:43.019802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 28 01:08:43.019802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Aug 28 01:08:43.019802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:08:43.019802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Aug 28 01:08:43.019802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 28 01:08:43.019802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff91d476-c963-4305-8763-95ca2ff620fc) Aug 28 01:08:43.019802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:08:43.019802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:08:43.019802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:08:43.019802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:08:43.019802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:08:43.019802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:08:43.022332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:08:43.022332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:08:43.022332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:08:43.022332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:08:43.022332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 28 01:08:43.022332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 28 01:08:43.022332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff91d476-c963-4305-8763-95ca2ff620fc) Aug 28 01:08:43.022332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:08:43.022332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:08:43.022332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:08:43.022332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:08:43.022332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:08:43.022332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:08:43.022332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:08:43.022332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:08:43.022332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff91d476-c963-4305-8763-95ca2ff620fc) Aug 28 01:08:43.022332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:08:43.025392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e21e3e30-547b-4dca-a045-fd6a11bbb1e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff91d476-c963-4305-8763-95ca2ff620fc) {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:08:43.025392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e21e3e30-547b-4dca-a045-fd6a11bbb1e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c" status: 400 len: 269 microversion: 2.1 time: 120.107593 Aug 28 01:08:43.025392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 465/1162] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Aug 28 01:06:42 2022] DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c => generated 269 bytes in 120110 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 28 01:08:43.119433 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8761b00a-2e8b-424f-ae3b-72056de404ee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:08:43.120463 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8761b00a-2e8b-424f-ae3b-72056de404ee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:08:43.131654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8761b00a-2e8b-424f-ae3b-72056de404ee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:08:43.132231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8761b00a-2e8b-424f-ae3b-72056de404ee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:08:43.132717 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8761b00a-2e8b-424f-ae3b-72056de404ee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:08:43.133603 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8761b00a-2e8b-424f-ae3b-72056de404ee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:08:43.134165 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8761b00a-2e8b-424f-ae3b-72056de404ee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:08:43.134686 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8761b00a-2e8b-424f-ae3b-72056de404ee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:08:43.205454 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8761b00a-2e8b-424f-ae3b-72056de404ee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:08:43.215516 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8761b00a-2e8b-424f-ae3b-72056de404ee tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.201228 Aug 28 01:08:43.216107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 466/1163] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:08:43 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:08:43.221636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c446a372-27b8-41b7-98f4-2eab85792372 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:08:43.224274 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c446a372-27b8-41b7-98f4-2eab85792372 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:08:43.231302 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c446a372-27b8-41b7-98f4-2eab85792372 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:08:43.231849 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c446a372-27b8-41b7-98f4-2eab85792372 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:08:43.232298 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c446a372-27b8-41b7-98f4-2eab85792372 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:08:43.233095 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c446a372-27b8-41b7-98f4-2eab85792372 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:08:43.233549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c446a372-27b8-41b7-98f4-2eab85792372 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:08:43.233975 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c446a372-27b8-41b7-98f4-2eab85792372 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:08:43.269911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c446a372-27b8-41b7-98f4-2eab85792372 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052517 Aug 28 01:08:43.270655 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 467/1164] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:08:43 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:08:43.276231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fa2dd7b0-f5a7-4e51-9301-deac6712b859 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:08:43.278575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fa2dd7b0-f5a7-4e51-9301-deac6712b859 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:08:43.285179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fa2dd7b0-f5a7-4e51-9301-deac6712b859 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:08:43.285670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fa2dd7b0-f5a7-4e51-9301-deac6712b859 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:08:43.286110 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fa2dd7b0-f5a7-4e51-9301-deac6712b859 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:08:43.286816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fa2dd7b0-f5a7-4e51-9301-deac6712b859 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:08:43.287279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fa2dd7b0-f5a7-4e51-9301-deac6712b859 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:08:43.287695 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fa2dd7b0-f5a7-4e51-9301-deac6712b859 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:08:43.324702 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fa2dd7b0-f5a7-4e51-9301-deac6712b859 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052899 Aug 28 01:08:43.325343 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 468/1165] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:08:43 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:08:43.330232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-af3169bd-7093-4339-8771-63156f9665f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:08:43.332717 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-af3169bd-7093-4339-8771-63156f9665f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:08:43.339863 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af3169bd-7093-4339-8771-63156f9665f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:08:43.340394 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af3169bd-7093-4339-8771-63156f9665f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:08:43.340914 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af3169bd-7093-4339-8771-63156f9665f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:08:43.342011 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af3169bd-7093-4339-8771-63156f9665f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:08:43.342461 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af3169bd-7093-4339-8771-63156f9665f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:08:43.343018 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af3169bd-7093-4339-8771-63156f9665f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:08:43.377633 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-af3169bd-7093-4339-8771-63156f9665f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051058 Aug 28 01:08:43.378233 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 469/1166] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:08:43 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:08:43.383587 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-736572fd-b416-4a3a-851b-af8df0cdcfe5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:08:43.384090 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-736572fd-b416-4a3a-851b-af8df0cdcfe5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:08:43.391005 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-736572fd-b416-4a3a-851b-af8df0cdcfe5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:08:43.391513 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-736572fd-b416-4a3a-851b-af8df0cdcfe5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:08:43.391955 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-736572fd-b416-4a3a-851b-af8df0cdcfe5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:08:43.392675 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-736572fd-b416-4a3a-851b-af8df0cdcfe5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:08:43.393158 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-736572fd-b416-4a3a-851b-af8df0cdcfe5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:08:43.393585 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-736572fd-b416-4a3a-851b-af8df0cdcfe5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:08:52.973680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=4 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:08:53.387609 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=10 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:08:53.415182 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=28 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:09:14.517466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d7303693-afb5-4523-866b-8f4b403f692d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "DELETE /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc/os-volume_attachments/7421938a-2d8f-4ae7-aac6-d08ec2a49354" status: 202 len: 0 microversion: 2.1 time: 120.178494 Aug 28 01:09:14.518339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 698/1167] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Aug 28 01:07:14 2022] DELETE /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc/os-volume_attachments/7421938a-2d8f-4ae7-aac6-d08ec2a49354 => generated 0 bytes in 120180 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 01:09:14.524712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1b91f5b1-c0ba-4a62-ae36-42bc1f9b7610 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:09:14.525413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1b91f5b1-c0ba-4a62-ae36-42bc1f9b7610 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:09:14.532652 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b91f5b1-c0ba-4a62-ae36-42bc1f9b7610 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:09:14.533491 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b91f5b1-c0ba-4a62-ae36-42bc1f9b7610 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:09:14.533682 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b91f5b1-c0ba-4a62-ae36-42bc1f9b7610 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:09:14.534551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b91f5b1-c0ba-4a62-ae36-42bc1f9b7610 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:09:14.534826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b91f5b1-c0ba-4a62-ae36-42bc1f9b7610 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:09:14.535446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b91f5b1-c0ba-4a62-ae36-42bc1f9b7610 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:09:14.559317 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.objects.instance [None req-736572fd-b416-4a3a-851b-af8df0cdcfe5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lazy-loading 'services' on Instance uuid 03969ac5-934e-4201-8087-0927eae2bbd2 {{(pid=120524) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:09:24.529803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=2 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:09:24.559285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=5,nova_cell1:INSERT=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:09:24.576722 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=4 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:10:43.555501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.objects.instance [None req-1b91f5b1-c0ba-4a62-ae36-42bc1f9b7610 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lazy-loading 'services' on Instance uuid 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc {{(pid=120525) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:10:43.699668 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-736572fd-b416-4a3a-851b-af8df0cdcfe5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-bd054a81-2dd0-479b-8092-986959d2d415) Aug 28 01:10:43.701766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-736572fd-b416-4a3a-851b-af8df0cdcfe5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Request method failure captured: Aug 28 01:10:43.701766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c HTTP/1.1 Aug 28 01:10:43.701766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:10:43.701766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:10:43.701766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:10:43.701766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:10:43.701766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:10:43.701766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:10:43.701766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:10:43.701766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:10:43.701766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCrzWcuiPK_0ynCrragXxXd_fRw7lHCmbNVMAqr4NhyopxGAZIiQrt0m3y3NawnXLWqyOVM4YbBc-yLRrJ4cXICU0MsT01bsu6VzxaCxGd6JRCChv8LlCV7kFwwyy7Ffwd9JpYuFLguuT-xhEcVlfMIZxgSYB22XTBbtvAeHM_AtK1Xg Aug 28 01:10:43.701766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:10:43.701766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:10:43.701766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:10:43.701766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:10:43.701766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:10:43.701766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:10:43.701766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 01:10:43.701766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:10:43.701766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:10:43.701766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:10:43.703168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_6f3acee4af62483ab1f2cbf3a47b09e1", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/6f3acee4af62483ab1f2cbf3a47b09e1"}]}] Aug 28 01:10:43.703168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:10:43.703168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 01:10:43.703168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:10:43.703168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 01:10:43.703168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:10:43.703168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:10:43.703168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: 1ab2fcb3759045c7a076e5c8222a421d Aug 28 01:10:43.703168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 01:10:43.703168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:10:43.703922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-bd054a81-2dd0-479b-8092-986959d2d415) Aug 28 01:10:43.703922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'server_id': '03969ac5-934e-4201-8087-0927eae2bbd2', 'id': '6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c'} Aug 28 01:10:43.703922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:10:43.703922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:10:43.703922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:10:43.703922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:10:43.703922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:10:43.703922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:10:43.703922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:10:43.703922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:10:43.703922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:10:43.703922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:10:43.703922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:10:43.703922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:10:43.703922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:10:43.703922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:10:43.703922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:10:43.705061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:10:43.705061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:10:43.705061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:10:43.705061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-bd054a81-2dd0-479b-8092-986959d2d415) Aug 28 01:10:43.705061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:10:43.705061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:10:43.705061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:10:43.705061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:10:43.705061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5192, in _detach_volume Aug 28 01:10:43.705061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 28 01:10:43.705061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 28 01:10:43.705061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 28 01:10:43.705061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 28 01:10:43.705061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 28 01:10:43.705061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:10:43.705061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:10:43.705061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:10:43.705061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:10:43.705061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:10:43.707654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:10:43.707654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:10:43.707654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:10:43.707654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:10:43.707654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:10:43.707654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:10:43.707654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:10:43.707654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:10:43.707654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:10:43.707654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:10:43.707654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:10:43.707654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-bd054a81-2dd0-479b-8092-986959d2d415) Aug 28 01:10:43.707654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:10:43.707654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:10:43.707654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:10:43.707654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:10:43.707654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 28 01:10:43.710342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 28 01:10:43.710342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Aug 28 01:10:43.710342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:10:43.710342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Aug 28 01:10:43.710342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 28 01:10:43.710342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5214, in detach_volume Aug 28 01:10:43.710342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 28 01:10:43.710342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Aug 28 01:10:43.710342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:10:43.710342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Aug 28 01:10:43.710342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 28 01:10:43.710342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-bd054a81-2dd0-479b-8092-986959d2d415) Aug 28 01:10:43.710342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:10:43.710342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:10:43.710342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:10:43.710342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:10:43.710342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:10:43.710342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:10:43.713662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:10:43.713662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:10:43.713662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:10:43.713662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:10:43.713662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 28 01:10:43.713662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 28 01:10:43.713662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-bd054a81-2dd0-479b-8092-986959d2d415) Aug 28 01:10:43.713662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:10:43.713662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:10:43.713662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:10:43.713662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:10:43.713662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:10:43.713662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:10:43.713662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:10:43.713662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:10:43.713662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-bd054a81-2dd0-479b-8092-986959d2d415) Aug 28 01:10:43.713662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:10:43.715362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-736572fd-b416-4a3a-851b-af8df0cdcfe5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-bd054a81-2dd0-479b-8092-986959d2d415) {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:10:43.715362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-736572fd-b416-4a3a-851b-af8df0cdcfe5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c" status: 400 len: 269 microversion: 2.1 time: 120.324040 Aug 28 01:10:43.715362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 470/1168] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Aug 28 01:08:43 2022] DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c => generated 269 bytes in 120325 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 28 01:10:43.715362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-130df746-ebbe-4a37-b16f-cc600d6bac91 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:10:43.715908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-130df746-ebbe-4a37-b16f-cc600d6bac91 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:10:43.717253 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-130df746-ebbe-4a37-b16f-cc600d6bac91 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:10:43.717803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-130df746-ebbe-4a37-b16f-cc600d6bac91 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:10:43.718185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-130df746-ebbe-4a37-b16f-cc600d6bac91 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:10:43.718895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-130df746-ebbe-4a37-b16f-cc600d6bac91 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:10:43.719343 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-130df746-ebbe-4a37-b16f-cc600d6bac91 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:10:43.719766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-130df746-ebbe-4a37-b16f-cc600d6bac91 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:10:43.746224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-1b91f5b1-c0ba-4a62-ae36-42bc1f9b7610 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-93e145da-01e5-4bdd-8fac-9684a1833348) Aug 28 01:10:43.749497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1b91f5b1-c0ba-4a62-ae36-42bc1f9b7610 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Request method failure captured: Aug 28 01:10:43.749497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: DELETE /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc/os-volume_attachments/7421938a-2d8f-4ae7-aac6-d08ec2a49354 HTTP/1.1 Aug 28 01:10:43.749497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:10:43.749497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:10:43.749497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:10:43.749497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:10:43.749497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:10:43.749497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:10:43.749497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:10:43.749497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:10:43.749497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCr23GaLQK8ziOdxVoCT0iJXGNnlukskDiaMFr5AAKY331s8vfxacqTI7dtr7vpNU3wmdlDrilDGQFSyuZY_hpQUhjunt7QszGbfuKj3y4RoNxch4hqMcqAU1XqZGWsNvn9naLr5gZpK2pS-XC7OQujv_HpFmzTNKexPx09BbB4-E_7E Aug 28 01:10:43.749497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:10:43.749497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:10:43.749497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:10:43.749497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:10:43.749497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:10:43.749497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:10:43.749497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 89dfee143ed147abb6f2339958412e7e Aug 28 01:10:43.749497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1940777549 Aug 28 01:10:43.749497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:10:43.749497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:10:43.751939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_89dfee143ed147abb6f2339958412e7e", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/89dfee143ed147abb6f2339958412e7e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/89dfee143ed147abb6f2339958412e7e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/89dfee143ed147abb6f2339958412e7e"}]}] Aug 28 01:10:43.751939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-VolumesSnapshotTestJSON-1940777549 Aug 28 01:10:43.751939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 89dfee143ed147abb6f2339958412e7e Aug 28 01:10:43.751939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1940777549 Aug 28 01:10:43.751939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-VolumesSnapshotTestJSON-1940777549-project Aug 28 01:10:43.751939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:10:43.751939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:10:43.751939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: 93ee4e8157364093bc9cdbe980bde4bb Aug 28 01:10:43.751939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-VolumesSnapshotTestJSON-1940777549-project Aug 28 01:10:43.751939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 01:10:43.753448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-93e145da-01e5-4bdd-8fac-9684a1833348) Aug 28 01:10:43.753448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'server_id': '2cbc0806-fe4b-4018-93bf-b4cf3c292bdc', 'id': '7421938a-2d8f-4ae7-aac6-d08ec2a49354'} Aug 28 01:10:43.753448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:10:43.753448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:10:43.753448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:10:43.753448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:10:43.753448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:10:43.753448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:10:43.753448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:10:43.753448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:10:43.753448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:10:43.753448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:10:43.753448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:10:43.753448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:10:43.753448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:10:43.753448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:10:43.753448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:10:43.755573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:10:43.755573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:10:43.755573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:10:43.755573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-93e145da-01e5-4bdd-8fac-9684a1833348) Aug 28 01:10:43.755573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:10:43.755573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:10:43.755573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:10:43.755573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:10:43.755573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5192, in _detach_volume Aug 28 01:10:43.755573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 28 01:10:43.755573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 28 01:10:43.755573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 28 01:10:43.755573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 28 01:10:43.755573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 28 01:10:43.755573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:10:43.755573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:10:43.755573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:10:43.755573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:10:43.755573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:10:43.758923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:10:43.758923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:10:43.758923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:10:43.758923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:10:43.758923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:10:43.758923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:10:43.758923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:10:43.758923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:10:43.758923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:10:43.758923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:10:43.758923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:10:43.758923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-93e145da-01e5-4bdd-8fac-9684a1833348) Aug 28 01:10:43.758923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:10:43.758923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:10:43.758923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:10:43.758923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:10:43.758923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 28 01:10:43.762608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 28 01:10:43.762608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Aug 28 01:10:43.762608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:10:43.762608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Aug 28 01:10:43.762608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 28 01:10:43.762608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5214, in detach_volume Aug 28 01:10:43.762608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 28 01:10:43.762608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Aug 28 01:10:43.762608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:10:43.762608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Aug 28 01:10:43.762608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 28 01:10:43.762608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-93e145da-01e5-4bdd-8fac-9684a1833348) Aug 28 01:10:43.762608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:10:43.762608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:10:43.762608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:10:43.762608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:10:43.762608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:10:43.762608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:10:43.765948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:10:43.765948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:10:43.765948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:10:43.765948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:10:43.765948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 28 01:10:43.765948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 28 01:10:43.765948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-93e145da-01e5-4bdd-8fac-9684a1833348) Aug 28 01:10:43.765948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:10:43.765948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:10:43.765948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:10:43.765948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:10:43.765948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:10:43.765948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:10:43.765948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:10:43.765948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:10:43.765948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-93e145da-01e5-4bdd-8fac-9684a1833348) Aug 28 01:10:43.765948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:10:43.769838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1b91f5b1-c0ba-4a62-ae36-42bc1f9b7610 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-93e145da-01e5-4bdd-8fac-9684a1833348) {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:10:43.769838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1b91f5b1-c0ba-4a62-ae36-42bc1f9b7610 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "DELETE /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc/os-volume_attachments/7421938a-2d8f-4ae7-aac6-d08ec2a49354" status: 400 len: 269 microversion: 2.1 time: 89.231018 Aug 28 01:10:43.769838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 699/1169] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Aug 28 01:09:14 2022] DELETE /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc/os-volume_attachments/7421938a-2d8f-4ae7-aac6-d08ec2a49354 => generated 269 bytes in 89232 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 28 01:10:43.769838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1309dbb4-17bb-4101-806d-30b791915fd3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:10:43.770711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1309dbb4-17bb-4101-806d-30b791915fd3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:10:43.775299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1309dbb4-17bb-4101-806d-30b791915fd3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:10:43.775797 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1309dbb4-17bb-4101-806d-30b791915fd3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:10:43.776241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1309dbb4-17bb-4101-806d-30b791915fd3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:10:43.777804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1309dbb4-17bb-4101-806d-30b791915fd3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:10:43.778259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1309dbb4-17bb-4101-806d-30b791915fd3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:10:43.778685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1309dbb4-17bb-4101-806d-30b791915fd3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:10:43.812853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1309dbb4-17bb-4101-806d-30b791915fd3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.056255 Aug 28 01:10:43.812853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 700/1170] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:10:43 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:10:43.815035 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-130df746-ebbe-4a37-b16f-cc600d6bac91 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:10:43.825958 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-130df746-ebbe-4a37-b16f-cc600d6bac91 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1225 microversion: 2.1 time: 0.121141 Aug 28 01:10:43.826552 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 471/1171] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:10:43 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1225 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:10:43.831416 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-629e4fde-875e-45a9-82c8-8cf591429467 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:10:43.833644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-629e4fde-875e-45a9-82c8-8cf591429467 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:10:43.841353 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-629e4fde-875e-45a9-82c8-8cf591429467 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:10:43.841841 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-629e4fde-875e-45a9-82c8-8cf591429467 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:10:43.842263 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-629e4fde-875e-45a9-82c8-8cf591429467 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:10:43.842995 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-629e4fde-875e-45a9-82c8-8cf591429467 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:10:43.843438 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-629e4fde-875e-45a9-82c8-8cf591429467 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:10:43.843857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-629e4fde-875e-45a9-82c8-8cf591429467 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:10:43.879482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-629e4fde-875e-45a9-82c8-8cf591429467 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051857 Aug 28 01:10:43.880371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 472/1172] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:10:43 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:10:43.885415 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ca15a805-ef0f-4c6b-9265-ff7d3f8b2313 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:10:43.886138 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ca15a805-ef0f-4c6b-9265-ff7d3f8b2313 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:10:43.892745 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ca15a805-ef0f-4c6b-9265-ff7d3f8b2313 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:10:43.893259 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ca15a805-ef0f-4c6b-9265-ff7d3f8b2313 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:10:43.893688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ca15a805-ef0f-4c6b-9265-ff7d3f8b2313 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:10:43.894384 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ca15a805-ef0f-4c6b-9265-ff7d3f8b2313 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:10:43.894829 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ca15a805-ef0f-4c6b-9265-ff7d3f8b2313 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:10:43.895239 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ca15a805-ef0f-4c6b-9265-ff7d3f8b2313 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:10:43.964313 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-760ffc04-387c-423c-a5f2-870b5a1d4c0a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:10:43.966652 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-760ffc04-387c-423c-a5f2-870b5a1d4c0a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:10:43.973301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-760ffc04-387c-423c-a5f2-870b5a1d4c0a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:10:43.973794 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-760ffc04-387c-423c-a5f2-870b5a1d4c0a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:10:43.974259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-760ffc04-387c-423c-a5f2-870b5a1d4c0a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:10:43.975055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-760ffc04-387c-423c-a5f2-870b5a1d4c0a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:10:43.975538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-760ffc04-387c-423c-a5f2-870b5a1d4c0a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:10:43.975991 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-760ffc04-387c-423c-a5f2-870b5a1d4c0a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:10:44.009584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-760ffc04-387c-423c-a5f2-870b5a1d4c0a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.195545 Aug 28 01:10:44.010178 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 701/1173] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:10:43 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:10:44.015245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e9e16c0d-29c4-4f97-aa90-3729f691c8f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:10:44.015732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e9e16c0d-29c4-4f97-aa90-3729f691c8f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:10:44.021975 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e9e16c0d-29c4-4f97-aa90-3729f691c8f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:10:44.022473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e9e16c0d-29c4-4f97-aa90-3729f691c8f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:10:44.022902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e9e16c0d-29c4-4f97-aa90-3729f691c8f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:10:44.023613 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e9e16c0d-29c4-4f97-aa90-3729f691c8f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:10:44.024050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e9e16c0d-29c4-4f97-aa90-3729f691c8f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:10:44.024457 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e9e16c0d-29c4-4f97-aa90-3729f691c8f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:10:53.890526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=6 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:10:53.915447 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=16 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:10:54.020011 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=6 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:10:54.043775 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=20 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:11:14.638558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.objects.instance [None req-ca15a805-ef0f-4c6b-9265-ff7d3f8b2313 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lazy-loading 'services' on Instance uuid 03969ac5-934e-4201-8087-0927eae2bbd2 {{(pid=120524) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:11:14.683224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.objects.instance [None req-e9e16c0d-29c4-4f97-aa90-3729f691c8f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lazy-loading 'services' on Instance uuid 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc {{(pid=120525) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:11:14.720736 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-ca15a805-ef0f-4c6b-9265-ff7d3f8b2313 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f86f185-6b8a-44a8-9d21-be00b3f2d605) Aug 28 01:11:14.722977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ca15a805-ef0f-4c6b-9265-ff7d3f8b2313 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Request method failure captured: Aug 28 01:11:14.722977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c HTTP/1.1 Aug 28 01:11:14.722977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:11:14.722977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:11:14.722977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:11:14.722977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:11:14.722977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:11:14.722977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:11:14.722977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:11:14.722977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:11:14.722977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCrzWcuiPK_0ynCrragXxXd_fRw7lHCmbNVMAqr4NhyopxGAZIiQrt0m3y3NawnXLWqyOVM4YbBc-yLRrJ4cXICU0MsT01bsu6VzxaCxGd6JRCChv8LlCV7kFwwyy7Ffwd9JpYuFLguuT-xhEcVlfMIZxgSYB22XTBbtvAeHM_AtK1Xg Aug 28 01:11:14.722977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:11:14.722977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:11:14.722977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:11:14.722977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:11:14.722977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:11:14.722977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:11:14.722977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 01:11:14.722977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:11:14.722977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:11:14.722977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:11:14.724423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_6f3acee4af62483ab1f2cbf3a47b09e1", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/6f3acee4af62483ab1f2cbf3a47b09e1"}]}] Aug 28 01:11:14.724423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:11:14.724423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 01:11:14.724423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:11:14.724423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 01:11:14.724423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:11:14.724423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:11:14.724423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: 1ab2fcb3759045c7a076e5c8222a421d Aug 28 01:11:14.724423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 01:11:14.724423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:11:14.725336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f86f185-6b8a-44a8-9d21-be00b3f2d605) Aug 28 01:11:14.725336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'server_id': '03969ac5-934e-4201-8087-0927eae2bbd2', 'id': '6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c'} Aug 28 01:11:14.725336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:11:14.725336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:14.725336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:11:14.725336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:11:14.725336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:11:14.725336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:11:14.725336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:11:14.725336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:11:14.725336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:11:14.725336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:11:14.725336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:11:14.725336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:11:14.725336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:11:14.725336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:11:14.725336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:11:14.726443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:11:14.726443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:11:14.726443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:11:14.726443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f86f185-6b8a-44a8-9d21-be00b3f2d605) Aug 28 01:11:14.726443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:14.726443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:14.726443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:14.726443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:14.726443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5192, in _detach_volume Aug 28 01:11:14.726443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 28 01:11:14.726443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 28 01:11:14.726443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 28 01:11:14.726443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 28 01:11:14.726443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 28 01:11:14.726443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:11:14.726443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:11:14.726443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:11:14.726443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:11:14.726443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:11:14.729168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:11:14.729168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:11:14.729168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:11:14.729168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:11:14.729168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:11:14.729168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:11:14.729168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:11:14.729168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:11:14.729168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:11:14.729168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:11:14.729168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:11:14.729168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f86f185-6b8a-44a8-9d21-be00b3f2d605) Aug 28 01:11:14.729168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:14.729168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:14.729168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:14.729168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:14.729168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 28 01:11:14.731108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 28 01:11:14.731108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Aug 28 01:11:14.731108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:11:14.731108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Aug 28 01:11:14.731108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 28 01:11:14.731108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5214, in detach_volume Aug 28 01:11:14.731108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 28 01:11:14.731108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Aug 28 01:11:14.731108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:11:14.731108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Aug 28 01:11:14.731108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 28 01:11:14.731108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f86f185-6b8a-44a8-9d21-be00b3f2d605) Aug 28 01:11:14.731108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:14.731108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:14.731108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:14.731108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:14.731108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:11:14.731108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:11:14.732480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:11:14.732480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:11:14.732480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:11:14.732480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:11:14.732480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 28 01:11:14.732480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 28 01:11:14.732480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f86f185-6b8a-44a8-9d21-be00b3f2d605) Aug 28 01:11:14.732480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:14.732480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:14.732480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:14.732480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:14.732480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:11:14.732480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:11:14.732480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:11:14.732480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:11:14.732480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f86f185-6b8a-44a8-9d21-be00b3f2d605) Aug 28 01:11:14.732480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:14.733741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ca15a805-ef0f-4c6b-9265-ff7d3f8b2313 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9f86f185-6b8a-44a8-9d21-be00b3f2d605) {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:11:14.733741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ca15a805-ef0f-4c6b-9265-ff7d3f8b2313 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c" status: 400 len: 269 microversion: 2.1 time: 30.843656 Aug 28 01:11:14.733741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 473/1174] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Aug 28 01:10:43 2022] DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c => generated 269 bytes in 30845 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 28 01:11:14.733741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-292c0088-442a-4982-8fe2-5bc523c3696c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:14.734094 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-292c0088-442a-4982-8fe2-5bc523c3696c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:14.740782 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-292c0088-442a-4982-8fe2-5bc523c3696c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:14.741320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-292c0088-442a-4982-8fe2-5bc523c3696c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:14.741755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-292c0088-442a-4982-8fe2-5bc523c3696c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:14.742460 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-292c0088-442a-4982-8fe2-5bc523c3696c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:14.743170 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-292c0088-442a-4982-8fe2-5bc523c3696c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:14.743593 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-292c0088-442a-4982-8fe2-5bc523c3696c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:14.764461 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-e9e16c0d-29c4-4f97-aa90-3729f691c8f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b6338c79-a46e-4d4b-8cbc-47a744ccacef) Aug 28 01:11:14.766716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e9e16c0d-29c4-4f97-aa90-3729f691c8f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Request method failure captured: Aug 28 01:11:14.766716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: DELETE /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc/os-volume_attachments/7421938a-2d8f-4ae7-aac6-d08ec2a49354 HTTP/1.1 Aug 28 01:11:14.766716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:11:14.766716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:11:14.766716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:11:14.766716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:11:14.766716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:11:14.766716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:11:14.766716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:11:14.766716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:11:14.766716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCr23GaLQK8ziOdxVoCT0iJXGNnlukskDiaMFr5AAKY331s8vfxacqTI7dtr7vpNU3wmdlDrilDGQFSyuZY_hpQUhjunt7QszGbfuKj3y4RoNxch4hqMcqAU1XqZGWsNvn9naLr5gZpK2pS-XC7OQujv_HpFmzTNKexPx09BbB4-E_7E Aug 28 01:11:14.766716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:11:14.766716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:11:14.766716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:11:14.766716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:11:14.766716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:11:14.766716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:11:14.766716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 89dfee143ed147abb6f2339958412e7e Aug 28 01:11:14.766716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1940777549 Aug 28 01:11:14.766716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:11:14.766716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:11:14.768063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_89dfee143ed147abb6f2339958412e7e", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/89dfee143ed147abb6f2339958412e7e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/89dfee143ed147abb6f2339958412e7e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/89dfee143ed147abb6f2339958412e7e"}]}] Aug 28 01:11:14.768063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-VolumesSnapshotTestJSON-1940777549 Aug 28 01:11:14.768063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 89dfee143ed147abb6f2339958412e7e Aug 28 01:11:14.768063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1940777549 Aug 28 01:11:14.768063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-VolumesSnapshotTestJSON-1940777549-project Aug 28 01:11:14.768063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:11:14.768063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:11:14.768063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: 93ee4e8157364093bc9cdbe980bde4bb Aug 28 01:11:14.768063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-VolumesSnapshotTestJSON-1940777549-project Aug 28 01:11:14.768063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 01:11:14.768731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b6338c79-a46e-4d4b-8cbc-47a744ccacef) Aug 28 01:11:14.768731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'server_id': '2cbc0806-fe4b-4018-93bf-b4cf3c292bdc', 'id': '7421938a-2d8f-4ae7-aac6-d08ec2a49354'} Aug 28 01:11:14.768731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:11:14.768731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:14.768731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:11:14.768731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:11:14.768731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:11:14.768731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:11:14.768731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:11:14.768731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:11:14.768731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:11:14.768731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:11:14.768731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:11:14.768731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:11:14.768731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:11:14.768731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:11:14.768731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:11:14.770479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:11:14.770479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:11:14.770479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:11:14.770479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b6338c79-a46e-4d4b-8cbc-47a744ccacef) Aug 28 01:11:14.770479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:14.770479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:14.770479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:14.770479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:14.770479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5192, in _detach_volume Aug 28 01:11:14.770479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 28 01:11:14.770479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 28 01:11:14.770479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 28 01:11:14.770479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 28 01:11:14.770479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 28 01:11:14.770479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:11:14.770479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:11:14.770479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:11:14.770479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:11:14.770479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:11:14.772719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:11:14.772719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:11:14.772719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:11:14.772719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:11:14.772719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:11:14.772719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:11:14.772719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:11:14.772719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:11:14.772719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:11:14.772719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:11:14.772719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:11:14.772719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b6338c79-a46e-4d4b-8cbc-47a744ccacef) Aug 28 01:11:14.772719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:14.772719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:14.772719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:14.772719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:14.772719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 28 01:11:14.774721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 28 01:11:14.774721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Aug 28 01:11:14.774721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:11:14.774721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Aug 28 01:11:14.774721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 28 01:11:14.774721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5214, in detach_volume Aug 28 01:11:14.774721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 28 01:11:14.774721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Aug 28 01:11:14.774721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:11:14.774721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Aug 28 01:11:14.774721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 28 01:11:14.774721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b6338c79-a46e-4d4b-8cbc-47a744ccacef) Aug 28 01:11:14.774721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:14.774721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:14.774721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:14.774721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:14.774721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:11:14.774721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:11:14.776615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:11:14.776615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:11:14.776615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:11:14.776615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:11:14.776615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 28 01:11:14.776615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 28 01:11:14.776615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b6338c79-a46e-4d4b-8cbc-47a744ccacef) Aug 28 01:11:14.776615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:14.776615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:14.776615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:14.776615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:14.776615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:11:14.776615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:11:14.776615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:11:14.776615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:11:14.776615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b6338c79-a46e-4d4b-8cbc-47a744ccacef) Aug 28 01:11:14.776615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:14.778661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e9e16c0d-29c4-4f97-aa90-3729f691c8f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b6338c79-a46e-4d4b-8cbc-47a744ccacef) {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:11:14.778661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e9e16c0d-29c4-4f97-aa90-3729f691c8f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "DELETE /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc/os-volume_attachments/7421938a-2d8f-4ae7-aac6-d08ec2a49354" status: 400 len: 269 microversion: 2.1 time: 30.757311 Aug 28 01:11:14.778661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 702/1175] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Aug 28 01:10:44 2022] DELETE /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc/os-volume_attachments/7421938a-2d8f-4ae7-aac6-d08ec2a49354 => generated 269 bytes in 30759 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 28 01:11:14.778661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-855fbbe5-48bf-4c72-9a7e-d649e7e6f4c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:14.779133 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-855fbbe5-48bf-4c72-9a7e-d649e7e6f4c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:14.779231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-292c0088-442a-4982-8fe2-5bc523c3696c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:14.779231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-292c0088-442a-4982-8fe2-5bc523c3696c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:14.779414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-292c0088-442a-4982-8fe2-5bc523c3696c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:14.779749 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.objects.instance [None req-292c0088-442a-4982-8fe2-5bc523c3696c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lazy-loading 'fault' on Instance uuid 1cc5e59c-500c-421e-920c-c08b8ae3a206 {{(pid=120524) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:11:14.783278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-855fbbe5-48bf-4c72-9a7e-d649e7e6f4c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:14.784556 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-855fbbe5-48bf-4c72-9a7e-d649e7e6f4c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:14.784857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-855fbbe5-48bf-4c72-9a7e-d649e7e6f4c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:14.785830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-855fbbe5-48bf-4c72-9a7e-d649e7e6f4c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:14.786501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-855fbbe5-48bf-4c72-9a7e-d649e7e6f4c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:14.787141 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-855fbbe5-48bf-4c72-9a7e-d649e7e6f4c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:14.820824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-855fbbe5-48bf-4c72-9a7e-d649e7e6f4c2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049973 Aug 28 01:11:14.821449 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 703/1176] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:11:14 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:11:14.826312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-292c0088-442a-4982-8fe2-5bc523c3696c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:11:14.827382 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ee21da57-63bd-49ed-b846-91c676e1dd1c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:14.829674 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ee21da57-63bd-49ed-b846-91c676e1dd1c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:14.834933 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-292c0088-442a-4982-8fe2-5bc523c3696c tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1230 microversion: 2.1 time: 0.108653 Aug 28 01:11:14.835753 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 474/1177] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:11:14 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1230 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:11:14.835908 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ee21da57-63bd-49ed-b846-91c676e1dd1c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:14.836711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ee21da57-63bd-49ed-b846-91c676e1dd1c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:14.837550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ee21da57-63bd-49ed-b846-91c676e1dd1c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:14.838569 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ee21da57-63bd-49ed-b846-91c676e1dd1c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:14.839184 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ee21da57-63bd-49ed-b846-91c676e1dd1c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:14.839966 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ee21da57-63bd-49ed-b846-91c676e1dd1c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:14.840714 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-96641a1d-4f40-4512-a1ce-2588fc8e7680 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:14.842955 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-96641a1d-4f40-4512-a1ce-2588fc8e7680 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:14.849137 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96641a1d-4f40-4512-a1ce-2588fc8e7680 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:14.849879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96641a1d-4f40-4512-a1ce-2588fc8e7680 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:14.850383 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96641a1d-4f40-4512-a1ce-2588fc8e7680 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:14.851792 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96641a1d-4f40-4512-a1ce-2588fc8e7680 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:14.852333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96641a1d-4f40-4512-a1ce-2588fc8e7680 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:14.852764 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96641a1d-4f40-4512-a1ce-2588fc8e7680 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:14.876172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ee21da57-63bd-49ed-b846-91c676e1dd1c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053671 Aug 28 01:11:14.876739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 704/1178] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:14 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:14.881833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fcb85d85-88d6-4537-9fc0-e2e610c4cd1a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:14.882326 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fcb85d85-88d6-4537-9fc0-e2e610c4cd1a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:14.886011 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-96641a1d-4f40-4512-a1ce-2588fc8e7680 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049135 Aug 28 01:11:14.886988 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 475/1179] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:14 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:14.888830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fcb85d85-88d6-4537-9fc0-e2e610c4cd1a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:14.889357 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fcb85d85-88d6-4537-9fc0-e2e610c4cd1a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:14.889794 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fcb85d85-88d6-4537-9fc0-e2e610c4cd1a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:14.890492 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fcb85d85-88d6-4537-9fc0-e2e610c4cd1a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:14.890942 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fcb85d85-88d6-4537-9fc0-e2e610c4cd1a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:14.891360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fcb85d85-88d6-4537-9fc0-e2e610c4cd1a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:14.892672 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-25847df8-8fcc-4a1a-b015-4f82a359b701 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:14.893197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-25847df8-8fcc-4a1a-b015-4f82a359b701 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:14.899209 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-25847df8-8fcc-4a1a-b015-4f82a359b701 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:14.899781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-25847df8-8fcc-4a1a-b015-4f82a359b701 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:14.900120 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-25847df8-8fcc-4a1a-b015-4f82a359b701 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:14.900852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-25847df8-8fcc-4a1a-b015-4f82a359b701 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:14.901338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-25847df8-8fcc-4a1a-b015-4f82a359b701 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:14.901769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-25847df8-8fcc-4a1a-b015-4f82a359b701 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:14.982608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.objects.instance [None req-fcb85d85-88d6-4537-9fc0-e2e610c4cd1a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lazy-loading 'services' on Instance uuid 03969ac5-934e-4201-8087-0927eae2bbd2 {{(pid=120525) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:11:15.021495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.objects.instance [None req-25847df8-8fcc-4a1a-b015-4f82a359b701 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lazy-loading 'services' on Instance uuid 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc {{(pid=120524) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:11:15.050913 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-fcb85d85-88d6-4537-9fc0-e2e610c4cd1a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1187c409-e6a8-48ca-a6f9-e1581746fbb5) Aug 28 01:11:15.053254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fcb85d85-88d6-4537-9fc0-e2e610c4cd1a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Request method failure captured: Aug 28 01:11:15.053254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c HTTP/1.1 Aug 28 01:11:15.053254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:11:15.053254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:11:15.053254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:11:15.053254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:11:15.053254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:11:15.053254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:11:15.053254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:11:15.053254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:11:15.053254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCrzWcuiPK_0ynCrragXxXd_fRw7lHCmbNVMAqr4NhyopxGAZIiQrt0m3y3NawnXLWqyOVM4YbBc-yLRrJ4cXICU0MsT01bsu6VzxaCxGd6JRCChv8LlCV7kFwwyy7Ffwd9JpYuFLguuT-xhEcVlfMIZxgSYB22XTBbtvAeHM_AtK1Xg Aug 28 01:11:15.053254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:11:15.053254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:11:15.053254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:11:15.053254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:11:15.053254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:11:15.053254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:11:15.053254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 01:11:15.053254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:11:15.053254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:11:15.053254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:11:15.054831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_6f3acee4af62483ab1f2cbf3a47b09e1", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/6f3acee4af62483ab1f2cbf3a47b09e1"}]}] Aug 28 01:11:15.054831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:11:15.054831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 01:11:15.054831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:11:15.054831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 01:11:15.054831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:11:15.054831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:11:15.054831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: 1ab2fcb3759045c7a076e5c8222a421d Aug 28 01:11:15.054831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 01:11:15.054831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 01:11:15.055768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1187c409-e6a8-48ca-a6f9-e1581746fbb5) Aug 28 01:11:15.055768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'server_id': '03969ac5-934e-4201-8087-0927eae2bbd2', 'id': '6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c'} Aug 28 01:11:15.055768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:11:15.055768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.055768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:11:15.055768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:11:15.055768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:11:15.055768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:11:15.055768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:11:15.055768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:11:15.055768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:11:15.055768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:11:15.055768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:11:15.055768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:11:15.055768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:11:15.055768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:11:15.055768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:11:15.056816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:11:15.056816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:11:15.056816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:11:15.056816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1187c409-e6a8-48ca-a6f9-e1581746fbb5) Aug 28 01:11:15.056816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.056816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:15.056816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.056816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.056816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5192, in _detach_volume Aug 28 01:11:15.056816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 28 01:11:15.056816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 28 01:11:15.056816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 28 01:11:15.056816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 28 01:11:15.056816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 28 01:11:15.056816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:11:15.056816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:11:15.056816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:11:15.056816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:11:15.056816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:11:15.057977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:11:15.057977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:11:15.057977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:11:15.057977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:11:15.057977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:11:15.057977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:11:15.057977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:11:15.057977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:11:15.057977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:11:15.057977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:11:15.057977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:11:15.057977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1187c409-e6a8-48ca-a6f9-e1581746fbb5) Aug 28 01:11:15.057977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.057977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:15.057977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.057977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.057977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 28 01:11:15.058960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 28 01:11:15.058960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Aug 28 01:11:15.058960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:11:15.058960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Aug 28 01:11:15.058960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 28 01:11:15.058960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5214, in detach_volume Aug 28 01:11:15.058960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 28 01:11:15.058960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Aug 28 01:11:15.058960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:11:15.058960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Aug 28 01:11:15.058960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 28 01:11:15.058960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1187c409-e6a8-48ca-a6f9-e1581746fbb5) Aug 28 01:11:15.058960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.058960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:15.058960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.058960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.058960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:11:15.058960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:11:15.060039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:11:15.060039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:11:15.060039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:11:15.060039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:11:15.060039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 28 01:11:15.060039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 28 01:11:15.060039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1187c409-e6a8-48ca-a6f9-e1581746fbb5) Aug 28 01:11:15.060039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.060039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:15.060039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.060039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.060039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:11:15.060039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:11:15.060039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:11:15.060039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:11:15.060039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1187c409-e6a8-48ca-a6f9-e1581746fbb5) Aug 28 01:11:15.060039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.061080 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fcb85d85-88d6-4537-9fc0-e2e610c4cd1a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-1187c409-e6a8-48ca-a6f9-e1581746fbb5) {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:11:15.061080 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fcb85d85-88d6-4537-9fc0-e2e610c4cd1a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c" status: 400 len: 269 microversion: 2.1 time: 0.176857 Aug 28 01:11:15.061080 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 705/1180] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Aug 28 01:11:14 2022] DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c => generated 269 bytes in 178 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 28 01:11:15.061080 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-60c9ac8c-a225-4bdd-be3b-b46be64ec7cc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:15.061405 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-60c9ac8c-a225-4bdd-be3b-b46be64ec7cc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:15.066497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60c9ac8c-a225-4bdd-be3b-b46be64ec7cc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.067005 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60c9ac8c-a225-4bdd-be3b-b46be64ec7cc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.067435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60c9ac8c-a225-4bdd-be3b-b46be64ec7cc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.068139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60c9ac8c-a225-4bdd-be3b-b46be64ec7cc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.068585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60c9ac8c-a225-4bdd-be3b-b46be64ec7cc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.069345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60c9ac8c-a225-4bdd-be3b-b46be64ec7cc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.089781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-25847df8-8fcc-4a1a-b015-4f82a359b701 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ed64e367-b050-4bbb-b494-c48ef94bade2) Aug 28 01:11:15.091928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-25847df8-8fcc-4a1a-b015-4f82a359b701 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Request method failure captured: Aug 28 01:11:15.091928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: DELETE /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc/os-volume_attachments/7421938a-2d8f-4ae7-aac6-d08ec2a49354 HTTP/1.1 Aug 28 01:11:15.091928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:11:15.091928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:11:15.091928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:11:15.091928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:11:15.091928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:11:15.091928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:11:15.091928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:11:15.091928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:11:15.091928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCr23GaLQK8ziOdxVoCT0iJXGNnlukskDiaMFr5AAKY331s8vfxacqTI7dtr7vpNU3wmdlDrilDGQFSyuZY_hpQUhjunt7QszGbfuKj3y4RoNxch4hqMcqAU1XqZGWsNvn9naLr5gZpK2pS-XC7OQujv_HpFmzTNKexPx09BbB4-E_7E Aug 28 01:11:15.091928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:11:15.091928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:11:15.091928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:11:15.091928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:11:15.091928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:11:15.091928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:11:15.091928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 89dfee143ed147abb6f2339958412e7e Aug 28 01:11:15.091928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1940777549 Aug 28 01:11:15.091928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:11:15.091928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:11:15.093581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_89dfee143ed147abb6f2339958412e7e", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/89dfee143ed147abb6f2339958412e7e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/89dfee143ed147abb6f2339958412e7e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/89dfee143ed147abb6f2339958412e7e"}]}] Aug 28 01:11:15.093581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-VolumesSnapshotTestJSON-1940777549 Aug 28 01:11:15.093581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 89dfee143ed147abb6f2339958412e7e Aug 28 01:11:15.093581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1940777549 Aug 28 01:11:15.093581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-VolumesSnapshotTestJSON-1940777549-project Aug 28 01:11:15.093581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:11:15.093581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:11:15.093581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: 93ee4e8157364093bc9cdbe980bde4bb Aug 28 01:11:15.093581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-VolumesSnapshotTestJSON-1940777549-project Aug 28 01:11:15.093581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:11:15.094309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ed64e367-b050-4bbb-b494-c48ef94bade2) Aug 28 01:11:15.094309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'server_id': '2cbc0806-fe4b-4018-93bf-b4cf3c292bdc', 'id': '7421938a-2d8f-4ae7-aac6-d08ec2a49354'} Aug 28 01:11:15.094309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:11:15.094309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.094309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:11:15.094309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:11:15.094309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:11:15.094309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:11:15.094309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:11:15.094309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:11:15.094309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:11:15.094309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:11:15.094309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:11:15.094309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:11:15.094309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:11:15.094309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:11:15.094309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:11:15.095538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:11:15.095538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:11:15.095538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:11:15.095538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ed64e367-b050-4bbb-b494-c48ef94bade2) Aug 28 01:11:15.095538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.095538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:15.095538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.095538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.095538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5192, in _detach_volume Aug 28 01:11:15.095538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 28 01:11:15.095538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 28 01:11:15.095538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 28 01:11:15.095538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 28 01:11:15.095538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 28 01:11:15.095538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:11:15.095538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:11:15.095538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:11:15.095538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:11:15.095538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:11:15.096889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:11:15.096889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:11:15.096889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:11:15.096889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:11:15.096889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:11:15.096889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:11:15.096889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:11:15.096889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:11:15.096889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:11:15.096889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:11:15.096889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:11:15.096889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ed64e367-b050-4bbb-b494-c48ef94bade2) Aug 28 01:11:15.096889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.096889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:15.096889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.096889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.096889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 28 01:11:15.098127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 28 01:11:15.098127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Aug 28 01:11:15.098127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:11:15.098127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Aug 28 01:11:15.098127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 28 01:11:15.098127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5214, in detach_volume Aug 28 01:11:15.098127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 28 01:11:15.098127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Aug 28 01:11:15.098127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:11:15.098127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Aug 28 01:11:15.098127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 28 01:11:15.098127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ed64e367-b050-4bbb-b494-c48ef94bade2) Aug 28 01:11:15.098127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.098127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:15.098127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.098127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.098127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:11:15.098127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:11:15.099628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:11:15.099628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:11:15.099628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:11:15.099628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:11:15.099628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 28 01:11:15.099628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 28 01:11:15.099628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ed64e367-b050-4bbb-b494-c48ef94bade2) Aug 28 01:11:15.099628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.099628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:15.099628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.099628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.099628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:11:15.099628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:11:15.099628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:11:15.099628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:11:15.099628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ed64e367-b050-4bbb-b494-c48ef94bade2) Aug 28 01:11:15.099628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.100888 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-25847df8-8fcc-4a1a-b015-4f82a359b701 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ed64e367-b050-4bbb-b494-c48ef94bade2) {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:11:15.100888 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-25847df8-8fcc-4a1a-b015-4f82a359b701 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "DELETE /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc/os-volume_attachments/7421938a-2d8f-4ae7-aac6-d08ec2a49354" status: 400 len: 269 microversion: 2.1 time: 0.205470 Aug 28 01:11:15.100888 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 476/1181] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Aug 28 01:11:14 2022] DELETE /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc/os-volume_attachments/7421938a-2d8f-4ae7-aac6-d08ec2a49354 => generated 269 bytes in 207 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 28 01:11:15.100888 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-39ff5427-7cde-4764-923a-a6877a5e0705 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:15.101270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-39ff5427-7cde-4764-923a-a6877a5e0705 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:15.106999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-39ff5427-7cde-4764-923a-a6877a5e0705 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.107152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60c9ac8c-a225-4bdd-be3b-b46be64ec7cc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.107482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-39ff5427-7cde-4764-923a-a6877a5e0705 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.107599 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60c9ac8c-a225-4bdd-be3b-b46be64ec7cc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.107917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-39ff5427-7cde-4764-923a-a6877a5e0705 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.108027 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60c9ac8c-a225-4bdd-be3b-b46be64ec7cc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.108469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.objects.instance [None req-60c9ac8c-a225-4bdd-be3b-b46be64ec7cc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lazy-loading 'fault' on Instance uuid 1cc5e59c-500c-421e-920c-c08b8ae3a206 {{(pid=120525) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:11:15.108573 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-39ff5427-7cde-4764-923a-a6877a5e0705 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.109062 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-39ff5427-7cde-4764-923a-a6877a5e0705 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.110536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-39ff5427-7cde-4764-923a-a6877a5e0705 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.147763 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-39ff5427-7cde-4764-923a-a6877a5e0705 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052542 Aug 28 01:11:15.148348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 477/1182] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:11:15 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:11:15.150757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-60c9ac8c-a225-4bdd-be3b-b46be64ec7cc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cac1ab72dcee40b19bcff4dd5a75d82d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287c855ef8e241978a4d1e7cb3c7a16f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:11:15.153148 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-02596111-d565-4f83-8e81-bb8b165d7999 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:15.155284 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-02596111-d565-4f83-8e81-bb8b165d7999 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:15.159007 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-60c9ac8c-a225-4bdd-be3b-b46be64ec7cc tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 200 len: 1230 microversion: 2.1 time: 0.102785 Aug 28 01:11:15.159579 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 706/1183] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:11:15 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 1230 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:11:15.161494 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-02596111-d565-4f83-8e81-bb8b165d7999 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.161977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-02596111-d565-4f83-8e81-bb8b165d7999 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.162429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-02596111-d565-4f83-8e81-bb8b165d7999 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.163118 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-02596111-d565-4f83-8e81-bb8b165d7999 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.163556 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-02596111-d565-4f83-8e81-bb8b165d7999 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.163964 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-02596111-d565-4f83-8e81-bb8b165d7999 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.164182 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1a547cd9-72c4-4257-b441-086a67a7d02e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:15.166519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1a547cd9-72c4-4257-b441-086a67a7d02e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:15.172303 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a547cd9-72c4-4257-b441-086a67a7d02e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.172784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a547cd9-72c4-4257-b441-086a67a7d02e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.173252 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a547cd9-72c4-4257-b441-086a67a7d02e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.173957 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a547cd9-72c4-4257-b441-086a67a7d02e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.174404 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a547cd9-72c4-4257-b441-086a67a7d02e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.174820 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a547cd9-72c4-4257-b441-086a67a7d02e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.195934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-02596111-d565-4f83-8e81-bb8b165d7999 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046323 Aug 28 01:11:15.196488 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 478/1184] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:15 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:15.201400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-091e264b-4157-4639-af23-ba77d36b8849 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:15.201893 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-091e264b-4157-4639-af23-ba77d36b8849 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:15.206638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1a547cd9-72c4-4257-b441-086a67a7d02e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045967 Aug 28 01:11:15.207427 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 707/1185] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:15 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:15.207886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-091e264b-4157-4639-af23-ba77d36b8849 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.208373 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-091e264b-4157-4639-af23-ba77d36b8849 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.208800 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-091e264b-4157-4639-af23-ba77d36b8849 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.209555 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-091e264b-4157-4639-af23-ba77d36b8849 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.210004 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-091e264b-4157-4639-af23-ba77d36b8849 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.210422 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-091e264b-4157-4639-af23-ba77d36b8849 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.211871 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8f288dc2-7ce8-48e8-b21a-99bb45fec8bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:15.212360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8f288dc2-7ce8-48e8-b21a-99bb45fec8bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:15.218481 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f288dc2-7ce8-48e8-b21a-99bb45fec8bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.218946 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f288dc2-7ce8-48e8-b21a-99bb45fec8bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.219372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f288dc2-7ce8-48e8-b21a-99bb45fec8bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.220072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f288dc2-7ce8-48e8-b21a-99bb45fec8bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.220506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f288dc2-7ce8-48e8-b21a-99bb45fec8bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.221006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f288dc2-7ce8-48e8-b21a-99bb45fec8bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.307937 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.objects.instance [None req-091e264b-4157-4639-af23-ba77d36b8849 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lazy-loading 'services' on Instance uuid 03969ac5-934e-4201-8087-0927eae2bbd2 {{(pid=120524) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:11:15.361742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.objects.instance [None req-8f288dc2-7ce8-48e8-b21a-99bb45fec8bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lazy-loading 'services' on Instance uuid 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc {{(pid=120525) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:11:15.391879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-091e264b-4157-4639-af23-ba77d36b8849 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-83e6adcb-a96e-4c34-afff-a30c529bf3f9) Aug 28 01:11:15.394221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-091e264b-4157-4639-af23-ba77d36b8849 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Request method failure captured: Aug 28 01:11:15.394221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c HTTP/1.1 Aug 28 01:11:15.394221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:11:15.394221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:11:15.394221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:11:15.394221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:11:15.394221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:11:15.394221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:11:15.394221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:11:15.394221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:11:15.394221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCrzWcuiPK_0ynCrragXxXd_fRw7lHCmbNVMAqr4NhyopxGAZIiQrt0m3y3NawnXLWqyOVM4YbBc-yLRrJ4cXICU0MsT01bsu6VzxaCxGd6JRCChv8LlCV7kFwwyy7Ffwd9JpYuFLguuT-xhEcVlfMIZxgSYB22XTBbtvAeHM_AtK1Xg Aug 28 01:11:15.394221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:11:15.394221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:11:15.394221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:11:15.394221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:11:15.394221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:11:15.394221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:11:15.394221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 01:11:15.394221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:11:15.394221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:11:15.394221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:11:15.402338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_6f3acee4af62483ab1f2cbf3a47b09e1", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/6f3acee4af62483ab1f2cbf3a47b09e1"}]}] Aug 28 01:11:15.402338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:11:15.402338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 01:11:15.402338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:11:15.402338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 01:11:15.402338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:11:15.402338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:11:15.402338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: 1ab2fcb3759045c7a076e5c8222a421d Aug 28 01:11:15.402338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 01:11:15.402338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:11:15.402957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-83e6adcb-a96e-4c34-afff-a30c529bf3f9) Aug 28 01:11:15.402957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'server_id': '03969ac5-934e-4201-8087-0927eae2bbd2', 'id': '6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c'} Aug 28 01:11:15.402957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:11:15.402957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.402957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:11:15.402957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:11:15.402957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:11:15.402957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:11:15.402957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:11:15.402957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:11:15.402957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:11:15.402957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:11:15.402957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:11:15.402957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:11:15.402957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:11:15.402957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:11:15.402957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:11:15.404917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:11:15.404917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:11:15.404917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:11:15.404917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-83e6adcb-a96e-4c34-afff-a30c529bf3f9) Aug 28 01:11:15.404917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.404917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:15.404917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.404917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.404917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5192, in _detach_volume Aug 28 01:11:15.404917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 28 01:11:15.404917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 28 01:11:15.404917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 28 01:11:15.404917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 28 01:11:15.404917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 28 01:11:15.404917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:11:15.404917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:11:15.404917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:11:15.404917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:11:15.404917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:11:15.407142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:11:15.407142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:11:15.407142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:11:15.407142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:11:15.407142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:11:15.407142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:11:15.407142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:11:15.407142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:11:15.407142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:11:15.407142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:11:15.407142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:11:15.407142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-83e6adcb-a96e-4c34-afff-a30c529bf3f9) Aug 28 01:11:15.407142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.407142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:15.407142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.407142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.407142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 28 01:11:15.409469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 28 01:11:15.409469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Aug 28 01:11:15.409469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:11:15.409469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Aug 28 01:11:15.409469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 28 01:11:15.409469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5214, in detach_volume Aug 28 01:11:15.409469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 28 01:11:15.409469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Aug 28 01:11:15.409469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:11:15.409469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Aug 28 01:11:15.409469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 28 01:11:15.409469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-83e6adcb-a96e-4c34-afff-a30c529bf3f9) Aug 28 01:11:15.409469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.409469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:15.409469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.409469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.409469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:11:15.409469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:11:15.412923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:11:15.412923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:11:15.412923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:11:15.412923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:11:15.412923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 28 01:11:15.412923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 28 01:11:15.412923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-83e6adcb-a96e-4c34-afff-a30c529bf3f9) Aug 28 01:11:15.412923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.412923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:15.412923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.412923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.412923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:11:15.412923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:11:15.412923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:11:15.412923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:11:15.412923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-83e6adcb-a96e-4c34-afff-a30c529bf3f9) Aug 28 01:11:15.412923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.414457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-091e264b-4157-4639-af23-ba77d36b8849 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-83e6adcb-a96e-4c34-afff-a30c529bf3f9) {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:11:15.414457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-091e264b-4157-4639-af23-ba77d36b8849 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c" status: 400 len: 269 microversion: 2.1 time: 0.198391 Aug 28 01:11:15.414457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 479/1186] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Aug 28 01:11:15 2022] DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments/6c52723b-72a2-4cfc-a9b7-f524e1a8dc1c => generated 269 bytes in 199 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 28 01:11:15.414457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-874d7f88-0d85-4e2b-aa20-f1da2aefac8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:15.414908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-874d7f88-0d85-4e2b-aa20-f1da2aefac8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:15.416231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-874d7f88-0d85-4e2b-aa20-f1da2aefac8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.416762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-874d7f88-0d85-4e2b-aa20-f1da2aefac8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.417249 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-874d7f88-0d85-4e2b-aa20-f1da2aefac8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.417934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-874d7f88-0d85-4e2b-aa20-f1da2aefac8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.418368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-874d7f88-0d85-4e2b-aa20-f1da2aefac8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.418853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-874d7f88-0d85-4e2b-aa20-f1da2aefac8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.434274 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-874d7f88-0d85-4e2b-aa20-f1da2aefac8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] HTTP exception thrown: Instance 1cc5e59c-500c-421e-920c-c08b8ae3a206 could not be found. Aug 28 01:11:15.435793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-874d7f88-0d85-4e2b-aa20-f1da2aefac8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Request method failure captured: Aug 28 01:11:15.435793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 HTTP/1.1 Aug 28 01:11:15.435793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:11:15.435793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:11:15.435793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:11:15.435793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:11:15.435793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:11:15.435793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:11:15.435793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:11:15.435793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:11:15.435793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCr1pxHIWV285Zf1gYz6DOmXQ2NMTOVf64lxBBAObx2MPbCdCBY1JwKpzcv5QOx9Y_YlZRIoc7sjAsi2krqODigup6LiCH54F6kQOVjkdNxhzdtOwuAeDHBQK6z6x3RCl6XGegaI-t8IJwZzgBuOc5uYF2CrOv547lV-tG7cEK7alrc0 Aug 28 01:11:15.435793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:11:15.435793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:11:15.435793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:11:15.435793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:11:15.435793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:11:15.435793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:11:15.435793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 287c855ef8e241978a4d1e7cb3c7a16f Aug 28 01:11:15.435793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-TestVolumeBootPattern-1331161477 Aug 28 01:11:15.435793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:11:15.435793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:11:15.437395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_287c855ef8e241978a4d1e7cb3c7a16f", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287c855ef8e241978a4d1e7cb3c7a16f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287c855ef8e241978a4d1e7cb3c7a16f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/287c855ef8e241978a4d1e7cb3c7a16f"}]}] Aug 28 01:11:15.437395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-TestVolumeBootPattern-1331161477 Aug 28 01:11:15.437395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 287c855ef8e241978a4d1e7cb3c7a16f Aug 28 01:11:15.437395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-TestVolumeBootPattern-1331161477 Aug 28 01:11:15.437395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-TestVolumeBootPattern-1331161477-project Aug 28 01:11:15.437395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:11:15.437395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:11:15.437395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: cac1ab72dcee40b19bcff4dd5a75d82d Aug 28 01:11:15.437395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-TestVolumeBootPattern-1331161477-project Aug 28 01:11:15.437395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:11:15.437395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Instance 1cc5e59c-500c-421e-920c-c08b8ae3a206 could not be found. Aug 28 01:11:15.437395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'id': '1cc5e59c-500c-421e-920c-c08b8ae3a206'} Aug 28 01:11:15.438351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:11:15.438351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.438351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:11:15.438351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:11:15.438351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:11:15.438351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:11:15.438351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:11:15.438351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:11:15.438351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:11:15.438351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:11:15.438351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1cc5e59c-500c-421e-920c-c08b8ae3a206 could not be found. Aug 28 01:11:15.438351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.438351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:15.438351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.438351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.438351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:11:15.438351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:11:15.438351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:11:15.438351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:11:15.438351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:11:15.439806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:11:15.439806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 01:11:15.439806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 01:11:15.439806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:11:15.439806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:11:15.439806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:11:15.439806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:11:15.439806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1cc5e59c-500c-421e-920c-c08b8ae3a206 could not be found. Aug 28 01:11:15.439806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.439806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:15.439806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.439806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.439806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:11:15.439806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:11:15.439806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:11:15.439806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:11:15.439806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1cc5e59c-500c-421e-920c-c08b8ae3a206 could not be found. Aug 28 01:11:15.439806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.441148 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-874d7f88-0d85-4e2b-aa20-f1da2aefac8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Returning 404 to user: Instance 1cc5e59c-500c-421e-920c-c08b8ae3a206 could not be found. {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:11:15.441148 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-874d7f88-0d85-4e2b-aa20-f1da2aefac8e tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 404 len: 111 microversion: 2.1 time: 0.039830 Aug 28 01:11:15.441148 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 480/1187] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:11:15 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:11:15.442500 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b782c35c-a72d-4d10-9243-4ad9ac568fba tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:15.445008 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b782c35c-a72d-4d10-9243-4ad9ac568fba tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:15.450410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-8f288dc2-7ce8-48e8-b21a-99bb45fec8bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9635d43a-3381-4941-883a-893e58b0b285) Aug 28 01:11:15.452706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8f288dc2-7ce8-48e8-b21a-99bb45fec8bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Request method failure captured: Aug 28 01:11:15.452706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: DELETE /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc/os-volume_attachments/7421938a-2d8f-4ae7-aac6-d08ec2a49354 HTTP/1.1 Aug 28 01:11:15.452706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:11:15.452706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:11:15.452706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:11:15.452706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:11:15.452706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:11:15.452706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:11:15.452706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:11:15.452706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:11:15.452706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCr23GaLQK8ziOdxVoCT0iJXGNnlukskDiaMFr5AAKY331s8vfxacqTI7dtr7vpNU3wmdlDrilDGQFSyuZY_hpQUhjunt7QszGbfuKj3y4RoNxch4hqMcqAU1XqZGWsNvn9naLr5gZpK2pS-XC7OQujv_HpFmzTNKexPx09BbB4-E_7E Aug 28 01:11:15.452706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:11:15.452706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:11:15.452706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:11:15.452706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:11:15.452706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:11:15.452706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:11:15.452706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 89dfee143ed147abb6f2339958412e7e Aug 28 01:11:15.452706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1940777549 Aug 28 01:11:15.452706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:11:15.452706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:11:15.454819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_89dfee143ed147abb6f2339958412e7e", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/89dfee143ed147abb6f2339958412e7e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/89dfee143ed147abb6f2339958412e7e"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/89dfee143ed147abb6f2339958412e7e"}]}] Aug 28 01:11:15.454819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-VolumesSnapshotTestJSON-1940777549 Aug 28 01:11:15.454819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 89dfee143ed147abb6f2339958412e7e Aug 28 01:11:15.454819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1940777549 Aug 28 01:11:15.454819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-VolumesSnapshotTestJSON-1940777549-project Aug 28 01:11:15.454819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:11:15.454819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:11:15.454819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: 93ee4e8157364093bc9cdbe980bde4bb Aug 28 01:11:15.454819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-VolumesSnapshotTestJSON-1940777549-project Aug 28 01:11:15.454819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 01:11:15.456476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9635d43a-3381-4941-883a-893e58b0b285) Aug 28 01:11:15.456476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'server_id': '2cbc0806-fe4b-4018-93bf-b4cf3c292bdc', 'id': '7421938a-2d8f-4ae7-aac6-d08ec2a49354'} Aug 28 01:11:15.456476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:11:15.456476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.456476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:11:15.456476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:11:15.456476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:11:15.456476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:11:15.456476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:11:15.456476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:11:15.456476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:11:15.456476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:11:15.456476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:11:15.456476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:11:15.456476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:11:15.456476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:11:15.456476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:11:15.458226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:11:15.458226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:11:15.458226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:11:15.458226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9635d43a-3381-4941-883a-893e58b0b285) Aug 28 01:11:15.458226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.458226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:15.458226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.458226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.458226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5192, in _detach_volume Aug 28 01:11:15.458226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Aug 28 01:11:15.458226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 28 01:11:15.458226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 28 01:11:15.458226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 28 01:11:15.458226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 28 01:11:15.458226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:11:15.458226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:11:15.458226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:11:15.458226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:11:15.458226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Aug 28 01:11:15.459880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Aug 28 01:11:15.459880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Aug 28 01:11:15.459880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Aug 28 01:11:15.459880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Aug 28 01:11:15.459880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:11:15.459880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:11:15.459880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:11:15.459880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:11:15.459880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:11:15.459880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:11:15.459880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:11:15.459880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9635d43a-3381-4941-883a-893e58b0b285) Aug 28 01:11:15.459880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.459880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:15.459880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.459880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.459880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Aug 28 01:11:15.462052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Aug 28 01:11:15.462052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Aug 28 01:11:15.462052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:11:15.462052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Aug 28 01:11:15.462052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Aug 28 01:11:15.462052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5214, in detach_volume Aug 28 01:11:15.462052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Aug 28 01:11:15.462052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Aug 28 01:11:15.462052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Aug 28 01:11:15.462052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Aug 28 01:11:15.462052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 28 01:11:15.462052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9635d43a-3381-4941-883a-893e58b0b285) Aug 28 01:11:15.462052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.462052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:15.462052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.462052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.462052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:11:15.462052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:11:15.464224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:11:15.464224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:11:15.464224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:11:15.464224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:11:15.464224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Aug 28 01:11:15.464224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Aug 28 01:11:15.464224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9635d43a-3381-4941-883a-893e58b0b285) Aug 28 01:11:15.464224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.464224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:15.464224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.464224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:15.464224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:11:15.464224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:11:15.464224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:11:15.464224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:11:15.464224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9635d43a-3381-4941-883a-893e58b0b285) Aug 28 01:11:15.464224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:15.469411 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b782c35c-a72d-4d10-9243-4ad9ac568fba tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.469531 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8f288dc2-7ce8-48e8-b21a-99bb45fec8bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9635d43a-3381-4941-883a-893e58b0b285) {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:11:15.469662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b782c35c-a72d-4d10-9243-4ad9ac568fba tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.469662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b782c35c-a72d-4d10-9243-4ad9ac568fba tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.469869 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8f288dc2-7ce8-48e8-b21a-99bb45fec8bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "DELETE /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc/os-volume_attachments/7421938a-2d8f-4ae7-aac6-d08ec2a49354" status: 400 len: 269 microversion: 2.1 time: 0.246045 Aug 28 01:11:15.469975 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b782c35c-a72d-4d10-9243-4ad9ac568fba tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.469975 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b782c35c-a72d-4d10-9243-4ad9ac568fba tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.469975 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b782c35c-a72d-4d10-9243-4ad9ac568fba tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.470227 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 708/1188] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Aug 28 01:11:15 2022] DELETE /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc/os-volume_attachments/7421938a-2d8f-4ae7-aac6-d08ec2a49354 => generated 269 bytes in 252 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 28 01:11:15.470227 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d6bcb31e-4a07-4ebc-ad7b-7e06f6140fe0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:15.472449 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d6bcb31e-4a07-4ebc-ad7b-7e06f6140fe0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:15.478245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6bcb31e-4a07-4ebc-ad7b-7e06f6140fe0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.478896 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6bcb31e-4a07-4ebc-ad7b-7e06f6140fe0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.479233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6bcb31e-4a07-4ebc-ad7b-7e06f6140fe0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.481188 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6bcb31e-4a07-4ebc-ad7b-7e06f6140fe0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.481757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6bcb31e-4a07-4ebc-ad7b-7e06f6140fe0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.482212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6bcb31e-4a07-4ebc-ad7b-7e06f6140fe0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.499293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b782c35c-a72d-4d10-9243-4ad9ac568fba tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.059922 Aug 28 01:11:15.500465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 481/1189] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Aug 28 01:11:15 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:11:15.505789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3fad2f5f-d1ca-48ec-b16a-0809386a5da2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:15.508867 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3fad2f5f-d1ca-48ec-b16a-0809386a5da2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:15.517096 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3fad2f5f-d1ca-48ec-b16a-0809386a5da2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.517702 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3fad2f5f-d1ca-48ec-b16a-0809386a5da2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.518177 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3fad2f5f-d1ca-48ec-b16a-0809386a5da2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.519039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3fad2f5f-d1ca-48ec-b16a-0809386a5da2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.519573 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3fad2f5f-d1ca-48ec-b16a-0809386a5da2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.520003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3fad2f5f-d1ca-48ec-b16a-0809386a5da2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.527219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d6bcb31e-4a07-4ebc-ad7b-7e06f6140fe0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.065566 Aug 28 01:11:15.528115 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 709/1190] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:15 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:15.532917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e84e1d25-fbd4-4e02-9ef0-fbf1bc2889d7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:15.535156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e84e1d25-fbd4-4e02-9ef0-fbf1bc2889d7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:15.541722 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e84e1d25-fbd4-4e02-9ef0-fbf1bc2889d7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.542469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e84e1d25-fbd4-4e02-9ef0-fbf1bc2889d7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.543081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e84e1d25-fbd4-4e02-9ef0-fbf1bc2889d7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.544118 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e84e1d25-fbd4-4e02-9ef0-fbf1bc2889d7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:15.544575 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e84e1d25-fbd4-4e02-9ef0-fbf1bc2889d7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:15.545016 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e84e1d25-fbd4-4e02-9ef0-fbf1bc2889d7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:15.558233 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3fad2f5f-d1ca-48ec-b16a-0809386a5da2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056405 Aug 28 01:11:15.558941 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 482/1191] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:15 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:15.563799 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4c19fb43-711c-42b7-a73d-9e49a43550be tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-990245770", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1578003485", "security_groups": [{"name": "tempest-secgroup-smoke-1006421122"}], "block_device_mapping_v2": [{"uuid": "69e463b3-ca8b-4bc8-b32a-40b13075b24d", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "63ecdb07-b041-4401-9ffd-486e98bce044"}]}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:11:15.583555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e84e1d25-fbd4-4e02-9ef0-fbf1bc2889d7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054321 Aug 28 01:11:15.584145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 710/1192] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:15 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:15.705189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.network.neutron [None req-4c19fb43-711c-42b7-a73d-9e49a43550be tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] validate_networks() for [('63ecdb07-b041-4401-9ffd-486e98bce044', None, None, None, None, None)] {{(pid=120524) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 28 01:11:16.180208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-4c19fb43-711c-42b7-a73d-9e49a43550be tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Flavor limits 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 28 01:11:16.180836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-4c19fb43-711c-42b7-a73d-9e49a43550be tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Image limits 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 28 01:11:16.181617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-4c19fb43-711c-42b7-a73d-9e49a43550be tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Flavor pref 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 28 01:11:16.182208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-4c19fb43-711c-42b7-a73d-9e49a43550be tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Image pref 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 28 01:11:16.182621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-4c19fb43-711c-42b7-a73d-9e49a43550be tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 28 01:11:16.191651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c19fb43-711c-42b7-a73d-9e49a43550be tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:16.192130 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c19fb43-711c-42b7-a73d-9e49a43550be tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:16.192559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c19fb43-711c-42b7-a73d-9e49a43550be tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:16.193335 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c19fb43-711c-42b7-a73d-9e49a43550be tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:16.193784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c19fb43-711c-42b7-a73d-9e49a43550be tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:16.194548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c19fb43-711c-42b7-a73d-9e49a43550be tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:16.211041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-4c19fb43-711c-42b7-a73d-9e49a43550be tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Getting quotas for project 287c855ef8e241978a4d1e7cb3c7a16f. Resources: {'ram', 'instances', 'cores'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 01:11:16.215364 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-4c19fb43-711c-42b7-a73d-9e49a43550be tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Getting quotas for user cac1ab72dcee40b19bcff4dd5a75d82d and project 287c855ef8e241978a4d1e7cb3c7a16f. Resources: {'ram', 'instances', 'cores'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 01:11:16.283803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4c19fb43-711c-42b7-a73d-9e49a43550be tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Going to run 1 instances... {{(pid=120524) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 28 01:11:16.287148 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4c19fb43-711c-42b7-a73d-9e49a43550be tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 359bc060-4600-458b-a538-ead703489337] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='69e463b3-ca8b-4bc8-b32a-40b13075b24d',volume_size=None,volume_type=None)] {{(pid=120524) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 28 01:11:16.361373 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-4c19fb43-711c-42b7-a73d-9e49a43550be tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Volume 69e463b3-ca8b-4bc8-b32a-40b13075b24d status must be available or downloading to reserve, but the current status is in-use. (HTTP 400) (Request-ID: req-5d7ecb78-de19-44e3-9728-0d14c2c5fb37) Aug 28 01:11:16.371836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4c19fb43-711c-42b7-a73d-9e49a43550be tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Request method failure captured: Aug 28 01:11:16.371836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: POST /compute/v2.1/servers HTTP/1.1 Aug 28 01:11:16.371836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:11:16.371836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:11:16.371836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:11:16.371836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 477 Aug 28 01:11:16.371836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:11:16.371836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:11:16.371836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:11:16.371836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:11:16.371836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCr1pxHIWV285Zf1gYz6DOmXQ2NMTOVf64lxBBAObx2MPbCdCBY1JwKpzcv5QOx9Y_YlZRIoc7sjAsi2krqODigup6LiCH54F6kQOVjkdNxhzdtOwuAeDHBQK6z6x3RCl6XGegaI-t8IJwZzgBuOc5uYF2CrOv547lV-tG7cEK7alrc0 Aug 28 01:11:16.371836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:11:16.371836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:11:16.371836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:11:16.371836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:11:16.371836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:11:16.371836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:11:16.371836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 287c855ef8e241978a4d1e7cb3c7a16f Aug 28 01:11:16.371836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-TestVolumeBootPattern-1331161477 Aug 28 01:11:16.371836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:11:16.371836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:11:16.373419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_287c855ef8e241978a4d1e7cb3c7a16f", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287c855ef8e241978a4d1e7cb3c7a16f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287c855ef8e241978a4d1e7cb3c7a16f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/287c855ef8e241978a4d1e7cb3c7a16f"}]}] Aug 28 01:11:16.373419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-TestVolumeBootPattern-1331161477 Aug 28 01:11:16.373419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 287c855ef8e241978a4d1e7cb3c7a16f Aug 28 01:11:16.373419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-TestVolumeBootPattern-1331161477 Aug 28 01:11:16.373419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-TestVolumeBootPattern-1331161477-project Aug 28 01:11:16.373419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:11:16.373419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:11:16.373419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: cac1ab72dcee40b19bcff4dd5a75d82d Aug 28 01:11:16.373419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-TestVolumeBootPattern-1331161477-project Aug 28 01:11:16.373419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Aug 28 01:11:16.374218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {"server": {"name": "tempest-TestVolumeBootPattern-server-990245770", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1578003485", "security_groups": [{"name": "tempest-secgroup-smoke-1006421122"}], "block_device_mapping_v2": [{"uuid": "69e463b3-ca8b-4bc8-b32a-40b13075b24d", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "63ecdb07-b041-4401-9ffd-486e98bce044"}]}} Aug 28 01:11:16.374218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:11:16.374218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Invalid volume: Invalid input received: Invalid volume: Volume 69e463b3-ca8b-4bc8-b32a-40b13075b24d status must be available or downloading to reserve, but the current status is in-use. (HTTP 400) (Request-ID: req-5d7ecb78-de19-44e3-9728-0d14c2c5fb37) Aug 28 01:11:16.374218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'body': {'server': {'name': 'tempest-TestVolumeBootPattern-server-990245770', 'imageRef': '', 'flavorRef': '452', 'key_name': 'tempest-TestVolumeBootPattern-1578003485', 'security_groups': [{'name': 'tempest-secgroup-smoke-1006421122'}], 'block_device_mapping_v2': [{'source_type': 'volume', 'destination_type': 'volume', 'boot_index': 0, 'delete_on_termination': False, 'volume_id': '69e463b3-ca8b-4bc8-b32a-40b13075b24d'}], 'networks': [{'uuid': '63ecdb07-b041-4401-9ffd-486e98bce044'}]}}} Aug 28 01:11:16.374218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:11:16.374218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:16.374218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:11:16.374218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:11:16.374218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:11:16.374218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:11:16.374957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 808, in attachment_create Aug 28 01:11:16.374957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi LOG.error('Create attachment failed for volume ' Aug 28 01:11:16.374957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 28 01:11:16.374957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self.force_reraise() Aug 28 01:11:16.374957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 28 01:11:16.374957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise self.value Aug 28 01:11:16.374957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 800, in attachment_create Aug 28 01:11:16.374957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi attachment_ref = cinderclient(context, '3.44').attachments.create( Aug 28 01:11:16.374957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/api_versions.py", line 421, in substitution Aug 28 01:11:16.374957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method.func(obj, *args, **kwargs) Aug 28 01:11:16.374957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/attachments.py", line 39, in create Aug 28 01:11:16.374957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi retval = self._create('/attachments', body, 'attachment') Aug 28 01:11:16.374957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 306, in _create Aug 28 01:11:16.374957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:11:16.374957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:11:16.374957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:11:16.374957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:11:16.374957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:11:16.376453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:11:16.376453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:11:16.376453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Volume 69e463b3-ca8b-4bc8-b32a-40b13075b24d status must be available or downloading to reserve, but the current status is in-use. (HTTP 400) (Request-ID: req-5d7ecb78-de19-44e3-9728-0d14c2c5fb37) Aug 28 01:11:16.376453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:16.376453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:16.376453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:16.376453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:16.376453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1971, in _validate_bdm Aug 28 01:11:16.376453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self._check_attach_and_reserve_volume( Aug 28 01:11:16.376453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5001, in _check_attach_and_reserve_volume Aug 28 01:11:16.376453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi attachment_id = self.volume_api.attachment_create( Aug 28 01:11:16.376453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Aug 28 01:11:16.376453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Aug 28 01:11:16.376453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 28 01:11:16.376453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 28 01:11:16.376453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:11:16.376453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:11:16.376453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:11:16.377844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:11:16.377844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 808, in attachment_create Aug 28 01:11:16.377844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi LOG.error('Create attachment failed for volume ' Aug 28 01:11:16.377844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 28 01:11:16.377844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self.force_reraise() Aug 28 01:11:16.377844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 28 01:11:16.377844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise self.value Aug 28 01:11:16.377844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 800, in attachment_create Aug 28 01:11:16.377844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi attachment_ref = cinderclient(context, '3.44').attachments.create( Aug 28 01:11:16.377844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/api_versions.py", line 421, in substitution Aug 28 01:11:16.377844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method.func(obj, *args, **kwargs) Aug 28 01:11:16.377844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/attachments.py", line 39, in create Aug 28 01:11:16.377844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi retval = self._create('/attachments', body, 'attachment') Aug 28 01:11:16.377844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 306, in _create Aug 28 01:11:16.377844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Aug 28 01:11:16.377844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Aug 28 01:11:16.377844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Aug 28 01:11:16.377844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:11:16.378993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:11:16.378993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:11:16.378993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:11:16.378993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Volume 69e463b3-ca8b-4bc8-b32a-40b13075b24d status must be available or downloading to reserve, but the current status is in-use. (HTTP 400) (Request-ID: req-5d7ecb78-de19-44e3-9728-0d14c2c5fb37) Aug 28 01:11:16.378993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:16.378993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:16.378993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:16.378993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:16.378993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Aug 28 01:11:16.378993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Aug 28 01:11:16.378993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Aug 28 01:11:16.378993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._create_instance( Aug 28 01:11:16.378993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1733, in _create_instance Aug 28 01:11:16.378993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instances_to_build = self._provision_instances( Aug 28 01:11:16.378993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1550, in _provision_instances Aug 28 01:11:16.378993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self._cleanup_build_artifacts(None, instances_to_build) Aug 28 01:11:16.378993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 28 01:11:16.378993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self.force_reraise() Aug 28 01:11:16.380201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 28 01:11:16.380201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise self.value Aug 28 01:11:16.380201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1460, in _provision_instances Aug 28 01:11:16.380201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self._bdm_validate_set_size_and_instance(context, Aug 28 01:11:16.380201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1866, in _bdm_validate_set_size_and_instance Aug 28 01:11:16.380201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self._validate_bdm( Aug 28 01:11:16.380201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1980, in _validate_bdm Aug 28 01:11:16.380201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Aug 28 01:11:16.380201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Volume 69e463b3-ca8b-4bc8-b32a-40b13075b24d status must be available or downloading to reserve, but the current status is in-use. (HTTP 400) (Request-ID: req-5d7ecb78-de19-44e3-9728-0d14c2c5fb37) Aug 28 01:11:16.380201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:16.380201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:16.380201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:16.380201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:16.380201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:11:16.380201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:11:16.380201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:11:16.380201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:11:16.380201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:11:16.381738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:11:16.381738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 28 01:11:16.381738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 28 01:11:16.381738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 28 01:11:16.381738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 28 01:11:16.381738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Aug 28 01:11:16.381738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Aug 28 01:11:16.381738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Aug 28 01:11:16.381738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 866, in create Aug 28 01:11:16.381738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=error.format_message()) Aug 28 01:11:16.381738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Volume 69e463b3-ca8b-4bc8-b32a-40b13075b24d status must be available or downloading to reserve, but the current status is in-use. (HTTP 400) (Request-ID: req-5d7ecb78-de19-44e3-9728-0d14c2c5fb37) Aug 28 01:11:16.381738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:16.381738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:16.381738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:16.381738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:16.381738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:11:16.381738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:11:16.381738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:11:16.381738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:11:16.383172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Volume 69e463b3-ca8b-4bc8-b32a-40b13075b24d status must be available or downloading to reserve, but the current status is in-use. (HTTP 400) (Request-ID: req-5d7ecb78-de19-44e3-9728-0d14c2c5fb37) Aug 28 01:11:16.383172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:16.383172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4c19fb43-711c-42b7-a73d-9e49a43550be tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Volume 69e463b3-ca8b-4bc8-b32a-40b13075b24d status must be available or downloading to reserve, but the current status is in-use. (HTTP 400) (Request-ID: req-5d7ecb78-de19-44e3-9728-0d14c2c5fb37) {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:11:16.383172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4c19fb43-711c-42b7-a73d-9e49a43550be tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 400 len: 295 microversion: 2.1 time: 0.813739 Aug 28 01:11:16.383172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 483/1193] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Aug 28 01:11:15 2022] POST /compute/v2.1/servers => generated 295 bytes in 815 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 28 01:11:16.512540 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-091609cc-9ccf-4731-a4cf-3b6788e3bf36 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:11:16.515021 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-091609cc-9ccf-4731-a4cf-3b6788e3bf36 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:16.521500 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-091609cc-9ccf-4731-a4cf-3b6788e3bf36 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:16.521980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-091609cc-9ccf-4731-a4cf-3b6788e3bf36 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:16.522413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-091609cc-9ccf-4731-a4cf-3b6788e3bf36 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:16.523112 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-091609cc-9ccf-4731-a4cf-3b6788e3bf36 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:16.523564 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-091609cc-9ccf-4731-a4cf-3b6788e3bf36 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:16.523996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-091609cc-9ccf-4731-a4cf-3b6788e3bf36 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:16.540297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f6ee63e1-56c2-4a2c-a132-6760bc91e6de tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:11:16.543286 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f6ee63e1-56c2-4a2c-a132-6760bc91e6de tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:16.552070 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f6ee63e1-56c2-4a2c-a132-6760bc91e6de tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:16.552667 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f6ee63e1-56c2-4a2c-a132-6760bc91e6de tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:16.553392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f6ee63e1-56c2-4a2c-a132-6760bc91e6de tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:16.554168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f6ee63e1-56c2-4a2c-a132-6760bc91e6de tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:16.554651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f6ee63e1-56c2-4a2c-a132-6760bc91e6de tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:16.555454 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f6ee63e1-56c2-4a2c-a132-6760bc91e6de tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:16.632947 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-091609cc-9ccf-4731-a4cf-3b6788e3bf36 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "POST /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/action" status: 200 len: 37863 microversion: 2.20 time: 0.124314 Aug 28 01:11:16.634319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 711/1194] 10.222.0.42 () {62 vars in 1319 bytes} [Sun Aug 28 01:11:16 2022] POST /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df/action => generated 37863 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 361 bytes (1 switches on core 0) Aug 28 01:11:16.639022 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-21e75042-a3ce-4ceb-86c0-4e0606619025 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:16.641619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-21e75042-a3ce-4ceb-86c0-4e0606619025 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:16.648979 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-21e75042-a3ce-4ceb-86c0-4e0606619025 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:16.649744 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-21e75042-a3ce-4ceb-86c0-4e0606619025 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:16.650208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-21e75042-a3ce-4ceb-86c0-4e0606619025 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:16.652367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-21e75042-a3ce-4ceb-86c0-4e0606619025 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:16.653023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-21e75042-a3ce-4ceb-86c0-4e0606619025 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:16.653456 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-21e75042-a3ce-4ceb-86c0-4e0606619025 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:16.690743 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-21e75042-a3ce-4ceb-86c0-4e0606619025 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055728 Aug 28 01:11:16.691513 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 712/1195] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:16 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:16.705085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c88c8064-926b-47c0-9e42-35a356eb91be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:16.707320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c88c8064-926b-47c0-9e42-35a356eb91be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:16.709457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f6ee63e1-56c2-4a2c-a132-6760bc91e6de tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "POST /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/action" status: 200 len: 41153 microversion: 2.1 time: 0.172969 Aug 28 01:11:16.714025 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 484/1196] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Aug 28 01:11:16 2022] POST /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c/action => generated 41153 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:11:16.717344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c88c8064-926b-47c0-9e42-35a356eb91be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:16.718106 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c88c8064-926b-47c0-9e42-35a356eb91be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:16.718757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c88c8064-926b-47c0-9e42-35a356eb91be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:16.719836 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c88c8064-926b-47c0-9e42-35a356eb91be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:16.720330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c88c8064-926b-47c0-9e42-35a356eb91be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:16.720912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c88c8064-926b-47c0-9e42-35a356eb91be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:16.761527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c88c8064-926b-47c0-9e42-35a356eb91be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.069116 Aug 28 01:11:16.762449 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 713/1197] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:16 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:16.874274 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ac5f8b07-9020-49ed-b140-de4c9e58ecb8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:16.874883 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ac5f8b07-9020-49ed-b140-de4c9e58ecb8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:16.881966 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac5f8b07-9020-49ed-b140-de4c9e58ecb8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:16.882439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac5f8b07-9020-49ed-b140-de4c9e58ecb8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:16.882868 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac5f8b07-9020-49ed-b140-de4c9e58ecb8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:16.883549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac5f8b07-9020-49ed-b140-de4c9e58ecb8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:16.883989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac5f8b07-9020-49ed-b140-de4c9e58ecb8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:16.884400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac5f8b07-9020-49ed-b140-de4c9e58ecb8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:16.898903 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-ac5f8b07-9020-49ed-b140-de4c9e58ecb8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] HTTP exception thrown: Instance 1cc5e59c-500c-421e-920c-c08b8ae3a206 could not be found. Aug 28 01:11:16.900622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ac5f8b07-9020-49ed-b140-de4c9e58ecb8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Request method failure captured: Aug 28 01:11:16.900622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: DELETE /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 HTTP/1.1 Aug 28 01:11:16.900622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:11:16.900622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:11:16.900622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:11:16.900622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:11:16.900622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:11:16.900622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:11:16.900622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:11:16.900622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:11:16.900622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCr1pxHIWV285Zf1gYz6DOmXQ2NMTOVf64lxBBAObx2MPbCdCBY1JwKpzcv5QOx9Y_YlZRIoc7sjAsi2krqODigup6LiCH54F6kQOVjkdNxhzdtOwuAeDHBQK6z6x3RCl6XGegaI-t8IJwZzgBuOc5uYF2CrOv547lV-tG7cEK7alrc0 Aug 28 01:11:16.900622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:11:16.900622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:11:16.900622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:11:16.900622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:11:16.900622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:11:16.900622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:11:16.900622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 287c855ef8e241978a4d1e7cb3c7a16f Aug 28 01:11:16.900622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-TestVolumeBootPattern-1331161477 Aug 28 01:11:16.900622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:11:16.900622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:11:16.902149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_287c855ef8e241978a4d1e7cb3c7a16f", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287c855ef8e241978a4d1e7cb3c7a16f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287c855ef8e241978a4d1e7cb3c7a16f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/287c855ef8e241978a4d1e7cb3c7a16f"}]}] Aug 28 01:11:16.902149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-TestVolumeBootPattern-1331161477 Aug 28 01:11:16.902149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 287c855ef8e241978a4d1e7cb3c7a16f Aug 28 01:11:16.902149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-TestVolumeBootPattern-1331161477 Aug 28 01:11:16.902149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-TestVolumeBootPattern-1331161477-project Aug 28 01:11:16.902149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:11:16.902149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:11:16.902149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: cac1ab72dcee40b19bcff4dd5a75d82d Aug 28 01:11:16.902149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-TestVolumeBootPattern-1331161477-project Aug 28 01:11:16.902149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:11:16.902149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Instance 1cc5e59c-500c-421e-920c-c08b8ae3a206 could not be found. Aug 28 01:11:16.902149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'id': '1cc5e59c-500c-421e-920c-c08b8ae3a206'} Aug 28 01:11:16.903073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:11:16.903073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:16.903073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:11:16.903073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:11:16.903073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:11:16.903073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:11:16.903073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:11:16.903073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:11:16.903073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:11:16.903073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:11:16.903073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1cc5e59c-500c-421e-920c-c08b8ae3a206 could not be found. Aug 28 01:11:16.903073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:16.903073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:16.903073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:16.903073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:16.903073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:11:16.903073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:11:16.903073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:11:16.903073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:11:16.903073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:11:16.904607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:11:16.904607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 28 01:11:16.904607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 28 01:11:16.904607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 28 01:11:16.904607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 28 01:11:16.904607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:11:16.904607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:11:16.904607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:11:16.904607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:11:16.904607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1cc5e59c-500c-421e-920c-c08b8ae3a206 could not be found. Aug 28 01:11:16.904607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:16.904607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:16.904607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:16.904607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:16.904607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:11:16.904607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:11:16.904607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:11:16.904607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:11:16.904607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1cc5e59c-500c-421e-920c-c08b8ae3a206 could not be found. Aug 28 01:11:16.904607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:11:16.906215 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ac5f8b07-9020-49ed-b140-de4c9e58ecb8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Returning 404 to user: Instance 1cc5e59c-500c-421e-920c-c08b8ae3a206 could not be found. {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:11:16.906215 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ac5f8b07-9020-49ed-b140-de4c9e58ecb8 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "DELETE /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 404 len: 111 microversion: 2.1 time: 0.032143 Aug 28 01:11:16.906439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 485/1198] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 01:11:16 2022] DELETE /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:11:16.913699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fe084fe6-55d7-4ff6-a843-49481753e3a1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:16.914678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fe084fe6-55d7-4ff6-a843-49481753e3a1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] [instance: 1cc5e59c-500c-421e-920c-c08b8ae3a206] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:16.924679 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe084fe6-55d7-4ff6-a843-49481753e3a1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:16.925442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe084fe6-55d7-4ff6-a843-49481753e3a1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:16.926149 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe084fe6-55d7-4ff6-a843-49481753e3a1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:16.927284 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe084fe6-55d7-4ff6-a843-49481753e3a1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:16.927924 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe084fe6-55d7-4ff6-a843-49481753e3a1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:16.928719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe084fe6-55d7-4ff6-a843-49481753e3a1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:16.944808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-fe084fe6-55d7-4ff6-a843-49481753e3a1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] HTTP exception thrown: Instance 1cc5e59c-500c-421e-920c-c08b8ae3a206 could not be found. Aug 28 01:11:16.946430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fe084fe6-55d7-4ff6-a843-49481753e3a1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Request method failure captured: Aug 28 01:11:16.946430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 HTTP/1.1 Aug 28 01:11:16.946430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:11:16.946430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:11:16.946430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:11:16.946430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:11:16.946430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:11:16.946430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:11:16.946430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:11:16.946430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:11:16.946430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCr1pxHIWV285Zf1gYz6DOmXQ2NMTOVf64lxBBAObx2MPbCdCBY1JwKpzcv5QOx9Y_YlZRIoc7sjAsi2krqODigup6LiCH54F6kQOVjkdNxhzdtOwuAeDHBQK6z6x3RCl6XGegaI-t8IJwZzgBuOc5uYF2CrOv547lV-tG7cEK7alrc0 Aug 28 01:11:16.946430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:11:16.946430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:11:16.946430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:11:16.946430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:11:16.946430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:11:16.946430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:11:16.946430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 287c855ef8e241978a4d1e7cb3c7a16f Aug 28 01:11:16.946430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-TestVolumeBootPattern-1331161477 Aug 28 01:11:16.946430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:11:16.946430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:11:16.947992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_287c855ef8e241978a4d1e7cb3c7a16f", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287c855ef8e241978a4d1e7cb3c7a16f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287c855ef8e241978a4d1e7cb3c7a16f"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/287c855ef8e241978a4d1e7cb3c7a16f"}]}] Aug 28 01:11:16.947992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-TestVolumeBootPattern-1331161477 Aug 28 01:11:16.947992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 287c855ef8e241978a4d1e7cb3c7a16f Aug 28 01:11:16.947992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-TestVolumeBootPattern-1331161477 Aug 28 01:11:16.947992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-TestVolumeBootPattern-1331161477-project Aug 28 01:11:16.947992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:11:16.947992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:11:16.947992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: cac1ab72dcee40b19bcff4dd5a75d82d Aug 28 01:11:16.947992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-TestVolumeBootPattern-1331161477-project Aug 28 01:11:16.947992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 01:11:16.947992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Instance 1cc5e59c-500c-421e-920c-c08b8ae3a206 could not be found. Aug 28 01:11:16.947992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'id': '1cc5e59c-500c-421e-920c-c08b8ae3a206'} Aug 28 01:11:16.948816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:11:16.948816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:16.948816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:11:16.948816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:11:16.948816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:11:16.948816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:11:16.948816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:11:16.948816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:11:16.948816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:11:16.948816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:11:16.948816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1cc5e59c-500c-421e-920c-c08b8ae3a206 could not be found. Aug 28 01:11:16.948816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:16.948816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:16.948816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:16.948816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:16.948816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:11:16.948816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:11:16.948816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:11:16.948816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:11:16.948816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:11:16.950295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:11:16.950295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 01:11:16.950295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 01:11:16.950295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:11:16.950295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:11:16.950295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:11:16.950295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:11:16.950295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1cc5e59c-500c-421e-920c-c08b8ae3a206 could not be found. Aug 28 01:11:16.950295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:16.950295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:11:16.950295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:16.950295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:11:16.950295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:11:16.950295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:11:16.950295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:11:16.950295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:11:16.950295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1cc5e59c-500c-421e-920c-c08b8ae3a206 could not be found. Aug 28 01:11:16.950295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:11:16.951554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fe084fe6-55d7-4ff6-a843-49481753e3a1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Returning 404 to user: Instance 1cc5e59c-500c-421e-920c-c08b8ae3a206 could not be found. {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:11:16.951554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fe084fe6-55d7-4ff6-a843-49481753e3a1 tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206" status: 404 len: 111 microversion: 2.1 time: 0.040621 Aug 28 01:11:16.951554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 714/1199] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:11:16 2022] GET /compute/v2.1/servers/1cc5e59c-500c-421e-920c-c08b8ae3a206 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:11:17.704582 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9a087092-0f79-4fa9-ad6e-8c3fc9ebeb2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:17.706943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9a087092-0f79-4fa9-ad6e-8c3fc9ebeb2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:17.713717 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9a087092-0f79-4fa9-ad6e-8c3fc9ebeb2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:17.714305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9a087092-0f79-4fa9-ad6e-8c3fc9ebeb2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:17.714633 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9a087092-0f79-4fa9-ad6e-8c3fc9ebeb2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:17.715324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9a087092-0f79-4fa9-ad6e-8c3fc9ebeb2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:17.715764 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9a087092-0f79-4fa9-ad6e-8c3fc9ebeb2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:17.716168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9a087092-0f79-4fa9-ad6e-8c3fc9ebeb2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:17.746862 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9a087092-0f79-4fa9-ad6e-8c3fc9ebeb2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046621 Aug 28 01:11:17.747432 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 486/1200] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:17 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:17.773977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-beb4510b-7e93-4008-a871-3fbd19a285c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:17.776126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-beb4510b-7e93-4008-a871-3fbd19a285c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:17.782645 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-beb4510b-7e93-4008-a871-3fbd19a285c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:17.783111 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-beb4510b-7e93-4008-a871-3fbd19a285c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:17.783540 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-beb4510b-7e93-4008-a871-3fbd19a285c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:17.784202 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-beb4510b-7e93-4008-a871-3fbd19a285c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:17.784642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-beb4510b-7e93-4008-a871-3fbd19a285c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:17.785078 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-beb4510b-7e93-4008-a871-3fbd19a285c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:17.815748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-beb4510b-7e93-4008-a871-3fbd19a285c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045778 Aug 28 01:11:17.816353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 715/1201] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:17 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:18.761827 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9640e162-2f86-41f1-80ef-cba8cbd5f61f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:18.764306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9640e162-2f86-41f1-80ef-cba8cbd5f61f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:18.770943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9640e162-2f86-41f1-80ef-cba8cbd5f61f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:18.771368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9640e162-2f86-41f1-80ef-cba8cbd5f61f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:18.771789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9640e162-2f86-41f1-80ef-cba8cbd5f61f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:18.772456 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9640e162-2f86-41f1-80ef-cba8cbd5f61f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:18.772915 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9640e162-2f86-41f1-80ef-cba8cbd5f61f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:18.773356 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9640e162-2f86-41f1-80ef-cba8cbd5f61f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:18.802484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9640e162-2f86-41f1-80ef-cba8cbd5f61f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046745 Aug 28 01:11:18.803022 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 487/1202] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:18 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:18.828164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bdfcdbe7-4c37-4d8b-b57e-92b4d233e644 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:18.830573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bdfcdbe7-4c37-4d8b-b57e-92b4d233e644 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:18.837096 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bdfcdbe7-4c37-4d8b-b57e-92b4d233e644 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:18.837577 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bdfcdbe7-4c37-4d8b-b57e-92b4d233e644 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:18.838007 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bdfcdbe7-4c37-4d8b-b57e-92b4d233e644 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:18.838702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bdfcdbe7-4c37-4d8b-b57e-92b4d233e644 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:18.839148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bdfcdbe7-4c37-4d8b-b57e-92b4d233e644 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:18.839576 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bdfcdbe7-4c37-4d8b-b57e-92b4d233e644 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:18.870967 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bdfcdbe7-4c37-4d8b-b57e-92b4d233e644 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046912 Aug 28 01:11:18.871513 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 716/1203] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:18 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:19.815048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ea69956b-ea1b-4967-82fa-9990126a611b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:19.817221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ea69956b-ea1b-4967-82fa-9990126a611b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:19.825280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea69956b-ea1b-4967-82fa-9990126a611b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:19.825775 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea69956b-ea1b-4967-82fa-9990126a611b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:19.826203 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea69956b-ea1b-4967-82fa-9990126a611b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:19.826895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea69956b-ea1b-4967-82fa-9990126a611b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:19.827334 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea69956b-ea1b-4967-82fa-9990126a611b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:19.827812 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea69956b-ea1b-4967-82fa-9990126a611b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:19.858192 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ea69956b-ea1b-4967-82fa-9990126a611b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047051 Aug 28 01:11:19.858781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 488/1204] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:19 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:19.883459 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-023088b0-dc7e-4bea-91a9-70ccd1f0d0d3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:19.885686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-023088b0-dc7e-4bea-91a9-70ccd1f0d0d3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:19.892333 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-023088b0-dc7e-4bea-91a9-70ccd1f0d0d3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:19.892800 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-023088b0-dc7e-4bea-91a9-70ccd1f0d0d3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:19.893260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-023088b0-dc7e-4bea-91a9-70ccd1f0d0d3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:19.893934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-023088b0-dc7e-4bea-91a9-70ccd1f0d0d3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:19.894375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-023088b0-dc7e-4bea-91a9-70ccd1f0d0d3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:19.894787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-023088b0-dc7e-4bea-91a9-70ccd1f0d0d3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:19.927346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-023088b0-dc7e-4bea-91a9-70ccd1f0d0d3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047857 Aug 28 01:11:19.927918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 717/1205] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:19 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:20.877078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cbf748a3-64e1-49f2-aad1-a0405454b0dc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:20.877078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cbf748a3-64e1-49f2-aad1-a0405454b0dc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:20.882813 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cbf748a3-64e1-49f2-aad1-a0405454b0dc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:20.883368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cbf748a3-64e1-49f2-aad1-a0405454b0dc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:20.883837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cbf748a3-64e1-49f2-aad1-a0405454b0dc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:20.884796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cbf748a3-64e1-49f2-aad1-a0405454b0dc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:20.885239 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cbf748a3-64e1-49f2-aad1-a0405454b0dc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:20.885712 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cbf748a3-64e1-49f2-aad1-a0405454b0dc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:20.917286 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cbf748a3-64e1-49f2-aad1-a0405454b0dc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049169 Aug 28 01:11:20.918036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 489/1206] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:20 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:20.940009 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f77ba5fe-319c-4bc7-8d4a-b73feedc436c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:20.942268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f77ba5fe-319c-4bc7-8d4a-b73feedc436c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:20.949199 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f77ba5fe-319c-4bc7-8d4a-b73feedc436c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:20.949673 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f77ba5fe-319c-4bc7-8d4a-b73feedc436c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:20.950172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f77ba5fe-319c-4bc7-8d4a-b73feedc436c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:20.950874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f77ba5fe-319c-4bc7-8d4a-b73feedc436c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:20.951316 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f77ba5fe-319c-4bc7-8d4a-b73feedc436c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:20.951730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f77ba5fe-319c-4bc7-8d4a-b73feedc436c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:20.977111 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:11:20.982511 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f77ba5fe-319c-4bc7-8d4a-b73feedc436c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046583 Aug 28 01:11:20.983086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 718/1207] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:20 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:21.932130 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d5107e28-edc8-4778-83dc-dd74e3d793b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:21.935448 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d5107e28-edc8-4778-83dc-dd74e3d793b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:21.942251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d5107e28-edc8-4778-83dc-dd74e3d793b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:21.942653 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d5107e28-edc8-4778-83dc-dd74e3d793b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:21.943089 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d5107e28-edc8-4778-83dc-dd74e3d793b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:21.943789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d5107e28-edc8-4778-83dc-dd74e3d793b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:21.944553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d5107e28-edc8-4778-83dc-dd74e3d793b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:21.945216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d5107e28-edc8-4778-83dc-dd74e3d793b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:21.975609 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d5107e28-edc8-4778-83dc-dd74e3d793b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049229 Aug 28 01:11:21.976218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 490/1208] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:21 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:21.995235 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fd3830c3-77a3-45b7-9ab4-bdf56dc63454 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:21.997559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fd3830c3-77a3-45b7-9ab4-bdf56dc63454 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:22.004081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd3830c3-77a3-45b7-9ab4-bdf56dc63454 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:22.004565 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd3830c3-77a3-45b7-9ab4-bdf56dc63454 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:22.005028 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd3830c3-77a3-45b7-9ab4-bdf56dc63454 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:22.005726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd3830c3-77a3-45b7-9ab4-bdf56dc63454 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:22.006178 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd3830c3-77a3-45b7-9ab4-bdf56dc63454 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:22.006600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd3830c3-77a3-45b7-9ab4-bdf56dc63454 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:22.036163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fd3830c3-77a3-45b7-9ab4-bdf56dc63454 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045130 Aug 28 01:11:22.036728 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 719/1209] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:21 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:22.988099 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f3d42c20-4248-4538-b8b1-6476e02eb7cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:22.990355 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f3d42c20-4248-4538-b8b1-6476e02eb7cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:22.997180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f3d42c20-4248-4538-b8b1-6476e02eb7cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:22.997743 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f3d42c20-4248-4538-b8b1-6476e02eb7cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:22.998284 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f3d42c20-4248-4538-b8b1-6476e02eb7cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:22.999019 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f3d42c20-4248-4538-b8b1-6476e02eb7cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:22.999481 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f3d42c20-4248-4538-b8b1-6476e02eb7cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:22.999895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f3d42c20-4248-4538-b8b1-6476e02eb7cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:23.004761 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:11:23.031400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f3d42c20-4248-4538-b8b1-6476e02eb7cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047289 Aug 28 01:11:23.031965 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 491/1210] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:22 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:23.048211 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-64afddd1-18a9-437d-b700-7b8b98b9578c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:23.050469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-64afddd1-18a9-437d-b700-7b8b98b9578c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:23.057087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64afddd1-18a9-437d-b700-7b8b98b9578c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:23.057574 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64afddd1-18a9-437d-b700-7b8b98b9578c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:23.058005 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64afddd1-18a9-437d-b700-7b8b98b9578c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:23.058701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64afddd1-18a9-437d-b700-7b8b98b9578c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:23.059146 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64afddd1-18a9-437d-b700-7b8b98b9578c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:23.059561 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64afddd1-18a9-437d-b700-7b8b98b9578c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:23.089988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-64afddd1-18a9-437d-b700-7b8b98b9578c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045896 Aug 28 01:11:23.090724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 720/1211] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:23 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:24.046023 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-12cbf283-0ed9-4d5c-80d4-29f0af76ca64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:24.048220 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-12cbf283-0ed9-4d5c-80d4-29f0af76ca64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:24.054737 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-12cbf283-0ed9-4d5c-80d4-29f0af76ca64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:24.055216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-12cbf283-0ed9-4d5c-80d4-29f0af76ca64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:24.055693 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-12cbf283-0ed9-4d5c-80d4-29f0af76ca64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:24.056340 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-12cbf283-0ed9-4d5c-80d4-29f0af76ca64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:24.056782 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-12cbf283-0ed9-4d5c-80d4-29f0af76ca64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:24.057219 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-12cbf283-0ed9-4d5c-80d4-29f0af76ca64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:24.090007 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-12cbf283-0ed9-4d5c-80d4-29f0af76ca64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049212 Aug 28 01:11:24.090752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 492/1212] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:24 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:24.102225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f41c38b8-cf73-4f62-a23c-c97b7cadaa09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:24.104395 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f41c38b8-cf73-4f62-a23c-c97b7cadaa09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:24.113607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f41c38b8-cf73-4f62-a23c-c97b7cadaa09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:24.113607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f41c38b8-cf73-4f62-a23c-c97b7cadaa09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:24.113607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f41c38b8-cf73-4f62-a23c-c97b7cadaa09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:24.113607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f41c38b8-cf73-4f62-a23c-c97b7cadaa09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:24.114278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f41c38b8-cf73-4f62-a23c-c97b7cadaa09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:24.114278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f41c38b8-cf73-4f62-a23c-c97b7cadaa09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:24.145166 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f41c38b8-cf73-4f62-a23c-c97b7cadaa09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046719 Aug 28 01:11:24.145747 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 721/1213] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:24 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:25.102857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-10d95524-ae95-4ec3-bbe4-be03ce967f5b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:25.104954 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-10d95524-ae95-4ec3-bbe4-be03ce967f5b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:25.111860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10d95524-ae95-4ec3-bbe4-be03ce967f5b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:25.112351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10d95524-ae95-4ec3-bbe4-be03ce967f5b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:25.112781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10d95524-ae95-4ec3-bbe4-be03ce967f5b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:25.113517 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10d95524-ae95-4ec3-bbe4-be03ce967f5b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:25.113988 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10d95524-ae95-4ec3-bbe4-be03ce967f5b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:25.114406 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10d95524-ae95-4ec3-bbe4-be03ce967f5b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:25.145882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-10d95524-ae95-4ec3-bbe4-be03ce967f5b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046913 Aug 28 01:11:25.146417 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 493/1214] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:25 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:25.157672 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9be86ef2-d4c3-4a87-a0d1-2be017c7e996 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:25.160019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9be86ef2-d4c3-4a87-a0d1-2be017c7e996 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:25.166453 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9be86ef2-d4c3-4a87-a0d1-2be017c7e996 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:25.167057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9be86ef2-d4c3-4a87-a0d1-2be017c7e996 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:25.167553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9be86ef2-d4c3-4a87-a0d1-2be017c7e996 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:25.168244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9be86ef2-d4c3-4a87-a0d1-2be017c7e996 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:25.168943 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9be86ef2-d4c3-4a87-a0d1-2be017c7e996 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:25.169389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9be86ef2-d4c3-4a87-a0d1-2be017c7e996 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:25.199245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9be86ef2-d4c3-4a87-a0d1-2be017c7e996 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045388 Aug 28 01:11:25.199687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 722/1215] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:25 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:26.159792 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f40d036f-22b1-4753-85cc-06726131aff2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:26.162070 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f40d036f-22b1-4753-85cc-06726131aff2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:26.168988 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f40d036f-22b1-4753-85cc-06726131aff2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:26.169484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f40d036f-22b1-4753-85cc-06726131aff2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:26.169960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f40d036f-22b1-4753-85cc-06726131aff2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:26.170668 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f40d036f-22b1-4753-85cc-06726131aff2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:26.171108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f40d036f-22b1-4753-85cc-06726131aff2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:26.171630 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f40d036f-22b1-4753-85cc-06726131aff2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:26.201490 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=2 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:11:26.206245 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f40d036f-22b1-4753-85cc-06726131aff2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050963 Aug 28 01:11:26.206245 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 494/1216] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:26 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:26.211937 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-99067028-0d87-4094-b1da-242ebd72c587 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:26.214104 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-99067028-0d87-4094-b1da-242ebd72c587 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:26.220810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-99067028-0d87-4094-b1da-242ebd72c587 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:26.221334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-99067028-0d87-4094-b1da-242ebd72c587 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:26.221766 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-99067028-0d87-4094-b1da-242ebd72c587 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:26.222446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-99067028-0d87-4094-b1da-242ebd72c587 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:26.222886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-99067028-0d87-4094-b1da-242ebd72c587 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:26.223304 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-99067028-0d87-4094-b1da-242ebd72c587 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:26.253540 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-99067028-0d87-4094-b1da-242ebd72c587 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045451 Aug 28 01:11:26.254012 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 723/1217] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:26 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:27.218158 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-68c0195a-c928-4d7e-a5c0-09cea9a39858 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:27.220294 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-68c0195a-c928-4d7e-a5c0-09cea9a39858 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:27.222165 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=6 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:11:27.228144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-68c0195a-c928-4d7e-a5c0-09cea9a39858 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:27.228648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-68c0195a-c928-4d7e-a5c0-09cea9a39858 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:27.229116 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-68c0195a-c928-4d7e-a5c0-09cea9a39858 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:27.229817 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-68c0195a-c928-4d7e-a5c0-09cea9a39858 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:27.230255 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-68c0195a-c928-4d7e-a5c0-09cea9a39858 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:27.230686 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-68c0195a-c928-4d7e-a5c0-09cea9a39858 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:27.265602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-eb8a26ff-d4a6-484a-b6fb-06b3500c2910 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:27.268293 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-eb8a26ff-d4a6-484a-b6fb-06b3500c2910 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:27.271004 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-68c0195a-c928-4d7e-a5c0-09cea9a39858 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057373 Aug 28 01:11:27.271904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 495/1218] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:27 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:27.275267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb8a26ff-d4a6-484a-b6fb-06b3500c2910 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:27.275746 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb8a26ff-d4a6-484a-b6fb-06b3500c2910 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:27.276174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb8a26ff-d4a6-484a-b6fb-06b3500c2910 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:27.276868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb8a26ff-d4a6-484a-b6fb-06b3500c2910 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:27.277340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb8a26ff-d4a6-484a-b6fb-06b3500c2910 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:27.277753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb8a26ff-d4a6-484a-b6fb-06b3500c2910 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:27.307436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-eb8a26ff-d4a6-484a-b6fb-06b3500c2910 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045761 Aug 28 01:11:27.307982 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 724/1219] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:27 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:28.284188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d836a218-2a33-4df0-9c6f-1be9276ca592 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:28.286350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d836a218-2a33-4df0-9c6f-1be9276ca592 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:28.292934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d836a218-2a33-4df0-9c6f-1be9276ca592 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:28.293482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d836a218-2a33-4df0-9c6f-1be9276ca592 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:28.293929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d836a218-2a33-4df0-9c6f-1be9276ca592 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:28.294605 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d836a218-2a33-4df0-9c6f-1be9276ca592 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:28.295109 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d836a218-2a33-4df0-9c6f-1be9276ca592 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:28.295536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d836a218-2a33-4df0-9c6f-1be9276ca592 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:28.329678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ad4f8173-13ee-421b-9d10-1bfd7b8e1338 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:28.329678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ad4f8173-13ee-421b-9d10-1bfd7b8e1338 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:28.331234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ad4f8173-13ee-421b-9d10-1bfd7b8e1338 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:28.331771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ad4f8173-13ee-421b-9d10-1bfd7b8e1338 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:28.332451 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ad4f8173-13ee-421b-9d10-1bfd7b8e1338 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:28.332897 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ad4f8173-13ee-421b-9d10-1bfd7b8e1338 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:28.333375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ad4f8173-13ee-421b-9d10-1bfd7b8e1338 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:28.333811 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ad4f8173-13ee-421b-9d10-1bfd7b8e1338 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:28.337931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d836a218-2a33-4df0-9c6f-1be9276ca592 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057640 Aug 28 01:11:28.338521 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 496/1220] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:28 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:11:28.367046 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ad4f8173-13ee-421b-9d10-1bfd7b8e1338 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049718 Aug 28 01:11:28.367046 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 725/1221] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:28 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:29.350398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-64792a82-f1e1-4e0d-9c2f-776ea11ee226 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:29.352554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-64792a82-f1e1-4e0d-9c2f-776ea11ee226 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:29.359413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-64792a82-f1e1-4e0d-9c2f-776ea11ee226 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:29.359897 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-64792a82-f1e1-4e0d-9c2f-776ea11ee226 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:29.360318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-64792a82-f1e1-4e0d-9c2f-776ea11ee226 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:29.361029 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-64792a82-f1e1-4e0d-9c2f-776ea11ee226 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:29.361472 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-64792a82-f1e1-4e0d-9c2f-776ea11ee226 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:29.361883 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-64792a82-f1e1-4e0d-9c2f-776ea11ee226 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:29.378487 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-775dd35e-2e07-44cd-b394-f710145c823a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:29.380677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-775dd35e-2e07-44cd-b394-f710145c823a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:29.392745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-775dd35e-2e07-44cd-b394-f710145c823a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:29.392745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-775dd35e-2e07-44cd-b394-f710145c823a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:29.392745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-775dd35e-2e07-44cd-b394-f710145c823a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:29.392745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-775dd35e-2e07-44cd-b394-f710145c823a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:29.393217 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-775dd35e-2e07-44cd-b394-f710145c823a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:29.393217 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-775dd35e-2e07-44cd-b394-f710145c823a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:29.407696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-64792a82-f1e1-4e0d-9c2f-776ea11ee226 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061109 Aug 28 01:11:29.408529 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 497/1222] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:29 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:29.427117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-775dd35e-2e07-44cd-b394-f710145c823a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052929 Aug 28 01:11:29.427703 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 726/1223] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:29 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:30.420645 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-82f453d9-ab75-4b85-a428-1402d0c346c0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:30.422813 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-82f453d9-ab75-4b85-a428-1402d0c346c0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:30.429551 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82f453d9-ab75-4b85-a428-1402d0c346c0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:30.430088 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82f453d9-ab75-4b85-a428-1402d0c346c0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:30.430516 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82f453d9-ab75-4b85-a428-1402d0c346c0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:30.431202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82f453d9-ab75-4b85-a428-1402d0c346c0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:30.431633 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82f453d9-ab75-4b85-a428-1402d0c346c0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:30.432054 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82f453d9-ab75-4b85-a428-1402d0c346c0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:30.439990 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-776062fa-6f26-4c67-bccb-02c66045f8ed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:30.442181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-776062fa-6f26-4c67-bccb-02c66045f8ed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:30.452066 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-776062fa-6f26-4c67-bccb-02c66045f8ed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:30.452544 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-776062fa-6f26-4c67-bccb-02c66045f8ed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:30.452944 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-776062fa-6f26-4c67-bccb-02c66045f8ed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:30.453743 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-776062fa-6f26-4c67-bccb-02c66045f8ed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:30.454174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-776062fa-6f26-4c67-bccb-02c66045f8ed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:30.454622 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-776062fa-6f26-4c67-bccb-02c66045f8ed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:30.477840 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-82f453d9-ab75-4b85-a428-1402d0c346c0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061105 Aug 28 01:11:30.478393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 498/1224] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:30 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:30.492260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-776062fa-6f26-4c67-bccb-02c66045f8ed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056342 Aug 28 01:11:30.492847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 727/1225] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:30 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:31.490896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e719b9ad-9dd6-43ac-95e8-501d4225ae96 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:31.493428 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e719b9ad-9dd6-43ac-95e8-501d4225ae96 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:31.501403 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e719b9ad-9dd6-43ac-95e8-501d4225ae96 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:31.501796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e719b9ad-9dd6-43ac-95e8-501d4225ae96 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:31.502218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e719b9ad-9dd6-43ac-95e8-501d4225ae96 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:31.502916 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e719b9ad-9dd6-43ac-95e8-501d4225ae96 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:31.503357 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e719b9ad-9dd6-43ac-95e8-501d4225ae96 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:31.503769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e719b9ad-9dd6-43ac-95e8-501d4225ae96 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:31.505186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-de2e198e-4ffc-4fde-ad99-9ca80d8c1757 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:31.507469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-de2e198e-4ffc-4fde-ad99-9ca80d8c1757 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:31.515086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de2e198e-4ffc-4fde-ad99-9ca80d8c1757 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:31.515851 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de2e198e-4ffc-4fde-ad99-9ca80d8c1757 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:31.516302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de2e198e-4ffc-4fde-ad99-9ca80d8c1757 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:31.517098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de2e198e-4ffc-4fde-ad99-9ca80d8c1757 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:31.517551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de2e198e-4ffc-4fde-ad99-9ca80d8c1757 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:31.517977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de2e198e-4ffc-4fde-ad99-9ca80d8c1757 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:31.539814 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e719b9ad-9dd6-43ac-95e8-501d4225ae96 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053048 Aug 28 01:11:31.543969 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 499/1226] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:31 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:31.556685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-de2e198e-4ffc-4fde-ad99-9ca80d8c1757 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055107 Aug 28 01:11:31.557871 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 728/1227] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:31 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:32.555067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a7a7f122-ac16-4ac9-a356-c13d0a4626e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:32.557257 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a7a7f122-ac16-4ac9-a356-c13d0a4626e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:32.565807 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7a7f122-ac16-4ac9-a356-c13d0a4626e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:32.566315 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7a7f122-ac16-4ac9-a356-c13d0a4626e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:32.566899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7a7f122-ac16-4ac9-a356-c13d0a4626e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:32.567615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7a7f122-ac16-4ac9-a356-c13d0a4626e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:32.568064 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7a7f122-ac16-4ac9-a356-c13d0a4626e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:32.568369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f5e79958-586e-49b2-b55b-03f79d9e2e5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:32.568862 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7a7f122-ac16-4ac9-a356-c13d0a4626e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:32.571393 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f5e79958-586e-49b2-b55b-03f79d9e2e5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:32.577999 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5e79958-586e-49b2-b55b-03f79d9e2e5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:32.578354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5e79958-586e-49b2-b55b-03f79d9e2e5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:32.589140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5e79958-586e-49b2-b55b-03f79d9e2e5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:32.589140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5e79958-586e-49b2-b55b-03f79d9e2e5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:32.589140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5e79958-586e-49b2-b55b-03f79d9e2e5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:32.589140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5e79958-586e-49b2-b55b-03f79d9e2e5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:32.603241 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a7a7f122-ac16-4ac9-a356-c13d0a4626e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052147 Aug 28 01:11:32.603937 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 500/1228] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:32 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:32.616003 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f5e79958-586e-49b2-b55b-03f79d9e2e5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051693 Aug 28 01:11:32.616567 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 729/1229] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:32 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:33.616374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-33fcd13d-148a-4cc5-bf91-def5479f4d18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:33.618601 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-33fcd13d-148a-4cc5-bf91-def5479f4d18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:33.625569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-33fcd13d-148a-4cc5-bf91-def5479f4d18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:33.626079 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-33fcd13d-148a-4cc5-bf91-def5479f4d18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:33.626518 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-33fcd13d-148a-4cc5-bf91-def5479f4d18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:33.627216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-33fcd13d-148a-4cc5-bf91-def5479f4d18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:33.627688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-33fcd13d-148a-4cc5-bf91-def5479f4d18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:33.628097 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-33fcd13d-148a-4cc5-bf91-def5479f4d18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:33.628605 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6ed3ee0a-10d2-4f30-bed2-07b2693f7708 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:33.631260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6ed3ee0a-10d2-4f30-bed2-07b2693f7708 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:33.637812 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ed3ee0a-10d2-4f30-bed2-07b2693f7708 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:33.638313 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ed3ee0a-10d2-4f30-bed2-07b2693f7708 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:33.638746 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ed3ee0a-10d2-4f30-bed2-07b2693f7708 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:33.639444 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ed3ee0a-10d2-4f30-bed2-07b2693f7708 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:33.639879 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ed3ee0a-10d2-4f30-bed2-07b2693f7708 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:33.640315 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ed3ee0a-10d2-4f30-bed2-07b2693f7708 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:33.662937 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-33fcd13d-148a-4cc5-bf91-def5479f4d18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050599 Aug 28 01:11:33.663532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 501/1230] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:33 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:33.677102 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6ed3ee0a-10d2-4f30-bed2-07b2693f7708 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052891 Aug 28 01:11:33.677795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 730/1231] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:33 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:34.676351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-273d98b7-ce46-498f-a8ab-f24174b83eb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:34.678795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-273d98b7-ce46-498f-a8ab-f24174b83eb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:34.686551 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-273d98b7-ce46-498f-a8ab-f24174b83eb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:34.687055 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-273d98b7-ce46-498f-a8ab-f24174b83eb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:34.687499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-273d98b7-ce46-498f-a8ab-f24174b83eb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:34.688224 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-273d98b7-ce46-498f-a8ab-f24174b83eb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:34.688686 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-273d98b7-ce46-498f-a8ab-f24174b83eb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:34.689150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-273d98b7-ce46-498f-a8ab-f24174b83eb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:34.693301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d23dc2c5-df9b-4996-b0cb-03d5fc96d332 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:34.696287 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d23dc2c5-df9b-4996-b0cb-03d5fc96d332 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:34.704515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d23dc2c5-df9b-4996-b0cb-03d5fc96d332 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:34.705167 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d23dc2c5-df9b-4996-b0cb-03d5fc96d332 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:34.706025 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d23dc2c5-df9b-4996-b0cb-03d5fc96d332 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:34.707192 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d23dc2c5-df9b-4996-b0cb-03d5fc96d332 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:34.707984 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d23dc2c5-df9b-4996-b0cb-03d5fc96d332 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:34.708686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d23dc2c5-df9b-4996-b0cb-03d5fc96d332 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:34.722169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-273d98b7-ce46-498f-a8ab-f24174b83eb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050495 Aug 28 01:11:34.722747 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 502/1232] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:34 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:34.748736 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d23dc2c5-df9b-4996-b0cb-03d5fc96d332 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061877 Aug 28 01:11:34.749568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 731/1233] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:34 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:11:35.735024 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a16dc1c4-15f1-4574-89e8-4df612894916 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:35.737430 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a16dc1c4-15f1-4574-89e8-4df612894916 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:35.744703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a16dc1c4-15f1-4574-89e8-4df612894916 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:35.745225 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a16dc1c4-15f1-4574-89e8-4df612894916 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:35.745651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a16dc1c4-15f1-4574-89e8-4df612894916 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:35.746359 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a16dc1c4-15f1-4574-89e8-4df612894916 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:35.746805 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a16dc1c4-15f1-4574-89e8-4df612894916 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:35.747216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a16dc1c4-15f1-4574-89e8-4df612894916 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:35.761759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f3ee0be7-e800-4b90-85ce-281ab3faedaa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:35.764081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f3ee0be7-e800-4b90-85ce-281ab3faedaa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:35.771461 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3ee0be7-e800-4b90-85ce-281ab3faedaa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:35.772257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3ee0be7-e800-4b90-85ce-281ab3faedaa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:35.772555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3ee0be7-e800-4b90-85ce-281ab3faedaa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:35.773316 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3ee0be7-e800-4b90-85ce-281ab3faedaa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:35.773777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3ee0be7-e800-4b90-85ce-281ab3faedaa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:35.774180 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3ee0be7-e800-4b90-85ce-281ab3faedaa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:35.800030 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a16dc1c4-15f1-4574-89e8-4df612894916 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.068996 Aug 28 01:11:35.801069 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 503/1234] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:35 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:35.809223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f3ee0be7-e800-4b90-85ce-281ab3faedaa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051777 Aug 28 01:11:35.810023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 732/1235] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:35 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:36.813359 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a9951f48-4f05-4d17-9cd4-d37e90f973ae tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:36.815584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a9951f48-4f05-4d17-9cd4-d37e90f973ae tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:36.823036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a9951f48-4f05-4d17-9cd4-d37e90f973ae tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:36.823542 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a9951f48-4f05-4d17-9cd4-d37e90f973ae tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:36.823908 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a3f03b3c-e0a1-4358-8a0d-8a06073a0c03 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:36.824379 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a9951f48-4f05-4d17-9cd4-d37e90f973ae tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:36.824735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a9951f48-4f05-4d17-9cd4-d37e90f973ae tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:36.825187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a9951f48-4f05-4d17-9cd4-d37e90f973ae tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:36.825614 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a9951f48-4f05-4d17-9cd4-d37e90f973ae tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:36.826271 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a3f03b3c-e0a1-4358-8a0d-8a06073a0c03 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:36.833671 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a3f03b3c-e0a1-4358-8a0d-8a06073a0c03 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:36.834189 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a3f03b3c-e0a1-4358-8a0d-8a06073a0c03 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:36.834640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a3f03b3c-e0a1-4358-8a0d-8a06073a0c03 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:36.835377 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a3f03b3c-e0a1-4358-8a0d-8a06073a0c03 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:36.835889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a3f03b3c-e0a1-4358-8a0d-8a06073a0c03 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:36.836282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a3f03b3c-e0a1-4358-8a0d-8a06073a0c03 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:36.871554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a3f03b3c-e0a1-4358-8a0d-8a06073a0c03 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052333 Aug 28 01:11:36.872115 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 733/1236] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:36 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:37.036433 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a9951f48-4f05-4d17-9cd4-d37e90f973ae tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.227163 Aug 28 01:11:37.037055 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 504/1237] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:36 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:37.884992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b5eba8a7-67e1-4804-8694-bf0078892d89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:37.887474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b5eba8a7-67e1-4804-8694-bf0078892d89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:37.894476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5eba8a7-67e1-4804-8694-bf0078892d89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:37.894967 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5eba8a7-67e1-4804-8694-bf0078892d89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:37.895419 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5eba8a7-67e1-4804-8694-bf0078892d89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:37.896103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5eba8a7-67e1-4804-8694-bf0078892d89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:37.896584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5eba8a7-67e1-4804-8694-bf0078892d89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:37.896999 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5eba8a7-67e1-4804-8694-bf0078892d89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:37.928628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b5eba8a7-67e1-4804-8694-bf0078892d89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047771 Aug 28 01:11:37.930565 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 734/1238] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:37 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:38.049683 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d53deebf-a6f0-43bd-bb14-0daf88445746 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:38.051951 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d53deebf-a6f0-43bd-bb14-0daf88445746 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:38.059318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d53deebf-a6f0-43bd-bb14-0daf88445746 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:38.059787 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d53deebf-a6f0-43bd-bb14-0daf88445746 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:38.060228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d53deebf-a6f0-43bd-bb14-0daf88445746 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:38.060920 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d53deebf-a6f0-43bd-bb14-0daf88445746 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:38.061404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d53deebf-a6f0-43bd-bb14-0daf88445746 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:38.061819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d53deebf-a6f0-43bd-bb14-0daf88445746 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:38.093357 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d53deebf-a6f0-43bd-bb14-0daf88445746 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047862 Aug 28 01:11:38.093877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 505/1239] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:38 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:38.943036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c73b8097-7893-4ecc-92f1-48e8ef6ead89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:38.945765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c73b8097-7893-4ecc-92f1-48e8ef6ead89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:38.953365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c73b8097-7893-4ecc-92f1-48e8ef6ead89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:38.954253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c73b8097-7893-4ecc-92f1-48e8ef6ead89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:38.954253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c73b8097-7893-4ecc-92f1-48e8ef6ead89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:38.954931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c73b8097-7893-4ecc-92f1-48e8ef6ead89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:38.955379 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c73b8097-7893-4ecc-92f1-48e8ef6ead89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:38.955818 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c73b8097-7893-4ecc-92f1-48e8ef6ead89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:38.973729 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:11:38.988768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c73b8097-7893-4ecc-92f1-48e8ef6ead89 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049307 Aug 28 01:11:38.989792 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 735/1240] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:38 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:39.106592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f4d7d756-de7f-4334-8d47-703428310bd8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:39.108752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f4d7d756-de7f-4334-8d47-703428310bd8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:39.115570 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4d7d756-de7f-4334-8d47-703428310bd8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:39.116315 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4d7d756-de7f-4334-8d47-703428310bd8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:39.116807 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4d7d756-de7f-4334-8d47-703428310bd8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:39.117537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4d7d756-de7f-4334-8d47-703428310bd8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:39.117986 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4d7d756-de7f-4334-8d47-703428310bd8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:39.118407 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4d7d756-de7f-4334-8d47-703428310bd8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:39.149058 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f4d7d756-de7f-4334-8d47-703428310bd8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046658 Aug 28 01:11:39.149961 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 506/1241] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:39 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:40.002656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1ee8844e-fd1f-4fa5-a80d-3cdd4ebe8ef7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:40.004826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1ee8844e-fd1f-4fa5-a80d-3cdd4ebe8ef7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:40.011530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ee8844e-fd1f-4fa5-a80d-3cdd4ebe8ef7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:40.012036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ee8844e-fd1f-4fa5-a80d-3cdd4ebe8ef7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:40.012514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ee8844e-fd1f-4fa5-a80d-3cdd4ebe8ef7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:40.013234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ee8844e-fd1f-4fa5-a80d-3cdd4ebe8ef7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:40.013683 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ee8844e-fd1f-4fa5-a80d-3cdd4ebe8ef7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:40.014104 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ee8844e-fd1f-4fa5-a80d-3cdd4ebe8ef7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:40.049977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1ee8844e-fd1f-4fa5-a80d-3cdd4ebe8ef7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051299 Aug 28 01:11:40.050642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 736/1242] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:39 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:40.162473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f15b22ef-cb8a-44be-95fb-a6dbbe0e2c0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:40.164907 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f15b22ef-cb8a-44be-95fb-a6dbbe0e2c0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:40.171704 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f15b22ef-cb8a-44be-95fb-a6dbbe0e2c0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:40.172211 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f15b22ef-cb8a-44be-95fb-a6dbbe0e2c0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:40.172649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f15b22ef-cb8a-44be-95fb-a6dbbe0e2c0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:40.173621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f15b22ef-cb8a-44be-95fb-a6dbbe0e2c0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:40.174078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f15b22ef-cb8a-44be-95fb-a6dbbe0e2c0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:40.174508 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f15b22ef-cb8a-44be-95fb-a6dbbe0e2c0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:40.202090 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:11:40.207735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f15b22ef-cb8a-44be-95fb-a6dbbe0e2c0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049441 Aug 28 01:11:40.208308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 507/1243] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:40 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:41.063175 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-544e6f9d-8c72-4bc3-ad52-4cc23f3e5d16 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:41.065405 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-544e6f9d-8c72-4bc3-ad52-4cc23f3e5d16 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:41.073151 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-544e6f9d-8c72-4bc3-ad52-4cc23f3e5d16 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:41.073641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-544e6f9d-8c72-4bc3-ad52-4cc23f3e5d16 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:41.074075 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-544e6f9d-8c72-4bc3-ad52-4cc23f3e5d16 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:41.074807 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-544e6f9d-8c72-4bc3-ad52-4cc23f3e5d16 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:41.075260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-544e6f9d-8c72-4bc3-ad52-4cc23f3e5d16 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:41.075689 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-544e6f9d-8c72-4bc3-ad52-4cc23f3e5d16 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:41.106713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-544e6f9d-8c72-4bc3-ad52-4cc23f3e5d16 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047818 Aug 28 01:11:41.107604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 737/1244] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:41 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:41.340621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bc3f7914-0d85-407e-b7c5-5a9249d8c3d7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:41.342851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bc3f7914-0d85-407e-b7c5-5a9249d8c3d7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:41.350209 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc3f7914-0d85-407e-b7c5-5a9249d8c3d7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:41.351207 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc3f7914-0d85-407e-b7c5-5a9249d8c3d7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:41.351857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc3f7914-0d85-407e-b7c5-5a9249d8c3d7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:41.352936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc3f7914-0d85-407e-b7c5-5a9249d8c3d7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:41.353621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc3f7914-0d85-407e-b7c5-5a9249d8c3d7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:41.354270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc3f7914-0d85-407e-b7c5-5a9249d8c3d7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:41.397957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bc3f7914-0d85-407e-b7c5-5a9249d8c3d7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.181463 Aug 28 01:11:41.398781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 508/1245] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:41 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:42.120029 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-680f4269-4192-484e-aab6-6b9d4f4012d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:42.122608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-680f4269-4192-484e-aab6-6b9d4f4012d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:42.130101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-680f4269-4192-484e-aab6-6b9d4f4012d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:42.130628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-680f4269-4192-484e-aab6-6b9d4f4012d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:42.131067 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-680f4269-4192-484e-aab6-6b9d4f4012d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:42.131804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-680f4269-4192-484e-aab6-6b9d4f4012d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:42.132247 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-680f4269-4192-484e-aab6-6b9d4f4012d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:42.132676 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-680f4269-4192-484e-aab6-6b9d4f4012d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:42.165776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-680f4269-4192-484e-aab6-6b9d4f4012d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049884 Aug 28 01:11:42.166388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 738/1246] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:42 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:42.411401 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f4ba9740-7e61-497b-b4d8-20d9192cb904 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:42.413968 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f4ba9740-7e61-497b-b4d8-20d9192cb904 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:42.421658 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4ba9740-7e61-497b-b4d8-20d9192cb904 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:42.422158 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4ba9740-7e61-497b-b4d8-20d9192cb904 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:42.422600 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4ba9740-7e61-497b-b4d8-20d9192cb904 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:42.423373 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4ba9740-7e61-497b-b4d8-20d9192cb904 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:42.423826 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4ba9740-7e61-497b-b4d8-20d9192cb904 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:42.424222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4ba9740-7e61-497b-b4d8-20d9192cb904 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:42.456042 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f4ba9740-7e61-497b-b4d8-20d9192cb904 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048829 Aug 28 01:11:42.456618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 509/1247] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:42 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:43.178418 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0376bdbf-1309-488a-919f-fa8cc3b49c41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:43.180578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0376bdbf-1309-488a-919f-fa8cc3b49c41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:43.187220 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0376bdbf-1309-488a-919f-fa8cc3b49c41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:43.187700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0376bdbf-1309-488a-919f-fa8cc3b49c41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:43.188160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0376bdbf-1309-488a-919f-fa8cc3b49c41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:43.188888 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0376bdbf-1309-488a-919f-fa8cc3b49c41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:43.189337 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0376bdbf-1309-488a-919f-fa8cc3b49c41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:43.189754 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0376bdbf-1309-488a-919f-fa8cc3b49c41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:43.221684 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0376bdbf-1309-488a-919f-fa8cc3b49c41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047253 Aug 28 01:11:43.222833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 739/1248] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:43 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:43.470020 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-650ad4c5-bcd9-4b59-bb9a-c41bd7bcfb2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:43.472420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-650ad4c5-bcd9-4b59-bb9a-c41bd7bcfb2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:43.480053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-650ad4c5-bcd9-4b59-bb9a-c41bd7bcfb2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:43.480724 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-650ad4c5-bcd9-4b59-bb9a-c41bd7bcfb2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:43.481326 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-650ad4c5-bcd9-4b59-bb9a-c41bd7bcfb2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:43.482225 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-650ad4c5-bcd9-4b59-bb9a-c41bd7bcfb2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:43.482822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-650ad4c5-bcd9-4b59-bb9a-c41bd7bcfb2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:43.483356 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-650ad4c5-bcd9-4b59-bb9a-c41bd7bcfb2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:43.517777 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-650ad4c5-bcd9-4b59-bb9a-c41bd7bcfb2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052406 Aug 28 01:11:43.518256 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 510/1249] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:43 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:44.237508 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-61026141-35f8-4ac5-9a77-e29db6c0fcd1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:44.240325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-61026141-35f8-4ac5-9a77-e29db6c0fcd1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:44.243460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=77 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:11:44.249465 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61026141-35f8-4ac5-9a77-e29db6c0fcd1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:44.249919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61026141-35f8-4ac5-9a77-e29db6c0fcd1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:44.250348 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61026141-35f8-4ac5-9a77-e29db6c0fcd1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:44.251043 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61026141-35f8-4ac5-9a77-e29db6c0fcd1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:44.251490 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61026141-35f8-4ac5-9a77-e29db6c0fcd1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:44.251909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61026141-35f8-4ac5-9a77-e29db6c0fcd1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:44.283323 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-61026141-35f8-4ac5-9a77-e29db6c0fcd1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051825 Aug 28 01:11:44.283874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 740/1250] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:44 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:44.531032 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-23af8778-f004-45e4-97ef-941ed3df0eea tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:44.533282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-23af8778-f004-45e4-97ef-941ed3df0eea tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:44.535795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=89 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:11:44.541924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23af8778-f004-45e4-97ef-941ed3df0eea tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:44.542476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23af8778-f004-45e4-97ef-941ed3df0eea tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:44.543011 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23af8778-f004-45e4-97ef-941ed3df0eea tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:44.543953 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23af8778-f004-45e4-97ef-941ed3df0eea tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:44.544432 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23af8778-f004-45e4-97ef-941ed3df0eea tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:44.544925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23af8778-f004-45e4-97ef-941ed3df0eea tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:44.576877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-23af8778-f004-45e4-97ef-941ed3df0eea tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050144 Aug 28 01:11:44.577446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 511/1251] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:44 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:45.298508 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2e588222-597a-44c2-a991-1990256e688d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:45.300553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2e588222-597a-44c2-a991-1990256e688d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:45.307475 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2e588222-597a-44c2-a991-1990256e688d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:45.307929 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2e588222-597a-44c2-a991-1990256e688d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:45.308349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2e588222-597a-44c2-a991-1990256e688d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:45.309045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2e588222-597a-44c2-a991-1990256e688d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:45.309493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2e588222-597a-44c2-a991-1990256e688d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:45.309906 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2e588222-597a-44c2-a991-1990256e688d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:45.340641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2e588222-597a-44c2-a991-1990256e688d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047444 Aug 28 01:11:45.341239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 741/1252] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:45 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:45.589801 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-398dff4e-2907-4d0e-a870-f8728af68aab tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:45.592017 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-398dff4e-2907-4d0e-a870-f8728af68aab tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:45.598851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-398dff4e-2907-4d0e-a870-f8728af68aab tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:45.599271 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-398dff4e-2907-4d0e-a870-f8728af68aab tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:45.599738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-398dff4e-2907-4d0e-a870-f8728af68aab tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:45.600443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-398dff4e-2907-4d0e-a870-f8728af68aab tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:45.600892 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-398dff4e-2907-4d0e-a870-f8728af68aab tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:45.601342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-398dff4e-2907-4d0e-a870-f8728af68aab tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:45.630953 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-398dff4e-2907-4d0e-a870-f8728af68aab tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045119 Aug 28 01:11:45.631500 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 512/1253] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:45 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:46.352929 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-24989914-17e3-4492-8f83-1fc2768a4393 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:46.355096 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-24989914-17e3-4492-8f83-1fc2768a4393 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:46.362197 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24989914-17e3-4492-8f83-1fc2768a4393 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:46.362678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24989914-17e3-4492-8f83-1fc2768a4393 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:46.363108 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24989914-17e3-4492-8f83-1fc2768a4393 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:46.363847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24989914-17e3-4492-8f83-1fc2768a4393 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:46.364274 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24989914-17e3-4492-8f83-1fc2768a4393 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:46.364695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24989914-17e3-4492-8f83-1fc2768a4393 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:46.396132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-24989914-17e3-4492-8f83-1fc2768a4393 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047354 Aug 28 01:11:46.396729 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 742/1254] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:46 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:46.644431 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-98d0da49-85e5-4844-8040-481e4295b69b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:46.646649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-98d0da49-85e5-4844-8040-481e4295b69b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:46.653435 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98d0da49-85e5-4844-8040-481e4295b69b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:46.653914 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98d0da49-85e5-4844-8040-481e4295b69b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:46.654351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98d0da49-85e5-4844-8040-481e4295b69b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:46.655047 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98d0da49-85e5-4844-8040-481e4295b69b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:46.655499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98d0da49-85e5-4844-8040-481e4295b69b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:46.655925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98d0da49-85e5-4844-8040-481e4295b69b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:46.685443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-98d0da49-85e5-4844-8040-481e4295b69b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045006 Aug 28 01:11:46.686007 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 513/1255] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:46 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:47.408113 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-15462a3a-eb01-4478-86b3-774bd4c1c2ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:47.410363 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-15462a3a-eb01-4478-86b3-774bd4c1c2ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:47.417191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15462a3a-eb01-4478-86b3-774bd4c1c2ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:47.417674 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15462a3a-eb01-4478-86b3-774bd4c1c2ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:47.418111 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15462a3a-eb01-4478-86b3-774bd4c1c2ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:47.418800 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15462a3a-eb01-4478-86b3-774bd4c1c2ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:47.419250 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15462a3a-eb01-4478-86b3-774bd4c1c2ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:47.419668 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-15462a3a-eb01-4478-86b3-774bd4c1c2ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:47.451152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-15462a3a-eb01-4478-86b3-774bd4c1c2ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047015 Aug 28 01:11:47.452078 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 743/1256] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:47 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:47.699740 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c9b27d9f-f82c-4959-a9f0-4ad597a95bbd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:47.702662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c9b27d9f-f82c-4959-a9f0-4ad597a95bbd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:47.710184 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9b27d9f-f82c-4959-a9f0-4ad597a95bbd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:47.710747 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9b27d9f-f82c-4959-a9f0-4ad597a95bbd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:47.711492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9b27d9f-f82c-4959-a9f0-4ad597a95bbd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:47.712401 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9b27d9f-f82c-4959-a9f0-4ad597a95bbd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:47.712867 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9b27d9f-f82c-4959-a9f0-4ad597a95bbd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:47.713327 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9b27d9f-f82c-4959-a9f0-4ad597a95bbd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:47.748182 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c9b27d9f-f82c-4959-a9f0-4ad597a95bbd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054218 Aug 28 01:11:47.748818 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 514/1257] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:47 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:48.466946 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-528b1d2f-fb9f-4495-81f1-bc57af526921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:48.469602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-528b1d2f-fb9f-4495-81f1-bc57af526921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:48.476573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-528b1d2f-fb9f-4495-81f1-bc57af526921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:48.477124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-528b1d2f-fb9f-4495-81f1-bc57af526921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:48.477559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-528b1d2f-fb9f-4495-81f1-bc57af526921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:48.478246 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-528b1d2f-fb9f-4495-81f1-bc57af526921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:48.478708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-528b1d2f-fb9f-4495-81f1-bc57af526921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:48.479133 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-528b1d2f-fb9f-4495-81f1-bc57af526921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:48.519371 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-528b1d2f-fb9f-4495-81f1-bc57af526921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058871 Aug 28 01:11:48.520019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 744/1258] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:48 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:48.760508 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5a4d364d-676e-43e5-9333-6b0b28b58498 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:48.762734 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5a4d364d-676e-43e5-9333-6b0b28b58498 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:48.769845 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a4d364d-676e-43e5-9333-6b0b28b58498 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:48.770441 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a4d364d-676e-43e5-9333-6b0b28b58498 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:48.770936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a4d364d-676e-43e5-9333-6b0b28b58498 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:48.771804 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a4d364d-676e-43e5-9333-6b0b28b58498 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:48.772324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a4d364d-676e-43e5-9333-6b0b28b58498 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:48.772870 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a4d364d-676e-43e5-9333-6b0b28b58498 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:48.806056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5a4d364d-676e-43e5-9333-6b0b28b58498 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049918 Aug 28 01:11:48.806594 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 515/1259] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:48 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:49.535263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-62513fc0-c5e9-400e-a669-15644da4ba00 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:49.538424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-62513fc0-c5e9-400e-a669-15644da4ba00 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:49.548505 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62513fc0-c5e9-400e-a669-15644da4ba00 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:49.549321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62513fc0-c5e9-400e-a669-15644da4ba00 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:49.549973 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62513fc0-c5e9-400e-a669-15644da4ba00 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:49.551035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62513fc0-c5e9-400e-a669-15644da4ba00 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:49.551747 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62513fc0-c5e9-400e-a669-15644da4ba00 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:49.552167 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62513fc0-c5e9-400e-a669-15644da4ba00 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:49.584349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-62513fc0-c5e9-400e-a669-15644da4ba00 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053981 Aug 28 01:11:49.585306 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 745/1260] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:49 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:49.819438 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6f74c0d0-804c-4be0-a51c-2674f65e0c49 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:49.821691 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6f74c0d0-804c-4be0-a51c-2674f65e0c49 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:49.828647 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f74c0d0-804c-4be0-a51c-2674f65e0c49 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:49.829182 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f74c0d0-804c-4be0-a51c-2674f65e0c49 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:49.829609 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f74c0d0-804c-4be0-a51c-2674f65e0c49 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:49.830353 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f74c0d0-804c-4be0-a51c-2674f65e0c49 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:49.830801 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f74c0d0-804c-4be0-a51c-2674f65e0c49 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:49.831214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f74c0d0-804c-4be0-a51c-2674f65e0c49 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:49.869596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6f74c0d0-804c-4be0-a51c-2674f65e0c49 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054265 Aug 28 01:11:49.871300 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 516/1261] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:49 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:50.599815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-648083f5-a7eb-4d22-99eb-8f253ef4c375 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:50.602351 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-648083f5-a7eb-4d22-99eb-8f253ef4c375 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:50.609243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-648083f5-a7eb-4d22-99eb-8f253ef4c375 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:50.609732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-648083f5-a7eb-4d22-99eb-8f253ef4c375 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:50.610170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-648083f5-a7eb-4d22-99eb-8f253ef4c375 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:50.610870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-648083f5-a7eb-4d22-99eb-8f253ef4c375 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:50.611313 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-648083f5-a7eb-4d22-99eb-8f253ef4c375 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:50.611733 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-648083f5-a7eb-4d22-99eb-8f253ef4c375 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:50.641704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-648083f5-a7eb-4d22-99eb-8f253ef4c375 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047574 Aug 28 01:11:50.642331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 746/1262] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:50 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:11:50.884393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e986e908-02ce-4f25-bad4-5f0b3c1046c9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:50.887532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e986e908-02ce-4f25-bad4-5f0b3c1046c9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:50.894693 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e986e908-02ce-4f25-bad4-5f0b3c1046c9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:50.895209 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e986e908-02ce-4f25-bad4-5f0b3c1046c9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:50.896199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e986e908-02ce-4f25-bad4-5f0b3c1046c9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:50.896902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e986e908-02ce-4f25-bad4-5f0b3c1046c9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:50.897373 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e986e908-02ce-4f25-bad4-5f0b3c1046c9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:50.898059 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e986e908-02ce-4f25-bad4-5f0b3c1046c9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:50.928819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e986e908-02ce-4f25-bad4-5f0b3c1046c9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049296 Aug 28 01:11:50.929451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 517/1263] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:50 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:51.654643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6c19e1ed-bca3-4d5f-8b91-6a09251f8c0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:51.656835 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6c19e1ed-bca3-4d5f-8b91-6a09251f8c0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:51.664345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c19e1ed-bca3-4d5f-8b91-6a09251f8c0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:51.664820 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c19e1ed-bca3-4d5f-8b91-6a09251f8c0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:51.665280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c19e1ed-bca3-4d5f-8b91-6a09251f8c0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:51.665964 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c19e1ed-bca3-4d5f-8b91-6a09251f8c0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:51.666405 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c19e1ed-bca3-4d5f-8b91-6a09251f8c0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:51.666820 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c19e1ed-bca3-4d5f-8b91-6a09251f8c0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:51.698898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6c19e1ed-bca3-4d5f-8b91-6a09251f8c0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048239 Aug 28 01:11:51.699446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 747/1264] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:51 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:51.942049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-adc90688-110d-4d86-bc86-c932cd59ef3e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:51.944548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-adc90688-110d-4d86-bc86-c932cd59ef3e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:51.951530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-adc90688-110d-4d86-bc86-c932cd59ef3e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:51.952128 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-adc90688-110d-4d86-bc86-c932cd59ef3e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:51.952651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-adc90688-110d-4d86-bc86-c932cd59ef3e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:51.953617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-adc90688-110d-4d86-bc86-c932cd59ef3e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:51.954092 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-adc90688-110d-4d86-bc86-c932cd59ef3e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:51.954456 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-adc90688-110d-4d86-bc86-c932cd59ef3e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:51.986857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-adc90688-110d-4d86-bc86-c932cd59ef3e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048721 Aug 28 01:11:51.987413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 518/1265] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:51 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:52.710935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6e8792f0-f6f1-4545-b04f-b4d05e14a059 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:52.713182 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6e8792f0-f6f1-4545-b04f-b4d05e14a059 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:52.720242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e8792f0-f6f1-4545-b04f-b4d05e14a059 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:52.720733 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e8792f0-f6f1-4545-b04f-b4d05e14a059 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:52.721206 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e8792f0-f6f1-4545-b04f-b4d05e14a059 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:52.721915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e8792f0-f6f1-4545-b04f-b4d05e14a059 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:52.722367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e8792f0-f6f1-4545-b04f-b4d05e14a059 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:52.722789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e8792f0-f6f1-4545-b04f-b4d05e14a059 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:52.753883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6e8792f0-f6f1-4545-b04f-b4d05e14a059 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046927 Aug 28 01:11:52.754461 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 748/1266] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:52 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:53.002856 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-70227597-4480-43bb-9614-ee39ec05027b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:53.006215 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-70227597-4480-43bb-9614-ee39ec05027b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:53.028687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70227597-4480-43bb-9614-ee39ec05027b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:53.028687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70227597-4480-43bb-9614-ee39ec05027b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:53.028687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70227597-4480-43bb-9614-ee39ec05027b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:53.028687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70227597-4480-43bb-9614-ee39ec05027b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:53.029035 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70227597-4480-43bb-9614-ee39ec05027b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:53.029035 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70227597-4480-43bb-9614-ee39ec05027b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:53.052860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-70227597-4480-43bb-9614-ee39ec05027b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056260 Aug 28 01:11:53.053447 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 519/1267] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:52 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:53.768449 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b1eef6a6-35aa-4d9d-acde-9b6248632a2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:53.770649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b1eef6a6-35aa-4d9d-acde-9b6248632a2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:53.777684 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b1eef6a6-35aa-4d9d-acde-9b6248632a2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:53.778162 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b1eef6a6-35aa-4d9d-acde-9b6248632a2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:53.778688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b1eef6a6-35aa-4d9d-acde-9b6248632a2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:53.779383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b1eef6a6-35aa-4d9d-acde-9b6248632a2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:53.779823 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b1eef6a6-35aa-4d9d-acde-9b6248632a2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:53.780240 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b1eef6a6-35aa-4d9d-acde-9b6248632a2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:53.809409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b1eef6a6-35aa-4d9d-acde-9b6248632a2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044892 Aug 28 01:11:53.810039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 749/1268] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:53 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:54.065627 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e20a864b-92f2-465a-8508-14a7ee994cdb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:54.068446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e20a864b-92f2-465a-8508-14a7ee994cdb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:54.074839 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e20a864b-92f2-465a-8508-14a7ee994cdb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:54.075322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e20a864b-92f2-465a-8508-14a7ee994cdb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:54.076114 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e20a864b-92f2-465a-8508-14a7ee994cdb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:54.076796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e20a864b-92f2-465a-8508-14a7ee994cdb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:54.077387 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e20a864b-92f2-465a-8508-14a7ee994cdb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:54.077787 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e20a864b-92f2-465a-8508-14a7ee994cdb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:54.108490 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e20a864b-92f2-465a-8508-14a7ee994cdb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046651 Aug 28 01:11:54.109516 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 520/1269] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:54 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:54.822890 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7ca8d567-65e8-48ba-bf40-4a01dbdf7a52 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:54.825067 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7ca8d567-65e8-48ba-bf40-4a01dbdf7a52 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:54.832000 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ca8d567-65e8-48ba-bf40-4a01dbdf7a52 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:54.832478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ca8d567-65e8-48ba-bf40-4a01dbdf7a52 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:54.832911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ca8d567-65e8-48ba-bf40-4a01dbdf7a52 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:54.833644 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ca8d567-65e8-48ba-bf40-4a01dbdf7a52 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:54.834119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ca8d567-65e8-48ba-bf40-4a01dbdf7a52 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:54.834554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ca8d567-65e8-48ba-bf40-4a01dbdf7a52 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:54.866504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7ca8d567-65e8-48ba-bf40-4a01dbdf7a52 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047900 Aug 28 01:11:54.867065 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 750/1270] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:54 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:55.120136 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e6dfeeb9-ea23-46fe-9572-5bf072901b0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:55.123386 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e6dfeeb9-ea23-46fe-9572-5bf072901b0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:55.129988 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6dfeeb9-ea23-46fe-9572-5bf072901b0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:55.130475 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6dfeeb9-ea23-46fe-9572-5bf072901b0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:55.130901 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6dfeeb9-ea23-46fe-9572-5bf072901b0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:55.131589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6dfeeb9-ea23-46fe-9572-5bf072901b0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:55.132029 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6dfeeb9-ea23-46fe-9572-5bf072901b0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:55.132441 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6dfeeb9-ea23-46fe-9572-5bf072901b0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:55.167895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e6dfeeb9-ea23-46fe-9572-5bf072901b0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051618 Aug 28 01:11:55.169223 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 521/1271] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:55 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:55.880901 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5806c407-d13f-4405-ae9a-fc6a56693c14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:55.883626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5806c407-d13f-4405-ae9a-fc6a56693c14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:55.891330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5806c407-d13f-4405-ae9a-fc6a56693c14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:55.891810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5806c407-d13f-4405-ae9a-fc6a56693c14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:55.892245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5806c407-d13f-4405-ae9a-fc6a56693c14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:55.893082 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5806c407-d13f-4405-ae9a-fc6a56693c14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:55.893414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5806c407-d13f-4405-ae9a-fc6a56693c14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:55.893831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5806c407-d13f-4405-ae9a-fc6a56693c14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:55.924723 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5806c407-d13f-4405-ae9a-fc6a56693c14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048826 Aug 28 01:11:55.925316 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 751/1272] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:55 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:56.180110 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a3ddc3c8-9aed-4312-94c5-94978fa25bf7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:56.182280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a3ddc3c8-9aed-4312-94c5-94978fa25bf7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:56.189283 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3ddc3c8-9aed-4312-94c5-94978fa25bf7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:56.189803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3ddc3c8-9aed-4312-94c5-94978fa25bf7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:56.190222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3ddc3c8-9aed-4312-94c5-94978fa25bf7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:56.190921 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3ddc3c8-9aed-4312-94c5-94978fa25bf7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:56.191406 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3ddc3c8-9aed-4312-94c5-94978fa25bf7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:56.191781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3ddc3c8-9aed-4312-94c5-94978fa25bf7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:56.226270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a3ddc3c8-9aed-4312-94c5-94978fa25bf7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050025 Aug 28 01:11:56.227035 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 522/1273] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:56 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:56.938012 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0087ec0b-3702-4f55-a236-65b9cb68f0f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:56.940169 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0087ec0b-3702-4f55-a236-65b9cb68f0f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:56.947382 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0087ec0b-3702-4f55-a236-65b9cb68f0f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:56.947865 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0087ec0b-3702-4f55-a236-65b9cb68f0f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:56.948295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0087ec0b-3702-4f55-a236-65b9cb68f0f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:56.949007 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0087ec0b-3702-4f55-a236-65b9cb68f0f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:56.949474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0087ec0b-3702-4f55-a236-65b9cb68f0f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:56.949894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0087ec0b-3702-4f55-a236-65b9cb68f0f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:56.953745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:11:56.986803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0087ec0b-3702-4f55-a236-65b9cb68f0f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052476 Aug 28 01:11:56.987502 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 752/1274] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:56 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:57.238768 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-35d46149-0b73-4398-b7ac-67cd40b69522 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:57.240952 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-35d46149-0b73-4398-b7ac-67cd40b69522 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:57.247735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-35d46149-0b73-4398-b7ac-67cd40b69522 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:57.248226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-35d46149-0b73-4398-b7ac-67cd40b69522 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:57.248654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-35d46149-0b73-4398-b7ac-67cd40b69522 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:57.249393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-35d46149-0b73-4398-b7ac-67cd40b69522 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:57.249837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-35d46149-0b73-4398-b7ac-67cd40b69522 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:57.250252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-35d46149-0b73-4398-b7ac-67cd40b69522 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:57.281962 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-35d46149-0b73-4398-b7ac-67cd40b69522 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046903 Aug 28 01:11:57.286492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 523/1275] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:57 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:58.002814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-84991b05-da47-4f4d-baff-4e2393104c14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:58.005690 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-84991b05-da47-4f4d-baff-4e2393104c14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:58.014202 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84991b05-da47-4f4d-baff-4e2393104c14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:58.014677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84991b05-da47-4f4d-baff-4e2393104c14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:58.015105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84991b05-da47-4f4d-baff-4e2393104c14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:58.015789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84991b05-da47-4f4d-baff-4e2393104c14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:58.016231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84991b05-da47-4f4d-baff-4e2393104c14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:58.016646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84991b05-da47-4f4d-baff-4e2393104c14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:58.047293 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-84991b05-da47-4f4d-baff-4e2393104c14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049830 Aug 28 01:11:58.047858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 753/1276] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:57 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:58.294259 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bb56f210-9a51-48ab-afb6-88fb964c4151 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:58.296466 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bb56f210-9a51-48ab-afb6-88fb964c4151 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:58.303288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb56f210-9a51-48ab-afb6-88fb964c4151 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:58.303802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb56f210-9a51-48ab-afb6-88fb964c4151 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:58.304338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb56f210-9a51-48ab-afb6-88fb964c4151 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:58.305074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb56f210-9a51-48ab-afb6-88fb964c4151 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:58.305525 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb56f210-9a51-48ab-afb6-88fb964c4151 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:58.305941 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb56f210-9a51-48ab-afb6-88fb964c4151 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:58.322991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:11:58.337268 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bb56f210-9a51-48ab-afb6-88fb964c4151 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047094 Aug 28 01:11:58.338013 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 524/1277] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:58 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:59.060231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-06a31047-d64d-457a-b081-d8e232f50bb9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:59.062504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-06a31047-d64d-457a-b081-d8e232f50bb9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:59.069915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-06a31047-d64d-457a-b081-d8e232f50bb9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:59.070394 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-06a31047-d64d-457a-b081-d8e232f50bb9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:59.070821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-06a31047-d64d-457a-b081-d8e232f50bb9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:59.071510 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-06a31047-d64d-457a-b081-d8e232f50bb9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:59.071951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-06a31047-d64d-457a-b081-d8e232f50bb9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:59.072367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-06a31047-d64d-457a-b081-d8e232f50bb9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:59.105364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-06a31047-d64d-457a-b081-d8e232f50bb9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049092 Aug 28 01:11:59.105923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 754/1278] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:59 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:11:59.350083 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8fbdee01-8228-474a-b5bc-44b35cf37eb7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:11:59.352148 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8fbdee01-8228-474a-b5bc-44b35cf37eb7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:11:59.359264 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fbdee01-8228-474a-b5bc-44b35cf37eb7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:59.359755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fbdee01-8228-474a-b5bc-44b35cf37eb7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:59.360187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fbdee01-8228-474a-b5bc-44b35cf37eb7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:59.360894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fbdee01-8228-474a-b5bc-44b35cf37eb7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:11:59.361375 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fbdee01-8228-474a-b5bc-44b35cf37eb7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:11:59.361795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fbdee01-8228-474a-b5bc-44b35cf37eb7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:11:59.392685 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8fbdee01-8228-474a-b5bc-44b35cf37eb7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046599 Aug 28 01:11:59.393616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 525/1279] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:11:59 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:00.118261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-75d8d94a-e69b-436d-9d0d-14833a9d1caf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:00.120567 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-75d8d94a-e69b-436d-9d0d-14833a9d1caf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:00.127403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75d8d94a-e69b-436d-9d0d-14833a9d1caf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:00.127886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75d8d94a-e69b-436d-9d0d-14833a9d1caf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:00.128321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75d8d94a-e69b-436d-9d0d-14833a9d1caf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:00.129061 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75d8d94a-e69b-436d-9d0d-14833a9d1caf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:00.129512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75d8d94a-e69b-436d-9d0d-14833a9d1caf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:00.129938 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75d8d94a-e69b-436d-9d0d-14833a9d1caf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:00.163271 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-75d8d94a-e69b-436d-9d0d-14833a9d1caf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048912 Aug 28 01:12:00.163891 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 755/1280] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:00 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:00.405258 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5af9173b-39b6-4333-857b-d62cd05ab77d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:00.407423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5af9173b-39b6-4333-857b-d62cd05ab77d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:00.414878 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5af9173b-39b6-4333-857b-d62cd05ab77d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:00.415379 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5af9173b-39b6-4333-857b-d62cd05ab77d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:00.415823 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5af9173b-39b6-4333-857b-d62cd05ab77d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:00.416513 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5af9173b-39b6-4333-857b-d62cd05ab77d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:00.416998 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5af9173b-39b6-4333-857b-d62cd05ab77d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:00.417410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5af9173b-39b6-4333-857b-d62cd05ab77d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:00.448419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5af9173b-39b6-4333-857b-d62cd05ab77d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047118 Aug 28 01:12:00.449229 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 526/1281] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:00 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:01.178574 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-61bb5b29-a6a0-4c88-9d42-048e83aec447 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:01.180799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-61bb5b29-a6a0-4c88-9d42-048e83aec447 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:01.188014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61bb5b29-a6a0-4c88-9d42-048e83aec447 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:01.188496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61bb5b29-a6a0-4c88-9d42-048e83aec447 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:01.188944 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61bb5b29-a6a0-4c88-9d42-048e83aec447 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:01.189678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61bb5b29-a6a0-4c88-9d42-048e83aec447 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:01.190126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61bb5b29-a6a0-4c88-9d42-048e83aec447 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:01.190548 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61bb5b29-a6a0-4c88-9d42-048e83aec447 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:01.221406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-61bb5b29-a6a0-4c88-9d42-048e83aec447 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046968 Aug 28 01:12:01.221958 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 756/1282] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:01 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:01.461675 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c242098f-8ede-42a6-8bb7-d90639a5142f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:01.463928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c242098f-8ede-42a6-8bb7-d90639a5142f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:01.471324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c242098f-8ede-42a6-8bb7-d90639a5142f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:01.471794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c242098f-8ede-42a6-8bb7-d90639a5142f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:01.472266 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c242098f-8ede-42a6-8bb7-d90639a5142f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:01.472943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c242098f-8ede-42a6-8bb7-d90639a5142f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:01.473414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c242098f-8ede-42a6-8bb7-d90639a5142f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:01.473843 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c242098f-8ede-42a6-8bb7-d90639a5142f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:01.510145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c242098f-8ede-42a6-8bb7-d90639a5142f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052840 Aug 28 01:12:01.510737 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 527/1283] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:01 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:02.237868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-be00c5e0-d2b8-45e6-af04-b9d2e6de8fb5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:02.240012 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-be00c5e0-d2b8-45e6-af04-b9d2e6de8fb5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:02.247468 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-be00c5e0-d2b8-45e6-af04-b9d2e6de8fb5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:02.247944 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-be00c5e0-d2b8-45e6-af04-b9d2e6de8fb5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:02.248411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-be00c5e0-d2b8-45e6-af04-b9d2e6de8fb5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:02.249119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-be00c5e0-d2b8-45e6-af04-b9d2e6de8fb5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:02.249566 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-be00c5e0-d2b8-45e6-af04-b9d2e6de8fb5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:02.249982 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-be00c5e0-d2b8-45e6-af04-b9d2e6de8fb5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:02.281389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-be00c5e0-d2b8-45e6-af04-b9d2e6de8fb5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047500 Aug 28 01:12:02.282014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 757/1284] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:02 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:02.525016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-086cc8a6-938d-4200-9c8b-73792948e6d2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:02.527108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-086cc8a6-938d-4200-9c8b-73792948e6d2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:02.534142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-086cc8a6-938d-4200-9c8b-73792948e6d2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:02.534634 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-086cc8a6-938d-4200-9c8b-73792948e6d2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:02.535071 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-086cc8a6-938d-4200-9c8b-73792948e6d2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:02.535764 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-086cc8a6-938d-4200-9c8b-73792948e6d2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:02.536223 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-086cc8a6-938d-4200-9c8b-73792948e6d2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:02.536631 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-086cc8a6-938d-4200-9c8b-73792948e6d2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:02.568888 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-086cc8a6-938d-4200-9c8b-73792948e6d2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047960 Aug 28 01:12:02.569889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 528/1285] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:02 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:03.295599 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-671163be-1ec2-425c-89ee-b626d2c15c63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:03.298209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-671163be-1ec2-425c-89ee-b626d2c15c63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:03.307834 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-671163be-1ec2-425c-89ee-b626d2c15c63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:03.308455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-671163be-1ec2-425c-89ee-b626d2c15c63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:03.308994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-671163be-1ec2-425c-89ee-b626d2c15c63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:03.310116 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-671163be-1ec2-425c-89ee-b626d2c15c63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:03.310872 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-671163be-1ec2-425c-89ee-b626d2c15c63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:03.311700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-671163be-1ec2-425c-89ee-b626d2c15c63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:03.346998 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-671163be-1ec2-425c-89ee-b626d2c15c63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056271 Aug 28 01:12:03.347559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 758/1286] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:03 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:03.584653 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2e31f6d6-2e09-4174-8ac9-9f84a9da4565 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:03.587327 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2e31f6d6-2e09-4174-8ac9-9f84a9da4565 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:03.594875 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e31f6d6-2e09-4174-8ac9-9f84a9da4565 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:03.595358 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e31f6d6-2e09-4174-8ac9-9f84a9da4565 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:03.595781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e31f6d6-2e09-4174-8ac9-9f84a9da4565 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:03.596472 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e31f6d6-2e09-4174-8ac9-9f84a9da4565 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:03.596916 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e31f6d6-2e09-4174-8ac9-9f84a9da4565 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:03.597383 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e31f6d6-2e09-4174-8ac9-9f84a9da4565 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:03.628473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2e31f6d6-2e09-4174-8ac9-9f84a9da4565 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049499 Aug 28 01:12:03.629312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 529/1287] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:03 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:12:04.360318 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e75759ee-2c2a-4bea-950c-3d10de3f9d5b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:04.362631 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e75759ee-2c2a-4bea-950c-3d10de3f9d5b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:04.369555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e75759ee-2c2a-4bea-950c-3d10de3f9d5b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:04.370077 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e75759ee-2c2a-4bea-950c-3d10de3f9d5b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:04.370513 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e75759ee-2c2a-4bea-950c-3d10de3f9d5b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:04.371216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e75759ee-2c2a-4bea-950c-3d10de3f9d5b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:04.371660 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e75759ee-2c2a-4bea-950c-3d10de3f9d5b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:04.372060 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e75759ee-2c2a-4bea-950c-3d10de3f9d5b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:04.402933 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e75759ee-2c2a-4bea-950c-3d10de3f9d5b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046991 Aug 28 01:12:04.403546 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 759/1288] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:04 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:04.645506 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-82d38754-a36a-453f-8e34-ea5fee3b3752 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:04.648305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-82d38754-a36a-453f-8e34-ea5fee3b3752 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:04.657611 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82d38754-a36a-453f-8e34-ea5fee3b3752 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:04.658308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82d38754-a36a-453f-8e34-ea5fee3b3752 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:04.658848 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82d38754-a36a-453f-8e34-ea5fee3b3752 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:04.659727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82d38754-a36a-453f-8e34-ea5fee3b3752 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:04.660265 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82d38754-a36a-453f-8e34-ea5fee3b3752 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:04.660831 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82d38754-a36a-453f-8e34-ea5fee3b3752 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:04.694567 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-82d38754-a36a-453f-8e34-ea5fee3b3752 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055392 Aug 28 01:12:04.695116 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 530/1289] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:04 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:05.416143 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1f3f84dd-c5a2-4f6b-9111-87a05f5d5563 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:05.418706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1f3f84dd-c5a2-4f6b-9111-87a05f5d5563 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:05.425956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1f3f84dd-c5a2-4f6b-9111-87a05f5d5563 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:05.426449 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1f3f84dd-c5a2-4f6b-9111-87a05f5d5563 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:05.426884 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1f3f84dd-c5a2-4f6b-9111-87a05f5d5563 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:05.427610 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1f3f84dd-c5a2-4f6b-9111-87a05f5d5563 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:05.428048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1f3f84dd-c5a2-4f6b-9111-87a05f5d5563 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:05.428462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1f3f84dd-c5a2-4f6b-9111-87a05f5d5563 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:05.465639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1f3f84dd-c5a2-4f6b-9111-87a05f5d5563 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053890 Aug 28 01:12:05.466242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 760/1290] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:05 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:05.707780 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4892dfd0-a5a8-4f3f-a4ef-84d7fab32659 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:05.709933 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4892dfd0-a5a8-4f3f-a4ef-84d7fab32659 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:05.716782 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4892dfd0-a5a8-4f3f-a4ef-84d7fab32659 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:05.717306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4892dfd0-a5a8-4f3f-a4ef-84d7fab32659 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:05.717741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4892dfd0-a5a8-4f3f-a4ef-84d7fab32659 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:05.718428 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4892dfd0-a5a8-4f3f-a4ef-84d7fab32659 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:05.718864 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4892dfd0-a5a8-4f3f-a4ef-84d7fab32659 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:05.719270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4892dfd0-a5a8-4f3f-a4ef-84d7fab32659 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:05.750509 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4892dfd0-a5a8-4f3f-a4ef-84d7fab32659 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046973 Aug 28 01:12:05.751076 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 531/1291] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:05 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:06.479038 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b9186948-8bd0-4092-863e-3d5bdc39a0ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:06.481304 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b9186948-8bd0-4092-863e-3d5bdc39a0ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:06.488602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9186948-8bd0-4092-863e-3d5bdc39a0ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:06.489122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9186948-8bd0-4092-863e-3d5bdc39a0ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:06.489565 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9186948-8bd0-4092-863e-3d5bdc39a0ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:06.490260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9186948-8bd0-4092-863e-3d5bdc39a0ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:06.490703 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9186948-8bd0-4092-863e-3d5bdc39a0ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:06.491121 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9186948-8bd0-4092-863e-3d5bdc39a0ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:06.523663 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b9186948-8bd0-4092-863e-3d5bdc39a0ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048877 Aug 28 01:12:06.524268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 761/1292] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:06 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:06.766159 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-672bb474-0229-460b-bece-8fe71bb7c7bc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:06.768553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-672bb474-0229-460b-bece-8fe71bb7c7bc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:06.775297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-672bb474-0229-460b-bece-8fe71bb7c7bc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:06.775807 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-672bb474-0229-460b-bece-8fe71bb7c7bc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:06.776246 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-672bb474-0229-460b-bece-8fe71bb7c7bc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:06.776949 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-672bb474-0229-460b-bece-8fe71bb7c7bc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:06.777501 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-672bb474-0229-460b-bece-8fe71bb7c7bc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:06.777923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-672bb474-0229-460b-bece-8fe71bb7c7bc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:06.809928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-672bb474-0229-460b-bece-8fe71bb7c7bc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049750 Aug 28 01:12:06.810484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 532/1293] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:06 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:07.536136 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-33f06d78-66ef-40be-9ed8-e198cd33b308 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:07.538395 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-33f06d78-66ef-40be-9ed8-e198cd33b308 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:07.545164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-33f06d78-66ef-40be-9ed8-e198cd33b308 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:07.545641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-33f06d78-66ef-40be-9ed8-e198cd33b308 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:07.546050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-33f06d78-66ef-40be-9ed8-e198cd33b308 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:07.546734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-33f06d78-66ef-40be-9ed8-e198cd33b308 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:07.547175 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-33f06d78-66ef-40be-9ed8-e198cd33b308 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:07.547584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-33f06d78-66ef-40be-9ed8-e198cd33b308 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:07.579250 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-33f06d78-66ef-40be-9ed8-e198cd33b308 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047064 Aug 28 01:12:07.579803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 762/1294] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:07 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:07.823743 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6b4d790f-dc32-4781-8e34-a7c6cc42e24b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:07.826126 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6b4d790f-dc32-4781-8e34-a7c6cc42e24b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:07.834409 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b4d790f-dc32-4781-8e34-a7c6cc42e24b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:07.834912 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b4d790f-dc32-4781-8e34-a7c6cc42e24b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:07.835408 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b4d790f-dc32-4781-8e34-a7c6cc42e24b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:07.836144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b4d790f-dc32-4781-8e34-a7c6cc42e24b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:07.836726 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b4d790f-dc32-4781-8e34-a7c6cc42e24b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:07.837012 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b4d790f-dc32-4781-8e34-a7c6cc42e24b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:07.872684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6b4d790f-dc32-4781-8e34-a7c6cc42e24b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053629 Aug 28 01:12:07.873323 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 533/1295] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:07 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:08.592151 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e6c770ff-2d5f-44f5-a8ba-36ac24bf7e60 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:08.594331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e6c770ff-2d5f-44f5-a8ba-36ac24bf7e60 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:08.601124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6c770ff-2d5f-44f5-a8ba-36ac24bf7e60 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:08.601678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6c770ff-2d5f-44f5-a8ba-36ac24bf7e60 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:08.602135 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6c770ff-2d5f-44f5-a8ba-36ac24bf7e60 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:08.602887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6c770ff-2d5f-44f5-a8ba-36ac24bf7e60 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:08.603339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6c770ff-2d5f-44f5-a8ba-36ac24bf7e60 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:08.603756 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6c770ff-2d5f-44f5-a8ba-36ac24bf7e60 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:08.640661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e6c770ff-2d5f-44f5-a8ba-36ac24bf7e60 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052490 Aug 28 01:12:08.641307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 763/1296] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:08 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:08.886292 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0573de15-a170-48bc-b9ea-d673fbfd4713 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:08.888575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0573de15-a170-48bc-b9ea-d673fbfd4713 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:08.895764 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0573de15-a170-48bc-b9ea-d673fbfd4713 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:08.896264 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0573de15-a170-48bc-b9ea-d673fbfd4713 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:08.896695 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0573de15-a170-48bc-b9ea-d673fbfd4713 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:08.897439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0573de15-a170-48bc-b9ea-d673fbfd4713 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:08.897886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0573de15-a170-48bc-b9ea-d673fbfd4713 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:08.898311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0573de15-a170-48bc-b9ea-d673fbfd4713 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:08.931033 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0573de15-a170-48bc-b9ea-d673fbfd4713 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049020 Aug 28 01:12:08.931614 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 534/1297] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:08 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:09.653538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d4b1f0ee-0d2d-4c7b-9f7f-dcce56529af7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:09.655719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d4b1f0ee-0d2d-4c7b-9f7f-dcce56529af7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:09.662682 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4b1f0ee-0d2d-4c7b-9f7f-dcce56529af7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:09.663156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4b1f0ee-0d2d-4c7b-9f7f-dcce56529af7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:09.663587 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4b1f0ee-0d2d-4c7b-9f7f-dcce56529af7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:09.664269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4b1f0ee-0d2d-4c7b-9f7f-dcce56529af7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:09.664712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4b1f0ee-0d2d-4c7b-9f7f-dcce56529af7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:09.665164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4b1f0ee-0d2d-4c7b-9f7f-dcce56529af7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:09.696161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d4b1f0ee-0d2d-4c7b-9f7f-dcce56529af7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046565 Aug 28 01:12:09.696710 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 764/1298] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:09 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:09.944007 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-34fd847d-c0be-4466-a1a6-bea776d2008f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:09.946562 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-34fd847d-c0be-4466-a1a6-bea776d2008f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:09.953237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-34fd847d-c0be-4466-a1a6-bea776d2008f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:09.953728 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-34fd847d-c0be-4466-a1a6-bea776d2008f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:09.954131 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-34fd847d-c0be-4466-a1a6-bea776d2008f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:09.954817 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-34fd847d-c0be-4466-a1a6-bea776d2008f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:09.955295 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-34fd847d-c0be-4466-a1a6-bea776d2008f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:09.955719 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-34fd847d-c0be-4466-a1a6-bea776d2008f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:09.986516 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-34fd847d-c0be-4466-a1a6-bea776d2008f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046535 Aug 28 01:12:09.987078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 535/1299] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:09 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:10.709140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d799c4e1-f2ac-45f8-ba7e-2f9af031527e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:10.711304 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d799c4e1-f2ac-45f8-ba7e-2f9af031527e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:10.718833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d799c4e1-f2ac-45f8-ba7e-2f9af031527e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:10.719402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d799c4e1-f2ac-45f8-ba7e-2f9af031527e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:10.719920 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d799c4e1-f2ac-45f8-ba7e-2f9af031527e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:10.720755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d799c4e1-f2ac-45f8-ba7e-2f9af031527e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:10.721368 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d799c4e1-f2ac-45f8-ba7e-2f9af031527e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:10.721908 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d799c4e1-f2ac-45f8-ba7e-2f9af031527e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:10.754001 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d799c4e1-f2ac-45f8-ba7e-2f9af031527e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048899 Aug 28 01:12:10.754572 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 765/1300] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:10 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:10.999361 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f33465b0-d584-49f6-a3ad-2bc04a617a57 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:11.001580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f33465b0-d584-49f6-a3ad-2bc04a617a57 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:11.008512 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f33465b0-d584-49f6-a3ad-2bc04a617a57 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:11.008993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f33465b0-d584-49f6-a3ad-2bc04a617a57 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:11.009441 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f33465b0-d584-49f6-a3ad-2bc04a617a57 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:11.010126 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f33465b0-d584-49f6-a3ad-2bc04a617a57 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:11.010566 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f33465b0-d584-49f6-a3ad-2bc04a617a57 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:11.010981 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f33465b0-d584-49f6-a3ad-2bc04a617a57 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:11.041385 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f33465b0-d584-49f6-a3ad-2bc04a617a57 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045892 Aug 28 01:12:11.042234 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 536/1301] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:10 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:11.770624 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-632fc865-798f-43ee-82a4-53bbe2447972 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:11.772803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-632fc865-798f-43ee-82a4-53bbe2447972 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:11.780198 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-632fc865-798f-43ee-82a4-53bbe2447972 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:11.780694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-632fc865-798f-43ee-82a4-53bbe2447972 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:11.781165 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-632fc865-798f-43ee-82a4-53bbe2447972 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:11.781883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-632fc865-798f-43ee-82a4-53bbe2447972 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:11.782469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-632fc865-798f-43ee-82a4-53bbe2447972 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:11.782889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-632fc865-798f-43ee-82a4-53bbe2447972 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:11.814649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-632fc865-798f-43ee-82a4-53bbe2447972 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048331 Aug 28 01:12:11.815203 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 766/1302] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:11 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:12.053623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f0ff6b95-d5ed-4672-a9a9-2e56eaddc457 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:12.055785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f0ff6b95-d5ed-4672-a9a9-2e56eaddc457 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:12.062507 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0ff6b95-d5ed-4672-a9a9-2e56eaddc457 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:12.062991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0ff6b95-d5ed-4672-a9a9-2e56eaddc457 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:12.063432 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0ff6b95-d5ed-4672-a9a9-2e56eaddc457 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:12.064161 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0ff6b95-d5ed-4672-a9a9-2e56eaddc457 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:12.064613 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0ff6b95-d5ed-4672-a9a9-2e56eaddc457 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:12.065057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0ff6b95-d5ed-4672-a9a9-2e56eaddc457 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:12.094945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f0ff6b95-d5ed-4672-a9a9-2e56eaddc457 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045194 Aug 28 01:12:12.095510 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 537/1303] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:12 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:12.832629 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6eb7fc89-d2c5-475e-a049-9885d0f63c2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:12.836229 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6eb7fc89-d2c5-475e-a049-9885d0f63c2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:12.847078 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6eb7fc89-d2c5-475e-a049-9885d0f63c2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:12.847562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6eb7fc89-d2c5-475e-a049-9885d0f63c2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:12.847993 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6eb7fc89-d2c5-475e-a049-9885d0f63c2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:12.848698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6eb7fc89-d2c5-475e-a049-9885d0f63c2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:12.849191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6eb7fc89-d2c5-475e-a049-9885d0f63c2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:12.849619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6eb7fc89-d2c5-475e-a049-9885d0f63c2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:12.880010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6eb7fc89-d2c5-475e-a049-9885d0f63c2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054698 Aug 28 01:12:12.880558 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 767/1304] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:12 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:13.107799 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e496b151-870b-45d0-9e37-25dd69145ddb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:13.109999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e496b151-870b-45d0-9e37-25dd69145ddb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:13.117235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e496b151-870b-45d0-9e37-25dd69145ddb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:13.117730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e496b151-870b-45d0-9e37-25dd69145ddb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:13.118162 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e496b151-870b-45d0-9e37-25dd69145ddb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:13.118865 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e496b151-870b-45d0-9e37-25dd69145ddb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:13.119311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e496b151-870b-45d0-9e37-25dd69145ddb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:13.119696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e496b151-870b-45d0-9e37-25dd69145ddb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:13.151175 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e496b151-870b-45d0-9e37-25dd69145ddb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047255 Aug 28 01:12:13.151695 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 538/1305] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:13 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:13.895207 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2d03ecda-e02d-4459-b916-31eb4e497583 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:13.897543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2d03ecda-e02d-4459-b916-31eb4e497583 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:13.904521 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d03ecda-e02d-4459-b916-31eb4e497583 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:13.905029 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d03ecda-e02d-4459-b916-31eb4e497583 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:13.905474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d03ecda-e02d-4459-b916-31eb4e497583 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:13.906173 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d03ecda-e02d-4459-b916-31eb4e497583 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:13.906625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d03ecda-e02d-4459-b916-31eb4e497583 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:13.907056 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d03ecda-e02d-4459-b916-31eb4e497583 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:13.933065 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:12:13.937970 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2d03ecda-e02d-4459-b916-31eb4e497583 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048094 Aug 28 01:12:13.938553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 768/1306] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:13 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:14.164670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-08459c88-43d3-45d1-ad6b-2b83433780ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:14.167055 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-08459c88-43d3-45d1-ad6b-2b83433780ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:14.174448 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08459c88-43d3-45d1-ad6b-2b83433780ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:14.175194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08459c88-43d3-45d1-ad6b-2b83433780ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:14.175790 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08459c88-43d3-45d1-ad6b-2b83433780ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:14.176526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08459c88-43d3-45d1-ad6b-2b83433780ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:14.176973 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08459c88-43d3-45d1-ad6b-2b83433780ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:14.177397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08459c88-43d3-45d1-ad6b-2b83433780ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:14.209440 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-08459c88-43d3-45d1-ad6b-2b83433780ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049323 Aug 28 01:12:14.210015 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 539/1307] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:14 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:14.950595 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6be82747-22a6-4799-ae9c-3c28e628ece9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:14.952747 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6be82747-22a6-4799-ae9c-3c28e628ece9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:14.955393 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=58 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:12:14.960827 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6be82747-22a6-4799-ae9c-3c28e628ece9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:14.961308 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6be82747-22a6-4799-ae9c-3c28e628ece9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:14.961752 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6be82747-22a6-4799-ae9c-3c28e628ece9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:14.962409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6be82747-22a6-4799-ae9c-3c28e628ece9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:14.962848 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6be82747-22a6-4799-ae9c-3c28e628ece9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:14.963258 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6be82747-22a6-4799-ae9c-3c28e628ece9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:14.994757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6be82747-22a6-4799-ae9c-3c28e628ece9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048170 Aug 28 01:12:14.995345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 769/1308] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:14 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:15.227348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-46fc7049-64e0-49cf-9f2a-ee517423c220 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:15.230171 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-46fc7049-64e0-49cf-9f2a-ee517423c220 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:15.233401 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=58 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:12:15.240466 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-46fc7049-64e0-49cf-9f2a-ee517423c220 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:15.241058 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-46fc7049-64e0-49cf-9f2a-ee517423c220 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:15.241546 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-46fc7049-64e0-49cf-9f2a-ee517423c220 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:15.242390 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-46fc7049-64e0-49cf-9f2a-ee517423c220 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:15.242972 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-46fc7049-64e0-49cf-9f2a-ee517423c220 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:15.243486 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-46fc7049-64e0-49cf-9f2a-ee517423c220 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:15.283806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-46fc7049-64e0-49cf-9f2a-ee517423c220 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061758 Aug 28 01:12:15.284530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 540/1309] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:15 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:16.007529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-af217411-c998-4643-99a2-606d29d10c0b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:16.009927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-af217411-c998-4643-99a2-606d29d10c0b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:16.016927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-af217411-c998-4643-99a2-606d29d10c0b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:16.017471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-af217411-c998-4643-99a2-606d29d10c0b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:16.017913 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-af217411-c998-4643-99a2-606d29d10c0b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:16.018639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-af217411-c998-4643-99a2-606d29d10c0b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:16.019098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-af217411-c998-4643-99a2-606d29d10c0b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:16.019517 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-af217411-c998-4643-99a2-606d29d10c0b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:16.050351 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-af217411-c998-4643-99a2-606d29d10c0b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047134 Aug 28 01:12:16.050942 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 770/1310] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:16 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:16.297604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c1a2de8c-4861-4951-a976-173e6b68f73f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:16.299903 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c1a2de8c-4861-4951-a976-173e6b68f73f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:16.306803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1a2de8c-4861-4951-a976-173e6b68f73f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:16.307298 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1a2de8c-4861-4951-a976-173e6b68f73f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:16.307734 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1a2de8c-4861-4951-a976-173e6b68f73f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:16.308454 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1a2de8c-4861-4951-a976-173e6b68f73f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:16.308894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1a2de8c-4861-4951-a976-173e6b68f73f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:16.309378 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1a2de8c-4861-4951-a976-173e6b68f73f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:16.313377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:12:16.340230 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c1a2de8c-4861-4951-a976-173e6b68f73f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046745 Aug 28 01:12:16.340767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 541/1311] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:16 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:17.064347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bc407ca0-d601-4f72-bc31-773cd98e17f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:17.066761 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bc407ca0-d601-4f72-bc31-773cd98e17f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:17.074178 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc407ca0-d601-4f72-bc31-773cd98e17f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:17.074670 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc407ca0-d601-4f72-bc31-773cd98e17f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:17.075103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc407ca0-d601-4f72-bc31-773cd98e17f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:17.075851 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc407ca0-d601-4f72-bc31-773cd98e17f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:17.076315 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc407ca0-d601-4f72-bc31-773cd98e17f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:17.076734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc407ca0-d601-4f72-bc31-773cd98e17f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:17.107768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bc407ca0-d601-4f72-bc31-773cd98e17f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047774 Aug 28 01:12:17.108340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 771/1312] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:17 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:17.353289 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5716b4bd-120c-495f-a84a-82e173c27e77 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:17.355281 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5716b4bd-120c-495f-a84a-82e173c27e77 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:17.362279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5716b4bd-120c-495f-a84a-82e173c27e77 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:17.362772 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5716b4bd-120c-495f-a84a-82e173c27e77 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:17.363192 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5716b4bd-120c-495f-a84a-82e173c27e77 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:17.363922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5716b4bd-120c-495f-a84a-82e173c27e77 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:17.364548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5716b4bd-120c-495f-a84a-82e173c27e77 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:17.365177 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5716b4bd-120c-495f-a84a-82e173c27e77 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:17.397820 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5716b4bd-120c-495f-a84a-82e173c27e77 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048761 Aug 28 01:12:17.398417 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 542/1313] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:17 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:18.121568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-994315a1-5484-4fde-b2ea-c4984ff8fcef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:18.124723 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-994315a1-5484-4fde-b2ea-c4984ff8fcef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:18.135288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-994315a1-5484-4fde-b2ea-c4984ff8fcef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:18.135972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-994315a1-5484-4fde-b2ea-c4984ff8fcef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:18.136650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-994315a1-5484-4fde-b2ea-c4984ff8fcef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:18.137945 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-994315a1-5484-4fde-b2ea-c4984ff8fcef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:18.138622 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-994315a1-5484-4fde-b2ea-c4984ff8fcef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:18.139128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-994315a1-5484-4fde-b2ea-c4984ff8fcef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:18.180050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-994315a1-5484-4fde-b2ea-c4984ff8fcef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063695 Aug 28 01:12:18.181022 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 772/1314] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:18 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:18.410282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5e581d53-25a2-4159-8f3e-fe83dd9f1094 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:18.412588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5e581d53-25a2-4159-8f3e-fe83dd9f1094 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:18.419751 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e581d53-25a2-4159-8f3e-fe83dd9f1094 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:18.420249 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e581d53-25a2-4159-8f3e-fe83dd9f1094 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:18.420691 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e581d53-25a2-4159-8f3e-fe83dd9f1094 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:18.421440 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e581d53-25a2-4159-8f3e-fe83dd9f1094 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:18.421883 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e581d53-25a2-4159-8f3e-fe83dd9f1094 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:18.422321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e581d53-25a2-4159-8f3e-fe83dd9f1094 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:18.453837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5e581d53-25a2-4159-8f3e-fe83dd9f1094 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047497 Aug 28 01:12:18.454398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 543/1315] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:18 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:19.194050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d7db8122-60e4-47bd-bd5f-38ad7ab04a3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:19.196640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d7db8122-60e4-47bd-bd5f-38ad7ab04a3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:19.203370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7db8122-60e4-47bd-bd5f-38ad7ab04a3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:19.203850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7db8122-60e4-47bd-bd5f-38ad7ab04a3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:19.204275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7db8122-60e4-47bd-bd5f-38ad7ab04a3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:19.204954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7db8122-60e4-47bd-bd5f-38ad7ab04a3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:19.205423 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7db8122-60e4-47bd-bd5f-38ad7ab04a3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:19.205832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7db8122-60e4-47bd-bd5f-38ad7ab04a3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:19.237249 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d7db8122-60e4-47bd-bd5f-38ad7ab04a3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047333 Aug 28 01:12:19.237824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 773/1316] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:19 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:19.465776 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d6671ff8-4d42-447a-a11e-b50fb22e5045 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:19.467986 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d6671ff8-4d42-447a-a11e-b50fb22e5045 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:19.474247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6671ff8-4d42-447a-a11e-b50fb22e5045 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:19.474733 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6671ff8-4d42-447a-a11e-b50fb22e5045 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:19.475163 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6671ff8-4d42-447a-a11e-b50fb22e5045 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:19.475860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6671ff8-4d42-447a-a11e-b50fb22e5045 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:19.476316 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6671ff8-4d42-447a-a11e-b50fb22e5045 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:19.476737 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6671ff8-4d42-447a-a11e-b50fb22e5045 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:19.508549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d6671ff8-4d42-447a-a11e-b50fb22e5045 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046733 Aug 28 01:12:19.509143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 544/1317] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:19 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:20.250262 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e747743b-490b-4046-bcfc-7554375cab7d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:20.252459 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e747743b-490b-4046-bcfc-7554375cab7d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:20.259272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e747743b-490b-4046-bcfc-7554375cab7d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:20.259755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e747743b-490b-4046-bcfc-7554375cab7d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:20.260187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e747743b-490b-4046-bcfc-7554375cab7d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:20.260986 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e747743b-490b-4046-bcfc-7554375cab7d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:20.261467 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e747743b-490b-4046-bcfc-7554375cab7d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:20.261885 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e747743b-490b-4046-bcfc-7554375cab7d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:20.293483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e747743b-490b-4046-bcfc-7554375cab7d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047119 Aug 28 01:12:20.294089 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 774/1318] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:20 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:20.521791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e7cefeb9-fcc1-48c9-b1ae-d02989efa332 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:20.524103 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e7cefeb9-fcc1-48c9-b1ae-d02989efa332 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:20.531463 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e7cefeb9-fcc1-48c9-b1ae-d02989efa332 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:20.531987 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e7cefeb9-fcc1-48c9-b1ae-d02989efa332 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:20.532424 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e7cefeb9-fcc1-48c9-b1ae-d02989efa332 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:20.533150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e7cefeb9-fcc1-48c9-b1ae-d02989efa332 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:20.533599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e7cefeb9-fcc1-48c9-b1ae-d02989efa332 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:20.534014 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e7cefeb9-fcc1-48c9-b1ae-d02989efa332 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:20.565944 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e7cefeb9-fcc1-48c9-b1ae-d02989efa332 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048464 Aug 28 01:12:20.566573 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 545/1319] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:20 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:21.307883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9812490b-86a9-4d91-acf4-0263e6eecd49 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:21.310460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9812490b-86a9-4d91-acf4-0263e6eecd49 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:21.321503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9812490b-86a9-4d91-acf4-0263e6eecd49 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:21.322002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9812490b-86a9-4d91-acf4-0263e6eecd49 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:21.322434 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9812490b-86a9-4d91-acf4-0263e6eecd49 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:21.323181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9812490b-86a9-4d91-acf4-0263e6eecd49 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:21.323614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9812490b-86a9-4d91-acf4-0263e6eecd49 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:21.324028 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9812490b-86a9-4d91-acf4-0263e6eecd49 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:21.354988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9812490b-86a9-4d91-acf4-0263e6eecd49 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052065 Aug 28 01:12:21.355576 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 775/1320] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:21 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:21.580006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-23c20ad7-d0c7-441c-a870-20610604a18b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:21.582092 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-23c20ad7-d0c7-441c-a870-20610604a18b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:21.596767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23c20ad7-d0c7-441c-a870-20610604a18b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:21.597294 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23c20ad7-d0c7-441c-a870-20610604a18b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:21.597731 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23c20ad7-d0c7-441c-a870-20610604a18b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:21.598436 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23c20ad7-d0c7-441c-a870-20610604a18b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:21.598853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23c20ad7-d0c7-441c-a870-20610604a18b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:21.599306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23c20ad7-d0c7-441c-a870-20610604a18b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:21.629769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-23c20ad7-d0c7-441c-a870-20610604a18b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054290 Aug 28 01:12:21.630365 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 546/1321] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:21 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:22.368191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e2e708f0-43dc-470a-8a14-f4c221ea7e88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:22.370245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e2e708f0-43dc-470a-8a14-f4c221ea7e88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:22.376831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2e708f0-43dc-470a-8a14-f4c221ea7e88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:22.377348 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2e708f0-43dc-470a-8a14-f4c221ea7e88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:22.377782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2e708f0-43dc-470a-8a14-f4c221ea7e88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:22.378470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2e708f0-43dc-470a-8a14-f4c221ea7e88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:22.378924 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2e708f0-43dc-470a-8a14-f4c221ea7e88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:22.379346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2e708f0-43dc-470a-8a14-f4c221ea7e88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:22.411039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e2e708f0-43dc-470a-8a14-f4c221ea7e88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046787 Aug 28 01:12:22.411670 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 776/1322] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:22 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:22.643177 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0325f85b-e7d5-4157-89aa-7a28ad2a7ebd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:22.645372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0325f85b-e7d5-4157-89aa-7a28ad2a7ebd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:22.652269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0325f85b-e7d5-4157-89aa-7a28ad2a7ebd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:22.652269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0325f85b-e7d5-4157-89aa-7a28ad2a7ebd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:22.652555 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0325f85b-e7d5-4157-89aa-7a28ad2a7ebd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:22.653278 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0325f85b-e7d5-4157-89aa-7a28ad2a7ebd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:22.653764 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0325f85b-e7d5-4157-89aa-7a28ad2a7ebd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:22.654132 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0325f85b-e7d5-4157-89aa-7a28ad2a7ebd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:22.684763 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0325f85b-e7d5-4157-89aa-7a28ad2a7ebd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045814 Aug 28 01:12:22.685365 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 547/1323] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:22 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:23.424687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1092c557-7c25-43c7-bb38-a191660847d1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:23.427014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1092c557-7c25-43c7-bb38-a191660847d1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:23.435177 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1092c557-7c25-43c7-bb38-a191660847d1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:23.435654 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1092c557-7c25-43c7-bb38-a191660847d1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:23.436083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1092c557-7c25-43c7-bb38-a191660847d1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:23.436768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1092c557-7c25-43c7-bb38-a191660847d1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:23.437268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1092c557-7c25-43c7-bb38-a191660847d1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:23.437682 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1092c557-7c25-43c7-bb38-a191660847d1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:23.468955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1092c557-7c25-43c7-bb38-a191660847d1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048651 Aug 28 01:12:23.469640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 777/1324] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:23 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:23.697730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f42fff0c-4f2a-4270-87c4-cf361eef4e10 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:23.699771 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f42fff0c-4f2a-4270-87c4-cf361eef4e10 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:23.706811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f42fff0c-4f2a-4270-87c4-cf361eef4e10 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:23.707288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f42fff0c-4f2a-4270-87c4-cf361eef4e10 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:23.707717 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f42fff0c-4f2a-4270-87c4-cf361eef4e10 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:23.708395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f42fff0c-4f2a-4270-87c4-cf361eef4e10 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:23.708842 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f42fff0c-4f2a-4270-87c4-cf361eef4e10 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:23.710341 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f42fff0c-4f2a-4270-87c4-cf361eef4e10 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:23.746104 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f42fff0c-4f2a-4270-87c4-cf361eef4e10 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052788 Aug 28 01:12:23.746670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 548/1325] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:23 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:24.481682 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a91e36a5-3514-4456-98b8-af87c7f51493 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:24.484115 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a91e36a5-3514-4456-98b8-af87c7f51493 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:24.491036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a91e36a5-3514-4456-98b8-af87c7f51493 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:24.491523 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a91e36a5-3514-4456-98b8-af87c7f51493 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:24.491955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a91e36a5-3514-4456-98b8-af87c7f51493 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:24.492655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a91e36a5-3514-4456-98b8-af87c7f51493 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:24.493138 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a91e36a5-3514-4456-98b8-af87c7f51493 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:24.493557 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a91e36a5-3514-4456-98b8-af87c7f51493 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:24.525494 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a91e36a5-3514-4456-98b8-af87c7f51493 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048095 Aug 28 01:12:24.526071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 778/1326] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:24 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:24.759233 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ab520ab9-03ed-4dea-9b83-2b140ff655b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:24.761649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ab520ab9-03ed-4dea-9b83-2b140ff655b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:24.769495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab520ab9-03ed-4dea-9b83-2b140ff655b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:24.769977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab520ab9-03ed-4dea-9b83-2b140ff655b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:24.770411 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab520ab9-03ed-4dea-9b83-2b140ff655b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:24.771110 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab520ab9-03ed-4dea-9b83-2b140ff655b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:24.771559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab520ab9-03ed-4dea-9b83-2b140ff655b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:24.771980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab520ab9-03ed-4dea-9b83-2b140ff655b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:24.803936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ab520ab9-03ed-4dea-9b83-2b140ff655b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048793 Aug 28 01:12:24.804781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 549/1327] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:24 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:25.538481 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e1b7d601-2842-49b3-b06b-3d2f2479ed5b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:25.540640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e1b7d601-2842-49b3-b06b-3d2f2479ed5b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:25.547193 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1b7d601-2842-49b3-b06b-3d2f2479ed5b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:25.547670 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1b7d601-2842-49b3-b06b-3d2f2479ed5b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:25.548097 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1b7d601-2842-49b3-b06b-3d2f2479ed5b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:25.548783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1b7d601-2842-49b3-b06b-3d2f2479ed5b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:25.549260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1b7d601-2842-49b3-b06b-3d2f2479ed5b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:25.549674 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1b7d601-2842-49b3-b06b-3d2f2479ed5b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:25.580342 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e1b7d601-2842-49b3-b06b-3d2f2479ed5b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045835 Aug 28 01:12:25.580899 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 779/1328] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:25 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:25.817039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-88956268-904b-430b-96d7-2ec5cf22fb5b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:25.819447 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-88956268-904b-430b-96d7-2ec5cf22fb5b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:25.826567 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88956268-904b-430b-96d7-2ec5cf22fb5b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:25.827052 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88956268-904b-430b-96d7-2ec5cf22fb5b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:25.827482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88956268-904b-430b-96d7-2ec5cf22fb5b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:25.828186 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88956268-904b-430b-96d7-2ec5cf22fb5b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:25.828609 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88956268-904b-430b-96d7-2ec5cf22fb5b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:25.829099 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88956268-904b-430b-96d7-2ec5cf22fb5b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:25.859131 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-88956268-904b-430b-96d7-2ec5cf22fb5b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046372 Aug 28 01:12:25.859680 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 550/1329] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:25 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:26.592310 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f48e3ef5-3776-45b6-8b29-84ad7407dee7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:26.594471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f48e3ef5-3776-45b6-8b29-84ad7407dee7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:26.601318 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f48e3ef5-3776-45b6-8b29-84ad7407dee7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:26.601808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f48e3ef5-3776-45b6-8b29-84ad7407dee7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:26.602237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f48e3ef5-3776-45b6-8b29-84ad7407dee7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:26.602927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f48e3ef5-3776-45b6-8b29-84ad7407dee7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:26.603372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f48e3ef5-3776-45b6-8b29-84ad7407dee7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:26.603790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f48e3ef5-3776-45b6-8b29-84ad7407dee7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:26.634274 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f48e3ef5-3776-45b6-8b29-84ad7407dee7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045974 Aug 28 01:12:26.634823 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 780/1330] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:26 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:26.871985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-07900aec-0c94-4fe2-864e-16a66b7bba92 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:26.874341 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-07900aec-0c94-4fe2-864e-16a66b7bba92 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:26.881545 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07900aec-0c94-4fe2-864e-16a66b7bba92 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:26.882025 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07900aec-0c94-4fe2-864e-16a66b7bba92 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:26.882433 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07900aec-0c94-4fe2-864e-16a66b7bba92 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:26.883130 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07900aec-0c94-4fe2-864e-16a66b7bba92 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:26.883571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07900aec-0c94-4fe2-864e-16a66b7bba92 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:26.883987 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07900aec-0c94-4fe2-864e-16a66b7bba92 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:26.916898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-07900aec-0c94-4fe2-864e-16a66b7bba92 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048920 Aug 28 01:12:26.917764 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 551/1331] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:26 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:27.648595 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bf740fd9-ae7f-4c08-84c0-b11186c52cf7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:27.650794 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bf740fd9-ae7f-4c08-84c0-b11186c52cf7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:27.657814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf740fd9-ae7f-4c08-84c0-b11186c52cf7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:27.658299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf740fd9-ae7f-4c08-84c0-b11186c52cf7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:27.658736 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf740fd9-ae7f-4c08-84c0-b11186c52cf7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:27.659438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf740fd9-ae7f-4c08-84c0-b11186c52cf7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:27.659978 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf740fd9-ae7f-4c08-84c0-b11186c52cf7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:27.660317 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf740fd9-ae7f-4c08-84c0-b11186c52cf7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:27.690151 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bf740fd9-ae7f-4c08-84c0-b11186c52cf7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045650 Aug 28 01:12:27.690696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 781/1332] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:27 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:27.929513 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c725bc2c-03b1-40e7-a7e0-3762724b59ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:27.931613 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c725bc2c-03b1-40e7-a7e0-3762724b59ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:27.938440 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c725bc2c-03b1-40e7-a7e0-3762724b59ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:27.938920 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c725bc2c-03b1-40e7-a7e0-3762724b59ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:27.939329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c725bc2c-03b1-40e7-a7e0-3762724b59ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:27.940011 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c725bc2c-03b1-40e7-a7e0-3762724b59ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:27.940567 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c725bc2c-03b1-40e7-a7e0-3762724b59ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:27.940999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c725bc2c-03b1-40e7-a7e0-3762724b59ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:27.972284 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c725bc2c-03b1-40e7-a7e0-3762724b59ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046847 Aug 28 01:12:27.972839 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 552/1333] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:27 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:28.704645 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0c38e62b-3f92-417c-976c-ede6fd1da8ab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:28.706840 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0c38e62b-3f92-417c-976c-ede6fd1da8ab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:28.713862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c38e62b-3f92-417c-976c-ede6fd1da8ab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:28.714338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c38e62b-3f92-417c-976c-ede6fd1da8ab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:28.714763 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c38e62b-3f92-417c-976c-ede6fd1da8ab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:28.715447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c38e62b-3f92-417c-976c-ede6fd1da8ab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:28.715886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c38e62b-3f92-417c-976c-ede6fd1da8ab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:28.716315 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c38e62b-3f92-417c-976c-ede6fd1da8ab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:28.747104 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0c38e62b-3f92-417c-976c-ede6fd1da8ab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046471 Aug 28 01:12:28.747661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 782/1334] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:28 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:28.984537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ebf85ebf-c6d0-46bb-afa3-398bf48fd4e1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:28.986706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ebf85ebf-c6d0-46bb-afa3-398bf48fd4e1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:28.993696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ebf85ebf-c6d0-46bb-afa3-398bf48fd4e1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:28.994180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ebf85ebf-c6d0-46bb-afa3-398bf48fd4e1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:28.994604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ebf85ebf-c6d0-46bb-afa3-398bf48fd4e1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:28.995286 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ebf85ebf-c6d0-46bb-afa3-398bf48fd4e1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:28.995725 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ebf85ebf-c6d0-46bb-afa3-398bf48fd4e1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:28.996136 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ebf85ebf-c6d0-46bb-afa3-398bf48fd4e1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:29.028852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ebf85ebf-c6d0-46bb-afa3-398bf48fd4e1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048300 Aug 28 01:12:29.029460 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 553/1335] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:28 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:29.763382 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-697c9d77-db67-4dfa-a13d-a4d30715c991 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:29.766275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-697c9d77-db67-4dfa-a13d-a4d30715c991 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:29.774280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-697c9d77-db67-4dfa-a13d-a4d30715c991 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:29.774761 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-697c9d77-db67-4dfa-a13d-a4d30715c991 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:29.775219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-697c9d77-db67-4dfa-a13d-a4d30715c991 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:29.775911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-697c9d77-db67-4dfa-a13d-a4d30715c991 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:29.776359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-697c9d77-db67-4dfa-a13d-a4d30715c991 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:29.776779 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-697c9d77-db67-4dfa-a13d-a4d30715c991 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:29.807299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-697c9d77-db67-4dfa-a13d-a4d30715c991 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050310 Aug 28 01:12:29.807865 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 783/1336] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:29 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:30.042408 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-add2956f-f1f5-404a-b52d-96475390d7cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:30.044459 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-add2956f-f1f5-404a-b52d-96475390d7cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:30.051508 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-add2956f-f1f5-404a-b52d-96475390d7cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:30.051983 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-add2956f-f1f5-404a-b52d-96475390d7cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:30.052410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-add2956f-f1f5-404a-b52d-96475390d7cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:30.053162 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-add2956f-f1f5-404a-b52d-96475390d7cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:30.053609 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-add2956f-f1f5-404a-b52d-96475390d7cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:30.054027 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-add2956f-f1f5-404a-b52d-96475390d7cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:30.083816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-add2956f-f1f5-404a-b52d-96475390d7cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045493 Aug 28 01:12:30.084607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 554/1337] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:30 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:30.822891 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0ecdb23f-9169-4707-b7d7-2097ea5346d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:30.825092 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0ecdb23f-9169-4707-b7d7-2097ea5346d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:30.834668 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ecdb23f-9169-4707-b7d7-2097ea5346d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:30.835312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ecdb23f-9169-4707-b7d7-2097ea5346d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:30.835949 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ecdb23f-9169-4707-b7d7-2097ea5346d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:30.836930 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ecdb23f-9169-4707-b7d7-2097ea5346d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:30.837619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ecdb23f-9169-4707-b7d7-2097ea5346d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:30.838158 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ecdb23f-9169-4707-b7d7-2097ea5346d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:30.870400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0ecdb23f-9169-4707-b7d7-2097ea5346d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052986 Aug 28 01:12:30.871019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 784/1338] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:30 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:31.096783 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-88f2c2b0-9dcf-474a-b24b-4f4acb76a9c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:31.099039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-88f2c2b0-9dcf-474a-b24b-4f4acb76a9c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:31.106518 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88f2c2b0-9dcf-474a-b24b-4f4acb76a9c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:31.107019 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88f2c2b0-9dcf-474a-b24b-4f4acb76a9c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:31.107461 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88f2c2b0-9dcf-474a-b24b-4f4acb76a9c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:31.108178 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88f2c2b0-9dcf-474a-b24b-4f4acb76a9c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:31.108626 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88f2c2b0-9dcf-474a-b24b-4f4acb76a9c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:31.109107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-88f2c2b0-9dcf-474a-b24b-4f4acb76a9c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:31.146585 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-88f2c2b0-9dcf-474a-b24b-4f4acb76a9c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053871 Aug 28 01:12:31.147417 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 555/1339] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:31 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:31.886339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ce1dcf3c-1e50-4542-981c-793ff7520de3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:31.888889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ce1dcf3c-1e50-4542-981c-793ff7520de3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:31.895893 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce1dcf3c-1e50-4542-981c-793ff7520de3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:31.896393 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce1dcf3c-1e50-4542-981c-793ff7520de3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:31.896833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce1dcf3c-1e50-4542-981c-793ff7520de3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:31.897573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce1dcf3c-1e50-4542-981c-793ff7520de3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:31.897996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce1dcf3c-1e50-4542-981c-793ff7520de3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:31.898440 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce1dcf3c-1e50-4542-981c-793ff7520de3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:31.916258 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:12:31.930484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ce1dcf3c-1e50-4542-981c-793ff7520de3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050516 Aug 28 01:12:31.931080 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 785/1340] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:31 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:32.161758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-eb040e38-db68-44d7-b0e1-67a1e1ff120d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:32.163871 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-eb040e38-db68-44d7-b0e1-67a1e1ff120d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:32.170449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb040e38-db68-44d7-b0e1-67a1e1ff120d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:32.170932 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb040e38-db68-44d7-b0e1-67a1e1ff120d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:32.171358 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb040e38-db68-44d7-b0e1-67a1e1ff120d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:32.172051 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb040e38-db68-44d7-b0e1-67a1e1ff120d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:32.172496 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb040e38-db68-44d7-b0e1-67a1e1ff120d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:32.172913 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb040e38-db68-44d7-b0e1-67a1e1ff120d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:32.203293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-eb040e38-db68-44d7-b0e1-67a1e1ff120d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045829 Aug 28 01:12:32.203862 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 556/1341] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:32 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:32.945707 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cc4d915d-7b8d-439a-962c-e8d0f68762c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:32.948271 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cc4d915d-7b8d-439a-962c-e8d0f68762c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:32.956337 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cc4d915d-7b8d-439a-962c-e8d0f68762c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:32.956811 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cc4d915d-7b8d-439a-962c-e8d0f68762c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:32.957283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cc4d915d-7b8d-439a-962c-e8d0f68762c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:32.957981 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cc4d915d-7b8d-439a-962c-e8d0f68762c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:32.958405 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cc4d915d-7b8d-439a-962c-e8d0f68762c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:32.958855 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cc4d915d-7b8d-439a-962c-e8d0f68762c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:32.991855 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cc4d915d-7b8d-439a-962c-e8d0f68762c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051503 Aug 28 01:12:32.992390 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 786/1342] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:32 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:33.216161 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2307f7c2-bf28-4432-89e2-6fb85591e5a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:33.218297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2307f7c2-bf28-4432-89e2-6fb85591e5a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:33.225247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2307f7c2-bf28-4432-89e2-6fb85591e5a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:33.225709 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2307f7c2-bf28-4432-89e2-6fb85591e5a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:33.226131 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2307f7c2-bf28-4432-89e2-6fb85591e5a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:33.226852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2307f7c2-bf28-4432-89e2-6fb85591e5a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:33.227286 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2307f7c2-bf28-4432-89e2-6fb85591e5a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:33.227697 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2307f7c2-bf28-4432-89e2-6fb85591e5a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:33.256381 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:12:33.261562 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2307f7c2-bf28-4432-89e2-6fb85591e5a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049515 Aug 28 01:12:33.262099 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 557/1343] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:33 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:34.008860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-383dd54d-b99f-44ad-ae38-b5098241563c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:34.011092 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-383dd54d-b99f-44ad-ae38-b5098241563c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:34.019366 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-383dd54d-b99f-44ad-ae38-b5098241563c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:34.019850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-383dd54d-b99f-44ad-ae38-b5098241563c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:34.020289 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-383dd54d-b99f-44ad-ae38-b5098241563c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:34.021162 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-383dd54d-b99f-44ad-ae38-b5098241563c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:34.021674 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-383dd54d-b99f-44ad-ae38-b5098241563c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:34.022084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-383dd54d-b99f-44ad-ae38-b5098241563c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:34.059184 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-383dd54d-b99f-44ad-ae38-b5098241563c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056375 Aug 28 01:12:34.059739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 787/1344] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:34 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:34.274664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cd03f96f-7e80-442d-9ec4-54f694be34ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:34.276800 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cd03f96f-7e80-442d-9ec4-54f694be34ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:34.283619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cd03f96f-7e80-442d-9ec4-54f694be34ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:34.284095 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cd03f96f-7e80-442d-9ec4-54f694be34ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:34.284534 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cd03f96f-7e80-442d-9ec4-54f694be34ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:34.285262 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cd03f96f-7e80-442d-9ec4-54f694be34ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:34.285711 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cd03f96f-7e80-442d-9ec4-54f694be34ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:34.286129 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cd03f96f-7e80-442d-9ec4-54f694be34ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:34.317565 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cd03f96f-7e80-442d-9ec4-54f694be34ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047213 Aug 28 01:12:34.318120 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 558/1345] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:34 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:35.072482 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a3ae4658-bcb8-4d3d-b868-872907406a9d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:35.074604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a3ae4658-bcb8-4d3d-b868-872907406a9d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:35.081591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a3ae4658-bcb8-4d3d-b868-872907406a9d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:35.082075 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a3ae4658-bcb8-4d3d-b868-872907406a9d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:35.082514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a3ae4658-bcb8-4d3d-b868-872907406a9d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:35.083224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a3ae4658-bcb8-4d3d-b868-872907406a9d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:35.083673 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a3ae4658-bcb8-4d3d-b868-872907406a9d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:35.084088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a3ae4658-bcb8-4d3d-b868-872907406a9d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:35.114221 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a3ae4658-bcb8-4d3d-b868-872907406a9d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046176 Aug 28 01:12:35.114768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 788/1346] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:35 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:35.335429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-304d2fd8-6005-4e53-9c6c-2768bfb6bfb3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:35.338201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-304d2fd8-6005-4e53-9c6c-2768bfb6bfb3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:35.347423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-304d2fd8-6005-4e53-9c6c-2768bfb6bfb3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:35.347926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-304d2fd8-6005-4e53-9c6c-2768bfb6bfb3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:35.348357 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-304d2fd8-6005-4e53-9c6c-2768bfb6bfb3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:35.349094 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-304d2fd8-6005-4e53-9c6c-2768bfb6bfb3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:35.349543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-304d2fd8-6005-4e53-9c6c-2768bfb6bfb3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:35.349954 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-304d2fd8-6005-4e53-9c6c-2768bfb6bfb3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:35.380902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-304d2fd8-6005-4e53-9c6c-2768bfb6bfb3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051521 Aug 28 01:12:35.381493 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 559/1347] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:35 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:36.126791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c48dae37-d483-47c2-b9ad-dd71573cb25d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:36.129134 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c48dae37-d483-47c2-b9ad-dd71573cb25d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:36.135955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c48dae37-d483-47c2-b9ad-dd71573cb25d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:36.136449 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c48dae37-d483-47c2-b9ad-dd71573cb25d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:36.136877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c48dae37-d483-47c2-b9ad-dd71573cb25d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:36.137604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c48dae37-d483-47c2-b9ad-dd71573cb25d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:36.138053 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c48dae37-d483-47c2-b9ad-dd71573cb25d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:36.138472 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c48dae37-d483-47c2-b9ad-dd71573cb25d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:36.168147 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c48dae37-d483-47c2-b9ad-dd71573cb25d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045338 Aug 28 01:12:36.168732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 789/1348] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:36 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:36.394394 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-68593d74-26a9-48da-aacf-0f1c481c05a7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:36.396602 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-68593d74-26a9-48da-aacf-0f1c481c05a7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:36.403399 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-68593d74-26a9-48da-aacf-0f1c481c05a7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:36.403877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-68593d74-26a9-48da-aacf-0f1c481c05a7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:36.404305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-68593d74-26a9-48da-aacf-0f1c481c05a7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:36.404997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-68593d74-26a9-48da-aacf-0f1c481c05a7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:36.405453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-68593d74-26a9-48da-aacf-0f1c481c05a7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:36.405925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-68593d74-26a9-48da-aacf-0f1c481c05a7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:36.434783 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-68593d74-26a9-48da-aacf-0f1c481c05a7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044789 Aug 28 01:12:36.435322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 560/1349] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:36 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:37.185684 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-69391f2b-9056-4759-b19f-08e77da6e76f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:37.189153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-69391f2b-9056-4759-b19f-08e77da6e76f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:37.199664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69391f2b-9056-4759-b19f-08e77da6e76f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:37.200422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69391f2b-9056-4759-b19f-08e77da6e76f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:37.201152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69391f2b-9056-4759-b19f-08e77da6e76f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:37.202169 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69391f2b-9056-4759-b19f-08e77da6e76f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:37.202898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69391f2b-9056-4759-b19f-08e77da6e76f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:37.203318 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69391f2b-9056-4759-b19f-08e77da6e76f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:37.239441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-69391f2b-9056-4759-b19f-08e77da6e76f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059860 Aug 28 01:12:37.239996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 790/1350] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:37 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:37.451122 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-48e5f48a-1308-4a74-91ae-6bf3ad030bdc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:37.453253 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-48e5f48a-1308-4a74-91ae-6bf3ad030bdc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:37.462237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48e5f48a-1308-4a74-91ae-6bf3ad030bdc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:37.462836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48e5f48a-1308-4a74-91ae-6bf3ad030bdc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:37.463420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48e5f48a-1308-4a74-91ae-6bf3ad030bdc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:37.464373 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48e5f48a-1308-4a74-91ae-6bf3ad030bdc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:37.464848 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48e5f48a-1308-4a74-91ae-6bf3ad030bdc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:37.465389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48e5f48a-1308-4a74-91ae-6bf3ad030bdc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:37.497368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-48e5f48a-1308-4a74-91ae-6bf3ad030bdc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050737 Aug 28 01:12:37.497894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 561/1351] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:37 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:38.252213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-564e4f14-c1d2-40e0-8207-73efc6a910d5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:38.254352 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-564e4f14-c1d2-40e0-8207-73efc6a910d5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:38.260824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-564e4f14-c1d2-40e0-8207-73efc6a910d5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:38.261333 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-564e4f14-c1d2-40e0-8207-73efc6a910d5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:38.261767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-564e4f14-c1d2-40e0-8207-73efc6a910d5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:38.262450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-564e4f14-c1d2-40e0-8207-73efc6a910d5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:38.262889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-564e4f14-c1d2-40e0-8207-73efc6a910d5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:38.263312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-564e4f14-c1d2-40e0-8207-73efc6a910d5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:38.293177 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-564e4f14-c1d2-40e0-8207-73efc6a910d5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045029 Aug 28 01:12:38.293722 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 791/1352] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:38 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:38.510865 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2d87ec6d-a5a0-4e61-9322-381f69bd8997 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:38.513781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2d87ec6d-a5a0-4e61-9322-381f69bd8997 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:38.521847 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d87ec6d-a5a0-4e61-9322-381f69bd8997 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:38.522358 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d87ec6d-a5a0-4e61-9322-381f69bd8997 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:38.522772 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d87ec6d-a5a0-4e61-9322-381f69bd8997 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:38.523471 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d87ec6d-a5a0-4e61-9322-381f69bd8997 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:38.523911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d87ec6d-a5a0-4e61-9322-381f69bd8997 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:38.524325 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d87ec6d-a5a0-4e61-9322-381f69bd8997 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:38.556399 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2d87ec6d-a5a0-4e61-9322-381f69bd8997 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049731 Aug 28 01:12:38.556993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 562/1353] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:38 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:39.308796 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e654ab35-2669-4c91-bba4-2a3279f909d5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:39.311603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e654ab35-2669-4c91-bba4-2a3279f909d5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:39.321688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e654ab35-2669-4c91-bba4-2a3279f909d5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:39.322212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e654ab35-2669-4c91-bba4-2a3279f909d5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:39.322792 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e654ab35-2669-4c91-bba4-2a3279f909d5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:39.323653 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e654ab35-2669-4c91-bba4-2a3279f909d5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:39.324254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e654ab35-2669-4c91-bba4-2a3279f909d5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:39.324798 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e654ab35-2669-4c91-bba4-2a3279f909d5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:39.357665 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e654ab35-2669-4c91-bba4-2a3279f909d5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054593 Aug 28 01:12:39.358223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 792/1354] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:39 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:39.570396 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b3b6e97a-1c12-41d3-9f2c-5455c688c811 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:39.572528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b3b6e97a-1c12-41d3-9f2c-5455c688c811 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:39.579738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3b6e97a-1c12-41d3-9f2c-5455c688c811 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:39.580218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3b6e97a-1c12-41d3-9f2c-5455c688c811 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:39.580641 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3b6e97a-1c12-41d3-9f2c-5455c688c811 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:39.581366 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3b6e97a-1c12-41d3-9f2c-5455c688c811 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:39.581813 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3b6e97a-1c12-41d3-9f2c-5455c688c811 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:39.582215 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3b6e97a-1c12-41d3-9f2c-5455c688c811 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:39.613185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b3b6e97a-1c12-41d3-9f2c-5455c688c811 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047113 Aug 28 01:12:39.613750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 563/1355] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:39 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:40.372924 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-015b163d-59f5-444e-9a00-8ceba319cb2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:40.376274 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-015b163d-59f5-444e-9a00-8ceba319cb2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:40.384022 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-015b163d-59f5-444e-9a00-8ceba319cb2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:40.384500 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-015b163d-59f5-444e-9a00-8ceba319cb2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:40.384935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-015b163d-59f5-444e-9a00-8ceba319cb2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:40.385643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-015b163d-59f5-444e-9a00-8ceba319cb2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:40.386082 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-015b163d-59f5-444e-9a00-8ceba319cb2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:40.386503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-015b163d-59f5-444e-9a00-8ceba319cb2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:40.418543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-015b163d-59f5-444e-9a00-8ceba319cb2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051963 Aug 28 01:12:40.419119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 793/1356] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:40 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:40.629131 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-047c1fa3-2c8a-4bd3-9f30-ed674892cac5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:40.631308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-047c1fa3-2c8a-4bd3-9f30-ed674892cac5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:40.638317 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-047c1fa3-2c8a-4bd3-9f30-ed674892cac5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:40.638799 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-047c1fa3-2c8a-4bd3-9f30-ed674892cac5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:40.639231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-047c1fa3-2c8a-4bd3-9f30-ed674892cac5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:40.639924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-047c1fa3-2c8a-4bd3-9f30-ed674892cac5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:40.640368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-047c1fa3-2c8a-4bd3-9f30-ed674892cac5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:40.640788 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-047c1fa3-2c8a-4bd3-9f30-ed674892cac5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:40.672273 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-047c1fa3-2c8a-4bd3-9f30-ed674892cac5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049344 Aug 28 01:12:40.672822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 564/1357] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:40 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:41.431690 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a6fa82d4-5a28-43fe-bcd4-cb9d5a49016d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:41.433828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a6fa82d4-5a28-43fe-bcd4-cb9d5a49016d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:41.440601 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6fa82d4-5a28-43fe-bcd4-cb9d5a49016d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:41.441118 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6fa82d4-5a28-43fe-bcd4-cb9d5a49016d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:41.441551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6fa82d4-5a28-43fe-bcd4-cb9d5a49016d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:41.442242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6fa82d4-5a28-43fe-bcd4-cb9d5a49016d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:41.442686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6fa82d4-5a28-43fe-bcd4-cb9d5a49016d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:41.443102 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6fa82d4-5a28-43fe-bcd4-cb9d5a49016d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:41.474354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a6fa82d4-5a28-43fe-bcd4-cb9d5a49016d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046827 Aug 28 01:12:41.474915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 794/1358] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:41 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:41.687633 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-04b26132-aae6-4e06-9b9d-e46bb71176f0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:41.690578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-04b26132-aae6-4e06-9b9d-e46bb71176f0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:41.700700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04b26132-aae6-4e06-9b9d-e46bb71176f0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:41.701394 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04b26132-aae6-4e06-9b9d-e46bb71176f0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:41.701996 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04b26132-aae6-4e06-9b9d-e46bb71176f0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:41.702980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04b26132-aae6-4e06-9b9d-e46bb71176f0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:41.703577 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04b26132-aae6-4e06-9b9d-e46bb71176f0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:41.704178 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04b26132-aae6-4e06-9b9d-e46bb71176f0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:41.743095 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-04b26132-aae6-4e06-9b9d-e46bb71176f0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061824 Aug 28 01:12:41.743624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 565/1359] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:41 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:42.487400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-959e4e26-4630-4c00-8e76-1603bc6f00ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:42.489668 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-959e4e26-4630-4c00-8e76-1603bc6f00ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:42.496490 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-959e4e26-4630-4c00-8e76-1603bc6f00ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:42.496899 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-959e4e26-4630-4c00-8e76-1603bc6f00ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:42.497368 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-959e4e26-4630-4c00-8e76-1603bc6f00ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:42.498056 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-959e4e26-4630-4c00-8e76-1603bc6f00ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:42.498525 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-959e4e26-4630-4c00-8e76-1603bc6f00ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:42.498921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-959e4e26-4630-4c00-8e76-1603bc6f00ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:42.535554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-959e4e26-4630-4c00-8e76-1603bc6f00ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052273 Aug 28 01:12:42.536109 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 795/1360] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:42 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:42.756356 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f86bddc9-6da9-4759-8352-be56b6d716cf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:42.758483 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f86bddc9-6da9-4759-8352-be56b6d716cf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:42.765521 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f86bddc9-6da9-4759-8352-be56b6d716cf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:42.766014 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f86bddc9-6da9-4759-8352-be56b6d716cf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:42.766447 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f86bddc9-6da9-4759-8352-be56b6d716cf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:42.767144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f86bddc9-6da9-4759-8352-be56b6d716cf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:42.767603 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f86bddc9-6da9-4759-8352-be56b6d716cf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:42.768014 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f86bddc9-6da9-4759-8352-be56b6d716cf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:42.800303 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f86bddc9-6da9-4759-8352-be56b6d716cf tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048094 Aug 28 01:12:42.800862 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 566/1361] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:42 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:43.548314 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-08d6f7ce-6d55-4907-a598-74d6cd0f9411 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:43.550497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-08d6f7ce-6d55-4907-a598-74d6cd0f9411 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:43.557651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08d6f7ce-6d55-4907-a598-74d6cd0f9411 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:43.558066 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08d6f7ce-6d55-4907-a598-74d6cd0f9411 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:43.558497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08d6f7ce-6d55-4907-a598-74d6cd0f9411 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:43.559186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08d6f7ce-6d55-4907-a598-74d6cd0f9411 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:43.559633 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08d6f7ce-6d55-4907-a598-74d6cd0f9411 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:43.560051 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08d6f7ce-6d55-4907-a598-74d6cd0f9411 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:43.601522 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-08d6f7ce-6d55-4907-a598-74d6cd0f9411 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057186 Aug 28 01:12:43.602087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 796/1362] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:43 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:43.815754 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6a67e2fc-78cf-4266-8fc6-5c43718252b4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:43.818529 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6a67e2fc-78cf-4266-8fc6-5c43718252b4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:43.828044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6a67e2fc-78cf-4266-8fc6-5c43718252b4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:43.828680 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6a67e2fc-78cf-4266-8fc6-5c43718252b4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:43.829370 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6a67e2fc-78cf-4266-8fc6-5c43718252b4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:43.830334 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6a67e2fc-78cf-4266-8fc6-5c43718252b4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:43.830946 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6a67e2fc-78cf-4266-8fc6-5c43718252b4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:43.831544 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6a67e2fc-78cf-4266-8fc6-5c43718252b4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:43.862379 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6a67e2fc-78cf-4266-8fc6-5c43718252b4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052146 Aug 28 01:12:43.862931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 567/1363] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:43 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:44.615646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-aa341ef7-4d3d-46f2-a950-cec9674c3cf0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:44.617919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-aa341ef7-4d3d-46f2-a950-cec9674c3cf0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:44.625179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa341ef7-4d3d-46f2-a950-cec9674c3cf0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:44.625656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa341ef7-4d3d-46f2-a950-cec9674c3cf0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:44.626080 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa341ef7-4d3d-46f2-a950-cec9674c3cf0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:44.626763 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa341ef7-4d3d-46f2-a950-cec9674c3cf0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:44.627206 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa341ef7-4d3d-46f2-a950-cec9674c3cf0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:44.627619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa341ef7-4d3d-46f2-a950-cec9674c3cf0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:44.657779 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-aa341ef7-4d3d-46f2-a950-cec9674c3cf0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046529 Aug 28 01:12:44.658329 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 797/1364] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:44 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:44.875744 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b57494ba-1724-4bc4-8a45-4e8d73da7c7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:44.877964 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b57494ba-1724-4bc4-8a45-4e8d73da7c7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:44.884977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b57494ba-1724-4bc4-8a45-4e8d73da7c7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:44.885437 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b57494ba-1724-4bc4-8a45-4e8d73da7c7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:44.885902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b57494ba-1724-4bc4-8a45-4e8d73da7c7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:44.886584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b57494ba-1724-4bc4-8a45-4e8d73da7c7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:44.887038 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b57494ba-1724-4bc4-8a45-4e8d73da7c7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:44.887443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b57494ba-1724-4bc4-8a45-4e8d73da7c7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:44.916998 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b57494ba-1724-4bc4-8a45-4e8d73da7c7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045578 Aug 28 01:12:44.917563 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 568/1365] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:44 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:45.672576 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-000a0e36-5cec-44f2-bf80-a6604d5c5822 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:45.674748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-000a0e36-5cec-44f2-bf80-a6604d5c5822 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:45.677119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=58 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:12:45.682500 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-000a0e36-5cec-44f2-bf80-a6604d5c5822 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:45.682978 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-000a0e36-5cec-44f2-bf80-a6604d5c5822 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:45.683410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-000a0e36-5cec-44f2-bf80-a6604d5c5822 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:45.684108 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-000a0e36-5cec-44f2-bf80-a6604d5c5822 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:45.684578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-000a0e36-5cec-44f2-bf80-a6604d5c5822 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:45.684993 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-000a0e36-5cec-44f2-bf80-a6604d5c5822 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:45.715157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-000a0e36-5cec-44f2-bf80-a6604d5c5822 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046590 Aug 28 01:12:45.715646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 798/1366] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:45 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:45.930209 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-605f1944-fed5-477e-aa95-b75f7a66630c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:45.932396 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-605f1944-fed5-477e-aa95-b75f7a66630c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:45.935174 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=58 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:12:45.940389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-605f1944-fed5-477e-aa95-b75f7a66630c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:45.940863 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-605f1944-fed5-477e-aa95-b75f7a66630c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:45.941314 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-605f1944-fed5-477e-aa95-b75f7a66630c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:45.942004 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-605f1944-fed5-477e-aa95-b75f7a66630c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:45.942443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-605f1944-fed5-477e-aa95-b75f7a66630c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:45.942855 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-605f1944-fed5-477e-aa95-b75f7a66630c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:45.973460 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-605f1944-fed5-477e-aa95-b75f7a66630c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047107 Aug 28 01:12:45.973919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 569/1367] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:45 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:46.730089 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3cf51d54-b304-490a-b8dc-dbdc4b52f037 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:46.733793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3cf51d54-b304-490a-b8dc-dbdc4b52f037 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:46.742722 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3cf51d54-b304-490a-b8dc-dbdc4b52f037 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:46.743229 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3cf51d54-b304-490a-b8dc-dbdc4b52f037 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:46.743717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3cf51d54-b304-490a-b8dc-dbdc4b52f037 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:46.744361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3cf51d54-b304-490a-b8dc-dbdc4b52f037 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:46.744793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3cf51d54-b304-490a-b8dc-dbdc4b52f037 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:46.745231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3cf51d54-b304-490a-b8dc-dbdc4b52f037 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:46.774614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3cf51d54-b304-490a-b8dc-dbdc4b52f037 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049587 Aug 28 01:12:46.775083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 799/1368] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:46 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:46.988270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1bc76905-0d71-49b8-a363-f598185cc97b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:46.990658 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1bc76905-0d71-49b8-a363-f598185cc97b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:46.997940 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bc76905-0d71-49b8-a363-f598185cc97b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:46.998444 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bc76905-0d71-49b8-a363-f598185cc97b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:46.998899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bc76905-0d71-49b8-a363-f598185cc97b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:46.999637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bc76905-0d71-49b8-a363-f598185cc97b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:47.000108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bc76905-0d71-49b8-a363-f598185cc97b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:47.000548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bc76905-0d71-49b8-a363-f598185cc97b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:47.032166 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1bc76905-0d71-49b8-a363-f598185cc97b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048015 Aug 28 01:12:47.032651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 570/1369] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:46 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:47.788909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e282b27c-190a-4acf-8111-63baf3f78aab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:47.791149 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e282b27c-190a-4acf-8111-63baf3f78aab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:47.797658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e282b27c-190a-4acf-8111-63baf3f78aab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:47.798148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e282b27c-190a-4acf-8111-63baf3f78aab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:47.798578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e282b27c-190a-4acf-8111-63baf3f78aab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:47.799276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e282b27c-190a-4acf-8111-63baf3f78aab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:47.799720 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e282b27c-190a-4acf-8111-63baf3f78aab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:47.800143 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e282b27c-190a-4acf-8111-63baf3f78aab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:47.830656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e282b27c-190a-4acf-8111-63baf3f78aab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045862 Aug 28 01:12:47.831128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 800/1370] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:47 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:48.045005 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-963f92b9-5440-4820-b5a6-26266604bf41 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:48.047230 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-963f92b9-5440-4820-b5a6-26266604bf41 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:48.054160 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-963f92b9-5440-4820-b5a6-26266604bf41 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:48.054646 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-963f92b9-5440-4820-b5a6-26266604bf41 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:48.055066 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-963f92b9-5440-4820-b5a6-26266604bf41 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:48.055757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-963f92b9-5440-4820-b5a6-26266604bf41 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:48.056203 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-963f92b9-5440-4820-b5a6-26266604bf41 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:48.056623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-963f92b9-5440-4820-b5a6-26266604bf41 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:48.087581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-963f92b9-5440-4820-b5a6-26266604bf41 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046539 Aug 28 01:12:48.088048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 571/1371] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:48 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:48.843487 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a6b2906d-8b43-4657-8c34-276b36a3c616 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:48.845811 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a6b2906d-8b43-4657-8c34-276b36a3c616 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:48.855497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6b2906d-8b43-4657-8c34-276b36a3c616 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:48.856249 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6b2906d-8b43-4657-8c34-276b36a3c616 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:48.856889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6b2906d-8b43-4657-8c34-276b36a3c616 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:48.857926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6b2906d-8b43-4657-8c34-276b36a3c616 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:48.858369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6b2906d-8b43-4657-8c34-276b36a3c616 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:48.858922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6b2906d-8b43-4657-8c34-276b36a3c616 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:48.891594 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a6b2906d-8b43-4657-8c34-276b36a3c616 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052103 Aug 28 01:12:48.892144 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 801/1372] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:48 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:49.102130 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2e1ce34c-ad11-4361-b682-fdb0ea656e04 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:49.104291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2e1ce34c-ad11-4361-b682-fdb0ea656e04 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:49.111236 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e1ce34c-ad11-4361-b682-fdb0ea656e04 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:49.111727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e1ce34c-ad11-4361-b682-fdb0ea656e04 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:49.112165 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e1ce34c-ad11-4361-b682-fdb0ea656e04 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:49.112880 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e1ce34c-ad11-4361-b682-fdb0ea656e04 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:49.113367 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e1ce34c-ad11-4361-b682-fdb0ea656e04 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:49.113790 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e1ce34c-ad11-4361-b682-fdb0ea656e04 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:49.145199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2e1ce34c-ad11-4361-b682-fdb0ea656e04 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046777 Aug 28 01:12:49.145676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 572/1373] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:49 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:49.904303 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c5b00413-881a-4e50-85cd-43a236de7f07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:49.906516 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c5b00413-881a-4e50-85cd-43a236de7f07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:49.913027 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5b00413-881a-4e50-85cd-43a236de7f07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:49.913511 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5b00413-881a-4e50-85cd-43a236de7f07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:49.913943 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5b00413-881a-4e50-85cd-43a236de7f07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:49.914634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5b00413-881a-4e50-85cd-43a236de7f07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:49.915088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5b00413-881a-4e50-85cd-43a236de7f07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:49.915508 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5b00413-881a-4e50-85cd-43a236de7f07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:49.919167 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:12:49.947068 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c5b00413-881a-4e50-85cd-43a236de7f07 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046664 Aug 28 01:12:49.947568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 802/1374] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:49 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:50.157312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4179b82e-2fd2-4b2c-9885-a43bf3e95c08 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:50.159518 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4179b82e-2fd2-4b2c-9885-a43bf3e95c08 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:50.166503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4179b82e-2fd2-4b2c-9885-a43bf3e95c08 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:50.166981 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4179b82e-2fd2-4b2c-9885-a43bf3e95c08 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:50.167410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4179b82e-2fd2-4b2c-9885-a43bf3e95c08 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:50.168102 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4179b82e-2fd2-4b2c-9885-a43bf3e95c08 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:50.168548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4179b82e-2fd2-4b2c-9885-a43bf3e95c08 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:50.168996 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4179b82e-2fd2-4b2c-9885-a43bf3e95c08 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:50.200749 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4179b82e-2fd2-4b2c-9885-a43bf3e95c08 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047331 Aug 28 01:12:50.201260 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 573/1375] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:50 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:50.959865 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a9944809-652c-4911-b32f-d7b5b05c98c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:50.962244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a9944809-652c-4911-b32f-d7b5b05c98c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:50.968556 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9944809-652c-4911-b32f-d7b5b05c98c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:50.969056 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9944809-652c-4911-b32f-d7b5b05c98c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:50.969486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9944809-652c-4911-b32f-d7b5b05c98c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:50.970175 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9944809-652c-4911-b32f-d7b5b05c98c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:50.970608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9944809-652c-4911-b32f-d7b5b05c98c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:50.971011 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9944809-652c-4911-b32f-d7b5b05c98c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:51.000797 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a9944809-652c-4911-b32f-d7b5b05c98c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045425 Aug 28 01:12:51.001385 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 803/1376] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:50 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:51.213993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-344db148-7bef-4199-8f4a-c3c4711d1629 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:51.216265 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-344db148-7bef-4199-8f4a-c3c4711d1629 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:51.223151 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-344db148-7bef-4199-8f4a-c3c4711d1629 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:51.223635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-344db148-7bef-4199-8f4a-c3c4711d1629 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:51.224067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-344db148-7bef-4199-8f4a-c3c4711d1629 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:51.224762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-344db148-7bef-4199-8f4a-c3c4711d1629 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:51.225325 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-344db148-7bef-4199-8f4a-c3c4711d1629 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:51.225746 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-344db148-7bef-4199-8f4a-c3c4711d1629 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:51.243661 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:12:51.259072 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-344db148-7bef-4199-8f4a-c3c4711d1629 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049600 Aug 28 01:12:51.259633 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 574/1377] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:51 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:52.015958 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9f9afd79-ba9f-4061-a38c-c2caeaf9cc74 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:52.019071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9f9afd79-ba9f-4061-a38c-c2caeaf9cc74 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:52.028307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9f9afd79-ba9f-4061-a38c-c2caeaf9cc74 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:52.028791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9f9afd79-ba9f-4061-a38c-c2caeaf9cc74 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:52.029254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9f9afd79-ba9f-4061-a38c-c2caeaf9cc74 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:52.029960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9f9afd79-ba9f-4061-a38c-c2caeaf9cc74 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:52.030438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9f9afd79-ba9f-4061-a38c-c2caeaf9cc74 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:52.030853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9f9afd79-ba9f-4061-a38c-c2caeaf9cc74 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:52.062384 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9f9afd79-ba9f-4061-a38c-c2caeaf9cc74 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051939 Aug 28 01:12:52.062950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 804/1378] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:52 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:52.271741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-59f14a4d-d5d9-417a-a3e7-c96a7eed67a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:52.273920 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-59f14a4d-d5d9-417a-a3e7-c96a7eed67a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:52.280851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-59f14a4d-d5d9-417a-a3e7-c96a7eed67a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:52.281362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-59f14a4d-d5d9-417a-a3e7-c96a7eed67a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:52.281790 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-59f14a4d-d5d9-417a-a3e7-c96a7eed67a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:52.282481 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-59f14a4d-d5d9-417a-a3e7-c96a7eed67a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:52.282926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-59f14a4d-d5d9-417a-a3e7-c96a7eed67a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:52.283341 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-59f14a4d-d5d9-417a-a3e7-c96a7eed67a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:52.322932 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-59f14a4d-d5d9-417a-a3e7-c96a7eed67a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055078 Aug 28 01:12:52.323488 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 575/1379] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:52 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:53.075753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c76f3cb1-43e1-4456-ad29-a50d9d4f307b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:53.077922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c76f3cb1-43e1-4456-ad29-a50d9d4f307b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:53.084771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c76f3cb1-43e1-4456-ad29-a50d9d4f307b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:53.085282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c76f3cb1-43e1-4456-ad29-a50d9d4f307b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:53.085709 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c76f3cb1-43e1-4456-ad29-a50d9d4f307b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:53.086400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c76f3cb1-43e1-4456-ad29-a50d9d4f307b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:53.086841 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c76f3cb1-43e1-4456-ad29-a50d9d4f307b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:53.087259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c76f3cb1-43e1-4456-ad29-a50d9d4f307b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:53.119760 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c76f3cb1-43e1-4456-ad29-a50d9d4f307b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048006 Aug 28 01:12:53.120694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 805/1380] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:53 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:53.335837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b8b03a08-9cf7-4b1e-b27b-096bf9e61544 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:53.338036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b8b03a08-9cf7-4b1e-b27b-096bf9e61544 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:53.344669 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b8b03a08-9cf7-4b1e-b27b-096bf9e61544 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:53.345193 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b8b03a08-9cf7-4b1e-b27b-096bf9e61544 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:53.345628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b8b03a08-9cf7-4b1e-b27b-096bf9e61544 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:53.346362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b8b03a08-9cf7-4b1e-b27b-096bf9e61544 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:53.346809 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b8b03a08-9cf7-4b1e-b27b-096bf9e61544 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:53.347234 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b8b03a08-9cf7-4b1e-b27b-096bf9e61544 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:53.378385 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b8b03a08-9cf7-4b1e-b27b-096bf9e61544 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046799 Aug 28 01:12:53.378943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 576/1381] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:53 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:54.132145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-60869614-d18e-4af1-b751-f535f76ca27c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:54.134399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-60869614-d18e-4af1-b751-f535f76ca27c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:54.141089 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60869614-d18e-4af1-b751-f535f76ca27c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:54.141578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60869614-d18e-4af1-b751-f535f76ca27c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:54.142005 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60869614-d18e-4af1-b751-f535f76ca27c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:54.142712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60869614-d18e-4af1-b751-f535f76ca27c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:54.143153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60869614-d18e-4af1-b751-f535f76ca27c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:54.143570 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60869614-d18e-4af1-b751-f535f76ca27c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:54.173333 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-60869614-d18e-4af1-b751-f535f76ca27c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045130 Aug 28 01:12:54.173894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 806/1382] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:54 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:54.391411 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0d14b97e-3d13-4221-8534-576b6e8085b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:54.393622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0d14b97e-3d13-4221-8534-576b6e8085b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:54.400432 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0d14b97e-3d13-4221-8534-576b6e8085b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:54.400917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0d14b97e-3d13-4221-8534-576b6e8085b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:54.401374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0d14b97e-3d13-4221-8534-576b6e8085b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:54.402074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0d14b97e-3d13-4221-8534-576b6e8085b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:54.402521 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0d14b97e-3d13-4221-8534-576b6e8085b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:54.402971 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0d14b97e-3d13-4221-8534-576b6e8085b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:54.433358 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0d14b97e-3d13-4221-8534-576b6e8085b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045986 Aug 28 01:12:54.433929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 577/1383] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:54 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:55.186073 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8799e1fe-482b-467f-9dbe-553796be23f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:55.188272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8799e1fe-482b-467f-9dbe-553796be23f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:55.195972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8799e1fe-482b-467f-9dbe-553796be23f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:55.196445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8799e1fe-482b-467f-9dbe-553796be23f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:55.196910 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8799e1fe-482b-467f-9dbe-553796be23f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:55.197692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8799e1fe-482b-467f-9dbe-553796be23f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:55.198137 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8799e1fe-482b-467f-9dbe-553796be23f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:55.198553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8799e1fe-482b-467f-9dbe-553796be23f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:55.229757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8799e1fe-482b-467f-9dbe-553796be23f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047696 Aug 28 01:12:55.230340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 807/1384] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:55 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:55.447588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1bf43c98-7a2c-412b-bd35-1e975130a74b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:55.449786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1bf43c98-7a2c-412b-bd35-1e975130a74b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:55.456642 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bf43c98-7a2c-412b-bd35-1e975130a74b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:55.457156 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bf43c98-7a2c-412b-bd35-1e975130a74b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:55.457590 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bf43c98-7a2c-412b-bd35-1e975130a74b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:55.458282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bf43c98-7a2c-412b-bd35-1e975130a74b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:55.458729 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bf43c98-7a2c-412b-bd35-1e975130a74b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:55.459146 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bf43c98-7a2c-412b-bd35-1e975130a74b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:55.489014 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1bf43c98-7a2c-412b-bd35-1e975130a74b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046496 Aug 28 01:12:55.489596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 578/1385] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:55 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:56.244902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ef88280e-83f2-46b9-9e99-e70ab37192dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:56.247088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ef88280e-83f2-46b9-9e99-e70ab37192dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:56.253899 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef88280e-83f2-46b9-9e99-e70ab37192dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:56.254382 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef88280e-83f2-46b9-9e99-e70ab37192dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:56.254818 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef88280e-83f2-46b9-9e99-e70ab37192dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:56.255517 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef88280e-83f2-46b9-9e99-e70ab37192dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:56.255970 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef88280e-83f2-46b9-9e99-e70ab37192dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:56.256395 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef88280e-83f2-46b9-9e99-e70ab37192dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:56.286362 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ef88280e-83f2-46b9-9e99-e70ab37192dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047095 Aug 28 01:12:56.286915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 808/1386] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:56 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:56.502367 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c1d526c3-d84c-4eb8-b8ef-9714d82cd63b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:56.504839 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c1d526c3-d84c-4eb8-b8ef-9714d82cd63b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:56.513164 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1d526c3-d84c-4eb8-b8ef-9714d82cd63b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:56.513715 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1d526c3-d84c-4eb8-b8ef-9714d82cd63b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:56.514224 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1d526c3-d84c-4eb8-b8ef-9714d82cd63b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:56.514979 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1d526c3-d84c-4eb8-b8ef-9714d82cd63b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:56.515505 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1d526c3-d84c-4eb8-b8ef-9714d82cd63b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:56.515981 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1d526c3-d84c-4eb8-b8ef-9714d82cd63b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:56.547073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c1d526c3-d84c-4eb8-b8ef-9714d82cd63b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048995 Aug 28 01:12:56.547612 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 579/1387] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:56 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:57.299472 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-19c4d989-9895-418c-9ed7-4686730384fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:57.301582 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-19c4d989-9895-418c-9ed7-4686730384fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:57.308566 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-19c4d989-9895-418c-9ed7-4686730384fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:57.309074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-19c4d989-9895-418c-9ed7-4686730384fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:57.309507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-19c4d989-9895-418c-9ed7-4686730384fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:57.310203 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-19c4d989-9895-418c-9ed7-4686730384fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:57.310647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-19c4d989-9895-418c-9ed7-4686730384fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:57.311066 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-19c4d989-9895-418c-9ed7-4686730384fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:57.342071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-19c4d989-9895-418c-9ed7-4686730384fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046622 Aug 28 01:12:57.342633 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 809/1388] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:57 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:57.562665 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c34c3b60-f6e6-4d1d-b401-e324bfed2abc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:57.565449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c34c3b60-f6e6-4d1d-b401-e324bfed2abc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:57.574216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c34c3b60-f6e6-4d1d-b401-e324bfed2abc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:57.574699 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c34c3b60-f6e6-4d1d-b401-e324bfed2abc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:57.575129 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c34c3b60-f6e6-4d1d-b401-e324bfed2abc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:57.575819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c34c3b60-f6e6-4d1d-b401-e324bfed2abc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:57.576269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c34c3b60-f6e6-4d1d-b401-e324bfed2abc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:57.576691 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c34c3b60-f6e6-4d1d-b401-e324bfed2abc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:57.606585 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c34c3b60-f6e6-4d1d-b401-e324bfed2abc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049675 Aug 28 01:12:57.607136 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 580/1389] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:57 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:58.354860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7fdab14c-1d12-4438-80c3-d2e97aa5b513 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:58.357145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7fdab14c-1d12-4438-80c3-d2e97aa5b513 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:58.364319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fdab14c-1d12-4438-80c3-d2e97aa5b513 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:58.364921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fdab14c-1d12-4438-80c3-d2e97aa5b513 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:58.365498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fdab14c-1d12-4438-80c3-d2e97aa5b513 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:58.366281 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fdab14c-1d12-4438-80c3-d2e97aa5b513 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:58.366749 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fdab14c-1d12-4438-80c3-d2e97aa5b513 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:58.367170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fdab14c-1d12-4438-80c3-d2e97aa5b513 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:58.398671 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7fdab14c-1d12-4438-80c3-d2e97aa5b513 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047795 Aug 28 01:12:58.399231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 810/1390] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:58 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:58.623298 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e8f0d974-8e4f-4d13-853c-e01e738a2f74 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:58.625472 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e8f0d974-8e4f-4d13-853c-e01e738a2f74 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:58.632269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8f0d974-8e4f-4d13-853c-e01e738a2f74 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:58.632750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8f0d974-8e4f-4d13-853c-e01e738a2f74 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:58.633216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8f0d974-8e4f-4d13-853c-e01e738a2f74 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:58.633920 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8f0d974-8e4f-4d13-853c-e01e738a2f74 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:58.634451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8f0d974-8e4f-4d13-853c-e01e738a2f74 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:58.634848 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8f0d974-8e4f-4d13-853c-e01e738a2f74 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:58.664378 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e8f0d974-8e4f-4d13-853c-e01e738a2f74 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044966 Aug 28 01:12:58.664809 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 581/1391] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:58 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:59.411695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3637f2bf-543c-4f2a-9414-de50c27cce26 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:59.413974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3637f2bf-543c-4f2a-9414-de50c27cce26 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:59.420791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3637f2bf-543c-4f2a-9414-de50c27cce26 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:59.421310 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3637f2bf-543c-4f2a-9414-de50c27cce26 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:59.421746 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3637f2bf-543c-4f2a-9414-de50c27cce26 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:59.422445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3637f2bf-543c-4f2a-9414-de50c27cce26 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:59.422899 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3637f2bf-543c-4f2a-9414-de50c27cce26 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:59.423322 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3637f2bf-543c-4f2a-9414-de50c27cce26 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:59.453921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3637f2bf-543c-4f2a-9414-de50c27cce26 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046219 Aug 28 01:12:59.454477 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 811/1392] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:59 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:12:59.679132 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d6e663f3-1574-4d6a-9332-12a435039d44 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:12:59.682166 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d6e663f3-1574-4d6a-9332-12a435039d44 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:12:59.691950 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6e663f3-1574-4d6a-9332-12a435039d44 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:59.692607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6e663f3-1574-4d6a-9332-12a435039d44 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:59.693291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6e663f3-1574-4d6a-9332-12a435039d44 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:59.694557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6e663f3-1574-4d6a-9332-12a435039d44 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:12:59.695206 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6e663f3-1574-4d6a-9332-12a435039d44 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:12:59.695797 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6e663f3-1574-4d6a-9332-12a435039d44 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:12:59.727149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d6e663f3-1574-4d6a-9332-12a435039d44 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053743 Aug 28 01:12:59.727713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 582/1393] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:12:59 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:00.469182 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fdd93c52-3dfb-44c2-ac92-4fabe45896e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:00.471899 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fdd93c52-3dfb-44c2-ac92-4fabe45896e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:00.480341 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fdd93c52-3dfb-44c2-ac92-4fabe45896e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:00.480850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fdd93c52-3dfb-44c2-ac92-4fabe45896e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:00.481324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fdd93c52-3dfb-44c2-ac92-4fabe45896e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:00.482028 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fdd93c52-3dfb-44c2-ac92-4fabe45896e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:00.482482 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fdd93c52-3dfb-44c2-ac92-4fabe45896e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:00.482908 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fdd93c52-3dfb-44c2-ac92-4fabe45896e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:00.512644 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fdd93c52-3dfb-44c2-ac92-4fabe45896e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049692 Aug 28 01:13:00.513235 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 812/1394] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:00 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:00.739729 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-14181b76-cbbb-4b72-84c3-11840d69ccc1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:00.741939 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-14181b76-cbbb-4b72-84c3-11840d69ccc1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:00.749278 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14181b76-cbbb-4b72-84c3-11840d69ccc1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:00.749793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14181b76-cbbb-4b72-84c3-11840d69ccc1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:00.750221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14181b76-cbbb-4b72-84c3-11840d69ccc1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:00.750906 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14181b76-cbbb-4b72-84c3-11840d69ccc1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:00.751350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14181b76-cbbb-4b72-84c3-11840d69ccc1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:00.751763 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14181b76-cbbb-4b72-84c3-11840d69ccc1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:00.782479 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-14181b76-cbbb-4b72-84c3-11840d69ccc1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046723 Aug 28 01:13:00.783056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 583/1395] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:00 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:01.527551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b7c59689-a251-4e47-99e7-ebf4dc044faf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:01.529749 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b7c59689-a251-4e47-99e7-ebf4dc044faf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:01.538143 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7c59689-a251-4e47-99e7-ebf4dc044faf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:01.538886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7c59689-a251-4e47-99e7-ebf4dc044faf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:01.539585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7c59689-a251-4e47-99e7-ebf4dc044faf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:01.541237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7c59689-a251-4e47-99e7-ebf4dc044faf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:01.541970 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7c59689-a251-4e47-99e7-ebf4dc044faf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:01.542645 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7c59689-a251-4e47-99e7-ebf4dc044faf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:01.578061 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b7c59689-a251-4e47-99e7-ebf4dc044faf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055511 Aug 28 01:13:01.578622 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 813/1396] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:01 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:01.797036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6325d052-baf4-4b43-a4fe-0e27b61ee4db tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:01.799602 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6325d052-baf4-4b43-a4fe-0e27b61ee4db tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:01.807796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6325d052-baf4-4b43-a4fe-0e27b61ee4db tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:01.808273 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6325d052-baf4-4b43-a4fe-0e27b61ee4db tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:01.808699 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6325d052-baf4-4b43-a4fe-0e27b61ee4db tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:01.809430 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6325d052-baf4-4b43-a4fe-0e27b61ee4db tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:01.809872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6325d052-baf4-4b43-a4fe-0e27b61ee4db tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:01.810287 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6325d052-baf4-4b43-a4fe-0e27b61ee4db tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:01.840643 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6325d052-baf4-4b43-a4fe-0e27b61ee4db tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048762 Aug 28 01:13:01.841168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 584/1397] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:01 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:02.591020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7050451a-96b3-40d0-b966-371249343265 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:02.593216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7050451a-96b3-40d0-b966-371249343265 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:02.599960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7050451a-96b3-40d0-b966-371249343265 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:02.600437 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7050451a-96b3-40d0-b966-371249343265 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:02.600868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7050451a-96b3-40d0-b966-371249343265 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:02.601588 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7050451a-96b3-40d0-b966-371249343265 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:02.602032 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7050451a-96b3-40d0-b966-371249343265 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:02.602446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7050451a-96b3-40d0-b966-371249343265 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:02.633226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7050451a-96b3-40d0-b966-371249343265 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046152 Aug 28 01:13:02.633750 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 814/1398] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:02 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:02.857664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-94916405-a940-4153-8a80-cddd1132006a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:02.860596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-94916405-a940-4153-8a80-cddd1132006a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:02.869585 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94916405-a940-4153-8a80-cddd1132006a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:02.870068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94916405-a940-4153-8a80-cddd1132006a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:02.870686 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94916405-a940-4153-8a80-cddd1132006a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:02.871446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94916405-a940-4153-8a80-cddd1132006a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:02.871963 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94916405-a940-4153-8a80-cddd1132006a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:02.872579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94916405-a940-4153-8a80-cddd1132006a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:02.906936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-94916405-a940-4153-8a80-cddd1132006a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056193 Aug 28 01:13:02.907485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 585/1399] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:02 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:03.649201 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-450ef18b-8406-4c28-b59a-83a35e2ac9b0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:03.652340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-450ef18b-8406-4c28-b59a-83a35e2ac9b0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:03.662085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-450ef18b-8406-4c28-b59a-83a35e2ac9b0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:03.662747 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-450ef18b-8406-4c28-b59a-83a35e2ac9b0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:03.663309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-450ef18b-8406-4c28-b59a-83a35e2ac9b0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:03.664280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-450ef18b-8406-4c28-b59a-83a35e2ac9b0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:03.664877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-450ef18b-8406-4c28-b59a-83a35e2ac9b0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:03.665547 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-450ef18b-8406-4c28-b59a-83a35e2ac9b0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:03.697926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-450ef18b-8406-4c28-b59a-83a35e2ac9b0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054372 Aug 28 01:13:03.698487 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 815/1400] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:03 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:03.921078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-62f3f257-00c3-4ae5-9490-0953f0243602 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:03.923589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-62f3f257-00c3-4ae5-9490-0953f0243602 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:03.931851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-62f3f257-00c3-4ae5-9490-0953f0243602 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:03.932337 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-62f3f257-00c3-4ae5-9490-0953f0243602 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:03.932767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-62f3f257-00c3-4ae5-9490-0953f0243602 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:03.933490 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-62f3f257-00c3-4ae5-9490-0953f0243602 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:03.933936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-62f3f257-00c3-4ae5-9490-0953f0243602 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:03.934351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-62f3f257-00c3-4ae5-9490-0953f0243602 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:03.964417 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-62f3f257-00c3-4ae5-9490-0953f0243602 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048139 Aug 28 01:13:03.964951 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 586/1401] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:03 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:04.712473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0c8dde59-7620-4521-9f42-6a53422a8b51 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:04.714654 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0c8dde59-7620-4521-9f42-6a53422a8b51 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:04.721733 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c8dde59-7620-4521-9f42-6a53422a8b51 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:04.722227 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c8dde59-7620-4521-9f42-6a53422a8b51 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:04.722664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c8dde59-7620-4521-9f42-6a53422a8b51 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:04.723373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c8dde59-7620-4521-9f42-6a53422a8b51 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:04.723884 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c8dde59-7620-4521-9f42-6a53422a8b51 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:04.724241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c8dde59-7620-4521-9f42-6a53422a8b51 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:04.757251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0c8dde59-7620-4521-9f42-6a53422a8b51 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048685 Aug 28 01:13:04.757691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 816/1402] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:04 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:04.979093 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-49d642be-605a-4edd-97c1-802b7b4040a8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:04.981288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-49d642be-605a-4edd-97c1-802b7b4040a8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:04.988082 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49d642be-605a-4edd-97c1-802b7b4040a8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:04.988613 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49d642be-605a-4edd-97c1-802b7b4040a8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:04.989067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49d642be-605a-4edd-97c1-802b7b4040a8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:04.989755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49d642be-605a-4edd-97c1-802b7b4040a8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:04.990202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49d642be-605a-4edd-97c1-802b7b4040a8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:04.990645 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49d642be-605a-4edd-97c1-802b7b4040a8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:05.020991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-49d642be-605a-4edd-97c1-802b7b4040a8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045882 Aug 28 01:13:05.021461 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 587/1403] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:04 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:05.769925 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-774a7057-9534-47ce-a12b-bd5ec1dd24ef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:05.772114 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-774a7057-9534-47ce-a12b-bd5ec1dd24ef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:05.779602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-774a7057-9534-47ce-a12b-bd5ec1dd24ef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:05.780082 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-774a7057-9534-47ce-a12b-bd5ec1dd24ef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:05.780514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-774a7057-9534-47ce-a12b-bd5ec1dd24ef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:05.781257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-774a7057-9534-47ce-a12b-bd5ec1dd24ef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:05.781710 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-774a7057-9534-47ce-a12b-bd5ec1dd24ef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:05.782132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-774a7057-9534-47ce-a12b-bd5ec1dd24ef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:05.812369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-774a7057-9534-47ce-a12b-bd5ec1dd24ef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046478 Aug 28 01:13:05.812925 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 817/1404] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:05 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:06.035773 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f1b72dde-7bae-43e0-ba13-262fbb2e912e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:06.037852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f1b72dde-7bae-43e0-ba13-262fbb2e912e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:06.045278 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1b72dde-7bae-43e0-ba13-262fbb2e912e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:06.045822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1b72dde-7bae-43e0-ba13-262fbb2e912e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:06.046222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1b72dde-7bae-43e0-ba13-262fbb2e912e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:06.047170 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1b72dde-7bae-43e0-ba13-262fbb2e912e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:06.047762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1b72dde-7bae-43e0-ba13-262fbb2e912e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:06.048297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1b72dde-7bae-43e0-ba13-262fbb2e912e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:06.083029 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f1b72dde-7bae-43e0-ba13-262fbb2e912e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053277 Aug 28 01:13:06.083595 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 588/1405] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:06 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:06.829912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6ad3c6ef-f390-4ac4-9fe1-8bad41bb40d4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:06.832579 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6ad3c6ef-f390-4ac4-9fe1-8bad41bb40d4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:06.840989 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ad3c6ef-f390-4ac4-9fe1-8bad41bb40d4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:06.841492 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ad3c6ef-f390-4ac4-9fe1-8bad41bb40d4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:06.841910 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ad3c6ef-f390-4ac4-9fe1-8bad41bb40d4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:06.842609 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ad3c6ef-f390-4ac4-9fe1-8bad41bb40d4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:06.843057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ad3c6ef-f390-4ac4-9fe1-8bad41bb40d4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:06.843478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ad3c6ef-f390-4ac4-9fe1-8bad41bb40d4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:06.870231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:13:06.875596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6ad3c6ef-f390-4ac4-9fe1-8bad41bb40d4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051902 Aug 28 01:13:06.876151 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 818/1406] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:06 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:07.095883 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3500c6c1-ec58-4084-af66-28342828836e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:07.098097 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3500c6c1-ec58-4084-af66-28342828836e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:07.105467 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3500c6c1-ec58-4084-af66-28342828836e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:07.105949 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3500c6c1-ec58-4084-af66-28342828836e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:07.106444 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3500c6c1-ec58-4084-af66-28342828836e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:07.107143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3500c6c1-ec58-4084-af66-28342828836e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:07.107581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3500c6c1-ec58-4084-af66-28342828836e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:07.107997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3500c6c1-ec58-4084-af66-28342828836e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:07.141282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3500c6c1-ec58-4084-af66-28342828836e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049304 Aug 28 01:13:07.141981 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 589/1407] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:07 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:07.890043 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c7596dfb-7b92-43d6-80ba-d71dbcc654c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:07.892198 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c7596dfb-7b92-43d6-80ba-d71dbcc654c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:07.899242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7596dfb-7b92-43d6-80ba-d71dbcc654c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:07.899712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7596dfb-7b92-43d6-80ba-d71dbcc654c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:07.900146 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7596dfb-7b92-43d6-80ba-d71dbcc654c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:07.901166 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7596dfb-7b92-43d6-80ba-d71dbcc654c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:07.901605 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7596dfb-7b92-43d6-80ba-d71dbcc654c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:07.902020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7596dfb-7b92-43d6-80ba-d71dbcc654c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:07.934216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c7596dfb-7b92-43d6-80ba-d71dbcc654c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048210 Aug 28 01:13:07.934793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 819/1408] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:07 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:08.154509 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b0994d6d-37dc-452a-bc39-19abb0d4c194 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:08.156606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b0994d6d-37dc-452a-bc39-19abb0d4c194 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:08.166253 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0994d6d-37dc-452a-bc39-19abb0d4c194 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:08.166893 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0994d6d-37dc-452a-bc39-19abb0d4c194 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:08.167350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0994d6d-37dc-452a-bc39-19abb0d4c194 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:08.168130 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0994d6d-37dc-452a-bc39-19abb0d4c194 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:08.168590 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0994d6d-37dc-452a-bc39-19abb0d4c194 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:08.169083 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0994d6d-37dc-452a-bc39-19abb0d4c194 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:08.205934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b0994d6d-37dc-452a-bc39-19abb0d4c194 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055594 Aug 28 01:13:08.206548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 590/1409] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:08 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:08.950770 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-927de686-d526-4913-b044-178b703a7c03 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:08.953069 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-927de686-d526-4913-b044-178b703a7c03 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:08.961616 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-927de686-d526-4913-b044-178b703a7c03 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:08.962264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-927de686-d526-4913-b044-178b703a7c03 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:08.962812 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-927de686-d526-4913-b044-178b703a7c03 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:08.963624 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-927de686-d526-4913-b044-178b703a7c03 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:08.964071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-927de686-d526-4913-b044-178b703a7c03 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:08.964575 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-927de686-d526-4913-b044-178b703a7c03 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:08.996110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-927de686-d526-4913-b044-178b703a7c03 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050418 Aug 28 01:13:08.996684 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 820/1410] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:08 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:09.224886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-135df751-fb3e-4279-95c0-3e7dca27fea5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:09.227989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-135df751-fb3e-4279-95c0-3e7dca27fea5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:09.237921 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-135df751-fb3e-4279-95c0-3e7dca27fea5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:09.238593 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-135df751-fb3e-4279-95c0-3e7dca27fea5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:09.239202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-135df751-fb3e-4279-95c0-3e7dca27fea5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:09.240177 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-135df751-fb3e-4279-95c0-3e7dca27fea5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:09.240806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-135df751-fb3e-4279-95c0-3e7dca27fea5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:09.241455 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-135df751-fb3e-4279-95c0-3e7dca27fea5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:09.246523 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:13:09.272295 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-135df751-fb3e-4279-95c0-3e7dca27fea5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053032 Aug 28 01:13:09.272838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 591/1411] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:09 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:10.008203 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7810b752-332d-461d-9c13-df384b65b464 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:10.010433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7810b752-332d-461d-9c13-df384b65b464 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:10.018628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7810b752-332d-461d-9c13-df384b65b464 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:10.019123 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7810b752-332d-461d-9c13-df384b65b464 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:10.019572 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7810b752-332d-461d-9c13-df384b65b464 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:10.020283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7810b752-332d-461d-9c13-df384b65b464 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:10.020736 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7810b752-332d-461d-9c13-df384b65b464 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:10.021200 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7810b752-332d-461d-9c13-df384b65b464 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:10.055809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7810b752-332d-461d-9c13-df384b65b464 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051685 Aug 28 01:13:10.056416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 821/1412] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:10 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:10.284360 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-61d897ab-9086-4a6d-baf5-b9614b1721fd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:10.286495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-61d897ab-9086-4a6d-baf5-b9614b1721fd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:10.294338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-61d897ab-9086-4a6d-baf5-b9614b1721fd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:10.294913 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-61d897ab-9086-4a6d-baf5-b9614b1721fd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:10.295348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-61d897ab-9086-4a6d-baf5-b9614b1721fd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:10.296057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-61d897ab-9086-4a6d-baf5-b9614b1721fd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:10.296508 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-61d897ab-9086-4a6d-baf5-b9614b1721fd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:10.296930 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-61d897ab-9086-4a6d-baf5-b9614b1721fd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:10.328430 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-61d897ab-9086-4a6d-baf5-b9614b1721fd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048117 Aug 28 01:13:10.328985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 592/1413] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:10 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:11.070469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6fa08b75-cef6-4017-ae53-bebc75b47f7e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:11.073696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6fa08b75-cef6-4017-ae53-bebc75b47f7e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:11.081865 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6fa08b75-cef6-4017-ae53-bebc75b47f7e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:11.082394 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6fa08b75-cef6-4017-ae53-bebc75b47f7e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:11.082896 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6fa08b75-cef6-4017-ae53-bebc75b47f7e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:11.083596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6fa08b75-cef6-4017-ae53-bebc75b47f7e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:11.084044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6fa08b75-cef6-4017-ae53-bebc75b47f7e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:11.084458 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6fa08b75-cef6-4017-ae53-bebc75b47f7e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:11.116515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6fa08b75-cef6-4017-ae53-bebc75b47f7e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052174 Aug 28 01:13:11.117070 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 822/1414] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:11 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:11.341983 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-209dbad1-4e83-4420-89fa-867993a02895 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:11.344110 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-209dbad1-4e83-4420-89fa-867993a02895 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:11.351328 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-209dbad1-4e83-4420-89fa-867993a02895 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:11.351817 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-209dbad1-4e83-4420-89fa-867993a02895 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:11.352919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-209dbad1-4e83-4420-89fa-867993a02895 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:11.353174 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-209dbad1-4e83-4420-89fa-867993a02895 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:11.353527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-209dbad1-4e83-4420-89fa-867993a02895 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:11.353952 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-209dbad1-4e83-4420-89fa-867993a02895 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:11.388179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-209dbad1-4e83-4420-89fa-867993a02895 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050630 Aug 28 01:13:11.388734 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 593/1415] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:11 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:12.129271 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ec20ccb1-6e2f-45c3-a11a-266e581aa9d0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:12.131554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ec20ccb1-6e2f-45c3-a11a-266e581aa9d0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:12.138844 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec20ccb1-6e2f-45c3-a11a-266e581aa9d0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:12.139339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec20ccb1-6e2f-45c3-a11a-266e581aa9d0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:12.139777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec20ccb1-6e2f-45c3-a11a-266e581aa9d0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:12.140520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec20ccb1-6e2f-45c3-a11a-266e581aa9d0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:12.141017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec20ccb1-6e2f-45c3-a11a-266e581aa9d0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:12.141477 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec20ccb1-6e2f-45c3-a11a-266e581aa9d0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:12.172244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ec20ccb1-6e2f-45c3-a11a-266e581aa9d0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046945 Aug 28 01:13:12.172816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 823/1416] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:12 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:12.400531 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-38287be3-291f-42df-bcc9-e38c3ee30ad8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:12.402762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-38287be3-291f-42df-bcc9-e38c3ee30ad8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:12.409724 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38287be3-291f-42df-bcc9-e38c3ee30ad8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:12.410208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38287be3-291f-42df-bcc9-e38c3ee30ad8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:12.410643 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38287be3-291f-42df-bcc9-e38c3ee30ad8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:12.411343 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38287be3-291f-42df-bcc9-e38c3ee30ad8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:12.411791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38287be3-291f-42df-bcc9-e38c3ee30ad8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:12.412210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38287be3-291f-42df-bcc9-e38c3ee30ad8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:12.443668 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-38287be3-291f-42df-bcc9-e38c3ee30ad8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047037 Aug 28 01:13:12.444576 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 594/1417] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:12 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:13.183918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8f60c05d-dc3a-4ab2-9320-f049d7292628 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:13.186375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8f60c05d-dc3a-4ab2-9320-f049d7292628 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:13.193424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f60c05d-dc3a-4ab2-9320-f049d7292628 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:13.193907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f60c05d-dc3a-4ab2-9320-f049d7292628 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:13.194654 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f60c05d-dc3a-4ab2-9320-f049d7292628 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:13.195350 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f60c05d-dc3a-4ab2-9320-f049d7292628 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:13.195787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f60c05d-dc3a-4ab2-9320-f049d7292628 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:13.196620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f60c05d-dc3a-4ab2-9320-f049d7292628 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:13.227646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8f60c05d-dc3a-4ab2-9320-f049d7292628 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047531 Aug 28 01:13:13.228220 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 824/1418] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:13 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:13.457965 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f69933b6-ce97-4123-9a4e-5eaa10a34bc5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:13.460219 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f69933b6-ce97-4123-9a4e-5eaa10a34bc5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:13.467730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f69933b6-ce97-4123-9a4e-5eaa10a34bc5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:13.468231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f69933b6-ce97-4123-9a4e-5eaa10a34bc5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:13.468653 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f69933b6-ce97-4123-9a4e-5eaa10a34bc5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:13.469395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f69933b6-ce97-4123-9a4e-5eaa10a34bc5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:13.469837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f69933b6-ce97-4123-9a4e-5eaa10a34bc5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:13.470261 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f69933b6-ce97-4123-9a4e-5eaa10a34bc5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:13.503565 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f69933b6-ce97-4123-9a4e-5eaa10a34bc5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050310 Aug 28 01:13:13.504106 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 595/1419] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:13 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:14.240477 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0b5407d5-2f93-4dc7-95ac-b8a197892b98 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:14.242856 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0b5407d5-2f93-4dc7-95ac-b8a197892b98 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:14.250383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0b5407d5-2f93-4dc7-95ac-b8a197892b98 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:14.250878 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0b5407d5-2f93-4dc7-95ac-b8a197892b98 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:14.251310 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0b5407d5-2f93-4dc7-95ac-b8a197892b98 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:14.252029 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0b5407d5-2f93-4dc7-95ac-b8a197892b98 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:14.252487 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0b5407d5-2f93-4dc7-95ac-b8a197892b98 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:14.252944 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0b5407d5-2f93-4dc7-95ac-b8a197892b98 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:14.286380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0b5407d5-2f93-4dc7-95ac-b8a197892b98 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049877 Aug 28 01:13:14.287045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 825/1420] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:14 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:14.519524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ba18c8ec-2fee-407a-b86f-c45da57e4e40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:13:14.521803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ba18c8ec-2fee-407a-b86f-c45da57e4e40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:14.528699 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba18c8ec-2fee-407a-b86f-c45da57e4e40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:14.529234 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba18c8ec-2fee-407a-b86f-c45da57e4e40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:14.529669 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba18c8ec-2fee-407a-b86f-c45da57e4e40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:14.530427 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba18c8ec-2fee-407a-b86f-c45da57e4e40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:14.530852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba18c8ec-2fee-407a-b86f-c45da57e4e40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:14.531275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba18c8ec-2fee-407a-b86f-c45da57e4e40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:14.637169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ba18c8ec-2fee-407a-b86f-c45da57e4e40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "POST /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/action" status: 200 len: 75769 microversion: 2.1 time: 0.121321 Aug 28 01:13:14.637829 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 596/1421] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Aug 28 01:13:14 2022] POST /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92/action => generated 75769 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:13:15.300148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ba0edc5b-a3a0-48ae-907b-4203341b3eb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:15.302379 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ba0edc5b-a3a0-48ae-907b-4203341b3eb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:15.313253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ba0edc5b-a3a0-48ae-907b-4203341b3eb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:15.314019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ba0edc5b-a3a0-48ae-907b-4203341b3eb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:15.314689 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ba0edc5b-a3a0-48ae-907b-4203341b3eb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:15.315788 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ba0edc5b-a3a0-48ae-907b-4203341b3eb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:15.316241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ba0edc5b-a3a0-48ae-907b-4203341b3eb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:15.316895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ba0edc5b-a3a0-48ae-907b-4203341b3eb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:15.362796 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ba0edc5b-a3a0-48ae-907b-4203341b3eb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.066920 Aug 28 01:13:15.363422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 826/1422] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:15 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:16.377026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-546b4fb1-8505-49f9-b937-e4579bcfca2a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:16.379474 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-546b4fb1-8505-49f9-b937-e4579bcfca2a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:16.382118 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=56 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:13:16.388385 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-546b4fb1-8505-49f9-b937-e4579bcfca2a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:16.388907 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-546b4fb1-8505-49f9-b937-e4579bcfca2a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:16.389372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-546b4fb1-8505-49f9-b937-e4579bcfca2a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:16.390070 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-546b4fb1-8505-49f9-b937-e4579bcfca2a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:16.390526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-546b4fb1-8505-49f9-b937-e4579bcfca2a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:16.390966 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-546b4fb1-8505-49f9-b937-e4579bcfca2a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:16.422844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-546b4fb1-8505-49f9-b937-e4579bcfca2a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050656 Aug 28 01:13:16.423792 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 597/1423] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:16 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:17.435717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-45d256a3-8b15-4d05-b33b-9c7eddfecbb0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:17.438075 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-45d256a3-8b15-4d05-b33b-9c7eddfecbb0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:17.441170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=58 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:13:17.447597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45d256a3-8b15-4d05-b33b-9c7eddfecbb0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:17.448091 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45d256a3-8b15-4d05-b33b-9c7eddfecbb0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:17.448530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45d256a3-8b15-4d05-b33b-9c7eddfecbb0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:17.449289 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45d256a3-8b15-4d05-b33b-9c7eddfecbb0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:17.449739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45d256a3-8b15-4d05-b33b-9c7eddfecbb0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:17.450161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45d256a3-8b15-4d05-b33b-9c7eddfecbb0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:17.486924 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-45d256a3-8b15-4d05-b33b-9c7eddfecbb0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055260 Aug 28 01:13:17.487497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 827/1424] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:17 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:18.500254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1b22f832-cc91-4858-a750-88fe145be391 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:18.502743 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1b22f832-cc91-4858-a750-88fe145be391 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:18.515418 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b22f832-cc91-4858-a750-88fe145be391 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:18.516091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b22f832-cc91-4858-a750-88fe145be391 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:18.516834 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b22f832-cc91-4858-a750-88fe145be391 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:18.518135 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b22f832-cc91-4858-a750-88fe145be391 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:18.518952 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b22f832-cc91-4858-a750-88fe145be391 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:18.519700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b22f832-cc91-4858-a750-88fe145be391 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:18.561879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1b22f832-cc91-4858-a750-88fe145be391 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.066805 Aug 28 01:13:18.562479 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 598/1425] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:18 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:19.577717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-32190cce-85e4-4656-bdc8-57e4e3b5b5a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:19.580279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-32190cce-85e4-4656-bdc8-57e4e3b5b5a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:19.586852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32190cce-85e4-4656-bdc8-57e4e3b5b5a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:19.587629 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32190cce-85e4-4656-bdc8-57e4e3b5b5a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:19.588064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32190cce-85e4-4656-bdc8-57e4e3b5b5a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:19.588789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32190cce-85e4-4656-bdc8-57e4e3b5b5a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:19.589261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32190cce-85e4-4656-bdc8-57e4e3b5b5a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:19.589673 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32190cce-85e4-4656-bdc8-57e4e3b5b5a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:19.623173 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-32190cce-85e4-4656-bdc8-57e4e3b5b5a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049617 Aug 28 01:13:19.623741 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 828/1426] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:19 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:20.636349 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8a9e5c26-2bac-49b2-a2cc-c6ea42b32e7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:20.638837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8a9e5c26-2bac-49b2-a2cc-c6ea42b32e7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:20.645747 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a9e5c26-2bac-49b2-a2cc-c6ea42b32e7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:20.646229 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a9e5c26-2bac-49b2-a2cc-c6ea42b32e7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:20.646659 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a9e5c26-2bac-49b2-a2cc-c6ea42b32e7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:20.647360 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a9e5c26-2bac-49b2-a2cc-c6ea42b32e7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:20.647800 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a9e5c26-2bac-49b2-a2cc-c6ea42b32e7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:20.648220 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a9e5c26-2bac-49b2-a2cc-c6ea42b32e7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:20.679370 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8a9e5c26-2bac-49b2-a2cc-c6ea42b32e7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047126 Aug 28 01:13:20.679952 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 599/1427] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:20 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:13:21.692258 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-777379e4-4467-43ca-b033-5eea32fb401f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:21.695719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-777379e4-4467-43ca-b033-5eea32fb401f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:21.703060 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-777379e4-4467-43ca-b033-5eea32fb401f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:21.703632 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-777379e4-4467-43ca-b033-5eea32fb401f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:21.704067 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-777379e4-4467-43ca-b033-5eea32fb401f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:21.704764 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-777379e4-4467-43ca-b033-5eea32fb401f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:21.705248 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-777379e4-4467-43ca-b033-5eea32fb401f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:21.705669 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-777379e4-4467-43ca-b033-5eea32fb401f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:21.739257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-777379e4-4467-43ca-b033-5eea32fb401f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050983 Aug 28 01:13:21.739860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 829/1428] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:21 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:22.751578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cb3c26c2-790c-49c4-835a-9ccfa4fc602c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:22.753846 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cb3c26c2-790c-49c4-835a-9ccfa4fc602c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:22.760766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb3c26c2-790c-49c4-835a-9ccfa4fc602c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:22.761358 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb3c26c2-790c-49c4-835a-9ccfa4fc602c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:22.761805 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb3c26c2-790c-49c4-835a-9ccfa4fc602c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:22.762540 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb3c26c2-790c-49c4-835a-9ccfa4fc602c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:22.763085 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb3c26c2-790c-49c4-835a-9ccfa4fc602c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:22.763526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb3c26c2-790c-49c4-835a-9ccfa4fc602c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:22.805749 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cb3c26c2-790c-49c4-835a-9ccfa4fc602c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058113 Aug 28 01:13:22.806403 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 600/1429] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:22 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:23.821052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-92137946-10c0-45ee-8872-1d7f599bc2c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:23.823588 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-92137946-10c0-45ee-8872-1d7f599bc2c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:23.831040 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92137946-10c0-45ee-8872-1d7f599bc2c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:23.831528 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92137946-10c0-45ee-8872-1d7f599bc2c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:23.831963 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92137946-10c0-45ee-8872-1d7f599bc2c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:23.832671 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92137946-10c0-45ee-8872-1d7f599bc2c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:23.833159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92137946-10c0-45ee-8872-1d7f599bc2c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:23.833555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92137946-10c0-45ee-8872-1d7f599bc2c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:23.864676 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-92137946-10c0-45ee-8872-1d7f599bc2c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049098 Aug 28 01:13:23.865297 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 830/1430] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:23 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:24.877670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bb3b03f3-3b32-4a44-bf4b-d3600d4e6bd1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:24.879911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bb3b03f3-3b32-4a44-bf4b-d3600d4e6bd1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:24.886924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb3b03f3-3b32-4a44-bf4b-d3600d4e6bd1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:24.887416 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb3b03f3-3b32-4a44-bf4b-d3600d4e6bd1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:24.887858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb3b03f3-3b32-4a44-bf4b-d3600d4e6bd1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:24.888566 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb3b03f3-3b32-4a44-bf4b-d3600d4e6bd1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:24.889034 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb3b03f3-3b32-4a44-bf4b-d3600d4e6bd1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:24.889457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb3b03f3-3b32-4a44-bf4b-d3600d4e6bd1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:24.921216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bb3b03f3-3b32-4a44-bf4b-d3600d4e6bd1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047502 Aug 28 01:13:24.921782 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 601/1431] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:24 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:25.934785 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8964851f-9e34-4063-9f8e-9d8e635fb5eb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:25.938119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8964851f-9e34-4063-9f8e-9d8e635fb5eb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:25.946045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8964851f-9e34-4063-9f8e-9d8e635fb5eb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:25.946564 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8964851f-9e34-4063-9f8e-9d8e635fb5eb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:25.946992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8964851f-9e34-4063-9f8e-9d8e635fb5eb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:25.947732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8964851f-9e34-4063-9f8e-9d8e635fb5eb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:25.948206 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8964851f-9e34-4063-9f8e-9d8e635fb5eb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:25.948624 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8964851f-9e34-4063-9f8e-9d8e635fb5eb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:25.980077 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8964851f-9e34-4063-9f8e-9d8e635fb5eb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050094 Aug 28 01:13:25.980607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 831/1432] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:25 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:26.993417 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9f9e1fb6-1412-4150-8d40-60eee7e87017 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:26.996061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9f9e1fb6-1412-4150-8d40-60eee7e87017 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:27.003599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f9e1fb6-1412-4150-8d40-60eee7e87017 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:27.004092 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f9e1fb6-1412-4150-8d40-60eee7e87017 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:27.004528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f9e1fb6-1412-4150-8d40-60eee7e87017 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:27.005265 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f9e1fb6-1412-4150-8d40-60eee7e87017 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:27.005764 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f9e1fb6-1412-4150-8d40-60eee7e87017 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:27.006170 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f9e1fb6-1412-4150-8d40-60eee7e87017 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:27.038645 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9f9e1fb6-1412-4150-8d40-60eee7e87017 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049376 Aug 28 01:13:27.039221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 602/1433] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:26 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:28.051294 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-69b6062e-3352-49a3-a243-6943191a16f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:28.053545 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-69b6062e-3352-49a3-a243-6943191a16f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:28.060769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69b6062e-3352-49a3-a243-6943191a16f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:28.061294 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69b6062e-3352-49a3-a243-6943191a16f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:28.061723 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69b6062e-3352-49a3-a243-6943191a16f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:28.062416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69b6062e-3352-49a3-a243-6943191a16f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:28.063030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69b6062e-3352-49a3-a243-6943191a16f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:28.063264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69b6062e-3352-49a3-a243-6943191a16f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:28.094344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-69b6062e-3352-49a3-a243-6943191a16f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047026 Aug 28 01:13:28.094922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 832/1434] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:28 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:29.107382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4ad90f67-911a-4769-975b-d9f311736c08 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:29.109599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4ad90f67-911a-4769-975b-d9f311736c08 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:29.116687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ad90f67-911a-4769-975b-d9f311736c08 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:29.117197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ad90f67-911a-4769-975b-d9f311736c08 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:29.117625 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ad90f67-911a-4769-975b-d9f311736c08 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:29.118345 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ad90f67-911a-4769-975b-d9f311736c08 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:29.118791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ad90f67-911a-4769-975b-d9f311736c08 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:29.119208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ad90f67-911a-4769-975b-d9f311736c08 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:29.149436 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4ad90f67-911a-4769-975b-d9f311736c08 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046728 Aug 28 01:13:29.150269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 603/1435] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:29 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:30.161699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-166eb91f-a2f5-48a0-8918-4a729413210e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:30.164042 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-166eb91f-a2f5-48a0-8918-4a729413210e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:30.170539 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-166eb91f-a2f5-48a0-8918-4a729413210e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:30.171030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-166eb91f-a2f5-48a0-8918-4a729413210e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:30.171461 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-166eb91f-a2f5-48a0-8918-4a729413210e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:30.172163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-166eb91f-a2f5-48a0-8918-4a729413210e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:30.173260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-166eb91f-a2f5-48a0-8918-4a729413210e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:30.173687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-166eb91f-a2f5-48a0-8918-4a729413210e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:30.206204 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-166eb91f-a2f5-48a0-8918-4a729413210e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048452 Aug 28 01:13:30.206804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 833/1436] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:30 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:31.219358 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-10db2235-d787-4106-a755-31b673fcb06c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:31.222990 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-10db2235-d787-4106-a755-31b673fcb06c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:31.228431 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10db2235-d787-4106-a755-31b673fcb06c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:31.228894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10db2235-d787-4106-a755-31b673fcb06c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:31.229357 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10db2235-d787-4106-a755-31b673fcb06c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:31.230086 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10db2235-d787-4106-a755-31b673fcb06c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:31.230514 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10db2235-d787-4106-a755-31b673fcb06c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:31.230956 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10db2235-d787-4106-a755-31b673fcb06c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:31.270334 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-10db2235-d787-4106-a755-31b673fcb06c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054814 Aug 28 01:13:31.271271 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 604/1437] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:31 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:32.286235 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1a8af8fc-8408-4841-b78a-11e6fbe07554 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:32.288300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1a8af8fc-8408-4841-b78a-11e6fbe07554 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:32.295357 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a8af8fc-8408-4841-b78a-11e6fbe07554 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:32.295846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a8af8fc-8408-4841-b78a-11e6fbe07554 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:32.296279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a8af8fc-8408-4841-b78a-11e6fbe07554 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:32.297007 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a8af8fc-8408-4841-b78a-11e6fbe07554 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:32.297461 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a8af8fc-8408-4841-b78a-11e6fbe07554 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:32.297917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a8af8fc-8408-4841-b78a-11e6fbe07554 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:32.330172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1a8af8fc-8408-4841-b78a-11e6fbe07554 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048080 Aug 28 01:13:32.330871 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 834/1438] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:32 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:33.342426 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-99651dd3-b662-4e99-8fa9-5a2fd676e0bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:33.344502 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-99651dd3-b662-4e99-8fa9-5a2fd676e0bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:33.351361 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-99651dd3-b662-4e99-8fa9-5a2fd676e0bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:33.352040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-99651dd3-b662-4e99-8fa9-5a2fd676e0bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:33.352483 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-99651dd3-b662-4e99-8fa9-5a2fd676e0bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:33.353263 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-99651dd3-b662-4e99-8fa9-5a2fd676e0bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:33.353811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-99651dd3-b662-4e99-8fa9-5a2fd676e0bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:33.354239 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-99651dd3-b662-4e99-8fa9-5a2fd676e0bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:33.386589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-99651dd3-b662-4e99-8fa9-5a2fd676e0bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048229 Aug 28 01:13:33.387339 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 605/1439] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:33 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:34.400951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-dfabccca-f458-4c7a-9de7-8189381393bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:34.404636 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-dfabccca-f458-4c7a-9de7-8189381393bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:34.416769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dfabccca-f458-4c7a-9de7-8189381393bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:34.417672 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dfabccca-f458-4c7a-9de7-8189381393bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:34.418452 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dfabccca-f458-4c7a-9de7-8189381393bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:34.419701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dfabccca-f458-4c7a-9de7-8189381393bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:34.420503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dfabccca-f458-4c7a-9de7-8189381393bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:34.421307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dfabccca-f458-4c7a-9de7-8189381393bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:34.455494 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:13:34.482503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-dfabccca-f458-4c7a-9de7-8189381393bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.086669 Aug 28 01:13:34.483932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 835/1440] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:34 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:35.495244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c259ecc5-bc57-403c-8dd6-5a6e21402f71 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:35.497530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c259ecc5-bc57-403c-8dd6-5a6e21402f71 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:35.504778 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c259ecc5-bc57-403c-8dd6-5a6e21402f71 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:35.505157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c259ecc5-bc57-403c-8dd6-5a6e21402f71 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:35.505592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c259ecc5-bc57-403c-8dd6-5a6e21402f71 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:35.506301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c259ecc5-bc57-403c-8dd6-5a6e21402f71 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:35.506736 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c259ecc5-bc57-403c-8dd6-5a6e21402f71 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:35.507156 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c259ecc5-bc57-403c-8dd6-5a6e21402f71 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:35.538787 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c259ecc5-bc57-403c-8dd6-5a6e21402f71 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047415 Aug 28 01:13:35.539947 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 606/1441] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:35 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:36.551953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-efb421d4-a50d-4194-9694-680e0dbc8068 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:36.554156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-efb421d4-a50d-4194-9694-680e0dbc8068 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:36.560740 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-efb421d4-a50d-4194-9694-680e0dbc8068 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:36.561244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-efb421d4-a50d-4194-9694-680e0dbc8068 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:36.561675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-efb421d4-a50d-4194-9694-680e0dbc8068 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:36.562388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-efb421d4-a50d-4194-9694-680e0dbc8068 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:36.562831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-efb421d4-a50d-4194-9694-680e0dbc8068 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:36.563221 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-efb421d4-a50d-4194-9694-680e0dbc8068 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:36.595698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-efb421d4-a50d-4194-9694-680e0dbc8068 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047800 Aug 28 01:13:36.596272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 836/1442] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:36 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:37.608828 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-96e113c3-3c4c-476d-84a5-467aad9d61b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:37.611024 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-96e113c3-3c4c-476d-84a5-467aad9d61b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:37.617618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96e113c3-3c4c-476d-84a5-467aad9d61b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:37.618112 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96e113c3-3c4c-476d-84a5-467aad9d61b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:37.618546 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96e113c3-3c4c-476d-84a5-467aad9d61b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:37.619248 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96e113c3-3c4c-476d-84a5-467aad9d61b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:37.619697 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96e113c3-3c4c-476d-84a5-467aad9d61b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:37.620118 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96e113c3-3c4c-476d-84a5-467aad9d61b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:37.656554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-96e113c3-3c4c-476d-84a5-467aad9d61b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051772 Aug 28 01:13:37.660445 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 607/1443] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:37 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:38.669373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5d7f8ff4-2cc9-4e39-974e-38cb670d3035 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:38.671665 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5d7f8ff4-2cc9-4e39-974e-38cb670d3035 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:38.679641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d7f8ff4-2cc9-4e39-974e-38cb670d3035 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:38.680121 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d7f8ff4-2cc9-4e39-974e-38cb670d3035 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:38.680546 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d7f8ff4-2cc9-4e39-974e-38cb670d3035 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:38.681290 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d7f8ff4-2cc9-4e39-974e-38cb670d3035 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:38.681733 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d7f8ff4-2cc9-4e39-974e-38cb670d3035 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:38.682146 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d7f8ff4-2cc9-4e39-974e-38cb670d3035 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:38.738200 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5d7f8ff4-2cc9-4e39-974e-38cb670d3035 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.072490 Aug 28 01:13:38.739212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 837/1444] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:38 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:39.753256 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e30a49c7-c7b8-48b3-8877-6d47ab34ec19 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:39.755487 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e30a49c7-c7b8-48b3-8877-6d47ab34ec19 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:39.762447 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e30a49c7-c7b8-48b3-8877-6d47ab34ec19 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:39.762927 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e30a49c7-c7b8-48b3-8877-6d47ab34ec19 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:39.763387 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e30a49c7-c7b8-48b3-8877-6d47ab34ec19 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:39.764086 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e30a49c7-c7b8-48b3-8877-6d47ab34ec19 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:39.764536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e30a49c7-c7b8-48b3-8877-6d47ab34ec19 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:39.765005 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e30a49c7-c7b8-48b3-8877-6d47ab34ec19 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:39.767406 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=99 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:13:39.796174 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e30a49c7-c7b8-48b3-8877-6d47ab34ec19 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047320 Aug 28 01:13:39.796662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 608/1445] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:39 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:40.811976 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2fc9ff07-2803-47b9-ba4e-c1a71f32ac6b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:40.815511 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2fc9ff07-2803-47b9-ba4e-c1a71f32ac6b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:40.826794 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2fc9ff07-2803-47b9-ba4e-c1a71f32ac6b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:40.827611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2fc9ff07-2803-47b9-ba4e-c1a71f32ac6b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:40.828415 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2fc9ff07-2803-47b9-ba4e-c1a71f32ac6b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:40.829199 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2fc9ff07-2803-47b9-ba4e-c1a71f32ac6b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:40.829655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2fc9ff07-2803-47b9-ba4e-c1a71f32ac6b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:40.830118 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2fc9ff07-2803-47b9-ba4e-c1a71f32ac6b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:40.861896 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2fc9ff07-2803-47b9-ba4e-c1a71f32ac6b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056976 Aug 28 01:13:40.862715 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 838/1446] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:40 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:41.876441 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2b02c85c-07f9-4fe4-8930-dbdb6e017397 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:41.878562 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2b02c85c-07f9-4fe4-8930-dbdb6e017397 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:41.885603 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b02c85c-07f9-4fe4-8930-dbdb6e017397 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:41.886022 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b02c85c-07f9-4fe4-8930-dbdb6e017397 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:41.886485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b02c85c-07f9-4fe4-8930-dbdb6e017397 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:41.887056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b02c85c-07f9-4fe4-8930-dbdb6e017397 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:41.887506 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b02c85c-07f9-4fe4-8930-dbdb6e017397 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:41.887930 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b02c85c-07f9-4fe4-8930-dbdb6e017397 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:41.920336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2b02c85c-07f9-4fe4-8930-dbdb6e017397 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049602 Aug 28 01:13:41.920927 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 609/1447] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:41 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:42.932714 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-83d77823-de5a-4a79-bab8-80266183a0b6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:42.935199 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-83d77823-de5a-4a79-bab8-80266183a0b6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:42.942740 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-83d77823-de5a-4a79-bab8-80266183a0b6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:42.943269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-83d77823-de5a-4a79-bab8-80266183a0b6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:42.943694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-83d77823-de5a-4a79-bab8-80266183a0b6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:42.944858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-83d77823-de5a-4a79-bab8-80266183a0b6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:42.945476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-83d77823-de5a-4a79-bab8-80266183a0b6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:42.945754 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-83d77823-de5a-4a79-bab8-80266183a0b6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:42.976759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-83d77823-de5a-4a79-bab8-80266183a0b6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048354 Aug 28 01:13:42.977363 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 839/1448] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:42 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:43.989709 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-eda89333-2e28-4500-917b-ca34a8720f2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:43.991792 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-eda89333-2e28-4500-917b-ca34a8720f2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:43.998836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eda89333-2e28-4500-917b-ca34a8720f2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:43.999312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eda89333-2e28-4500-917b-ca34a8720f2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:43.999744 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eda89333-2e28-4500-917b-ca34a8720f2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:44.000440 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eda89333-2e28-4500-917b-ca34a8720f2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:44.000877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eda89333-2e28-4500-917b-ca34a8720f2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:44.001333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eda89333-2e28-4500-917b-ca34a8720f2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:44.032592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-eda89333-2e28-4500-917b-ca34a8720f2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046951 Aug 28 01:13:44.033483 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 610/1449] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:43 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:45.046155 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-289518e3-984a-4e36-bc69-8a0c16263c57 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:45.048501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-289518e3-984a-4e36-bc69-8a0c16263c57 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:45.056272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-289518e3-984a-4e36-bc69-8a0c16263c57 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:45.056813 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-289518e3-984a-4e36-bc69-8a0c16263c57 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:45.057331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-289518e3-984a-4e36-bc69-8a0c16263c57 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:45.058109 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-289518e3-984a-4e36-bc69-8a0c16263c57 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:45.058613 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-289518e3-984a-4e36-bc69-8a0c16263c57 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:45.059087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-289518e3-984a-4e36-bc69-8a0c16263c57 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:45.095142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-289518e3-984a-4e36-bc69-8a0c16263c57 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053525 Aug 28 01:13:45.095679 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 840/1450] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:45 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:46.107895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-debca0a0-4174-48d4-a21b-2bf07885d47f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:46.110356 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-debca0a0-4174-48d4-a21b-2bf07885d47f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:46.117402 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-debca0a0-4174-48d4-a21b-2bf07885d47f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:46.117891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-debca0a0-4174-48d4-a21b-2bf07885d47f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:46.118324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-debca0a0-4174-48d4-a21b-2bf07885d47f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:46.119035 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-debca0a0-4174-48d4-a21b-2bf07885d47f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:46.119490 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-debca0a0-4174-48d4-a21b-2bf07885d47f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:46.119908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-debca0a0-4174-48d4-a21b-2bf07885d47f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:46.156987 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-debca0a0-4174-48d4-a21b-2bf07885d47f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053074 Aug 28 01:13:46.157578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 611/1451] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:46 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:47.170099 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2c56a8c8-cd42-4abe-8a12-a3a98c777a91 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:47.172299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2c56a8c8-cd42-4abe-8a12-a3a98c777a91 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:47.179282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c56a8c8-cd42-4abe-8a12-a3a98c777a91 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:47.179869 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c56a8c8-cd42-4abe-8a12-a3a98c777a91 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:47.180302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c56a8c8-cd42-4abe-8a12-a3a98c777a91 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:47.180995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c56a8c8-cd42-4abe-8a12-a3a98c777a91 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:47.181474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c56a8c8-cd42-4abe-8a12-a3a98c777a91 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:47.181884 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c56a8c8-cd42-4abe-8a12-a3a98c777a91 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:47.214609 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2c56a8c8-cd42-4abe-8a12-a3a98c777a91 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048259 Aug 28 01:13:47.215293 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 841/1452] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:47 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:48.228370 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-dc87025b-703d-4fb9-a423-e5b8b5bf01c0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:48.230788 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-dc87025b-703d-4fb9-a423-e5b8b5bf01c0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:48.232777 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=30 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:13:48.238156 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dc87025b-703d-4fb9-a423-e5b8b5bf01c0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:48.238647 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dc87025b-703d-4fb9-a423-e5b8b5bf01c0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:48.239088 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dc87025b-703d-4fb9-a423-e5b8b5bf01c0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:48.239792 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dc87025b-703d-4fb9-a423-e5b8b5bf01c0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:48.240238 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dc87025b-703d-4fb9-a423-e5b8b5bf01c0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:48.240662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dc87025b-703d-4fb9-a423-e5b8b5bf01c0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:48.270792 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-dc87025b-703d-4fb9-a423-e5b8b5bf01c0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047090 Aug 28 01:13:48.271368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 612/1453] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:48 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:49.284208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-92ec7427-869e-463f-a02b-19029949df09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:49.286328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-92ec7427-869e-463f-a02b-19029949df09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:49.289239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=30 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:13:49.295496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92ec7427-869e-463f-a02b-19029949df09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:49.296065 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92ec7427-869e-463f-a02b-19029949df09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:49.296498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92ec7427-869e-463f-a02b-19029949df09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:49.297276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92ec7427-869e-463f-a02b-19029949df09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:49.297776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92ec7427-869e-463f-a02b-19029949df09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:49.298203 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92ec7427-869e-463f-a02b-19029949df09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:49.332687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-92ec7427-869e-463f-a02b-19029949df09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052780 Aug 28 01:13:49.333621 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 842/1454] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:49 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:50.350296 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-eaf59fbd-98bf-4e32-849f-2b9a07666b6a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:50.353186 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-eaf59fbd-98bf-4e32-849f-2b9a07666b6a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:50.362597 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eaf59fbd-98bf-4e32-849f-2b9a07666b6a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:50.363084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eaf59fbd-98bf-4e32-849f-2b9a07666b6a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:50.363516 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eaf59fbd-98bf-4e32-849f-2b9a07666b6a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:50.364218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eaf59fbd-98bf-4e32-849f-2b9a07666b6a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:50.364673 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eaf59fbd-98bf-4e32-849f-2b9a07666b6a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:50.365100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eaf59fbd-98bf-4e32-849f-2b9a07666b6a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:50.398869 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-eaf59fbd-98bf-4e32-849f-2b9a07666b6a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054708 Aug 28 01:13:50.399422 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 613/1455] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:50 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:51.414497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-13672572-280b-40ee-a382-9db30832396c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:51.417517 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-13672572-280b-40ee-a382-9db30832396c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:51.425683 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-13672572-280b-40ee-a382-9db30832396c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:51.426168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-13672572-280b-40ee-a382-9db30832396c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:51.426603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-13672572-280b-40ee-a382-9db30832396c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:51.427301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-13672572-280b-40ee-a382-9db30832396c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:51.427751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-13672572-280b-40ee-a382-9db30832396c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:51.428170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-13672572-280b-40ee-a382-9db30832396c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:51.458738 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-13672572-280b-40ee-a382-9db30832396c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050012 Aug 28 01:13:51.459293 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 843/1456] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:51 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:52.472019 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1b977186-4a64-4837-aaf7-a4e6b3ccb6b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:52.474223 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1b977186-4a64-4837-aaf7-a4e6b3ccb6b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:52.481202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b977186-4a64-4837-aaf7-a4e6b3ccb6b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:52.481710 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b977186-4a64-4837-aaf7-a4e6b3ccb6b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:52.482146 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b977186-4a64-4837-aaf7-a4e6b3ccb6b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:52.482857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b977186-4a64-4837-aaf7-a4e6b3ccb6b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:52.483311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b977186-4a64-4837-aaf7-a4e6b3ccb6b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:52.483740 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b977186-4a64-4837-aaf7-a4e6b3ccb6b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:52.514146 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1b977186-4a64-4837-aaf7-a4e6b3ccb6b8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046201 Aug 28 01:13:52.514705 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 614/1457] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:52 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:53.526158 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-04126384-3cd6-4766-bdaa-916f72575342 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:53.528620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-04126384-3cd6-4766-bdaa-916f72575342 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:53.535558 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-04126384-3cd6-4766-bdaa-916f72575342 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:53.536039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-04126384-3cd6-4766-bdaa-916f72575342 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:53.536529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-04126384-3cd6-4766-bdaa-916f72575342 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:53.537258 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-04126384-3cd6-4766-bdaa-916f72575342 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:53.537706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-04126384-3cd6-4766-bdaa-916f72575342 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:53.538126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-04126384-3cd6-4766-bdaa-916f72575342 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:53.570491 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-04126384-3cd6-4766-bdaa-916f72575342 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048398 Aug 28 01:13:53.571057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 844/1458] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:53 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:54.585627 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a64ab8fd-929c-4a57-92cc-97dc6ac5c066 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:54.587933 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a64ab8fd-929c-4a57-92cc-97dc6ac5c066 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:54.595234 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a64ab8fd-929c-4a57-92cc-97dc6ac5c066 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:54.595856 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a64ab8fd-929c-4a57-92cc-97dc6ac5c066 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:54.596291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a64ab8fd-929c-4a57-92cc-97dc6ac5c066 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:54.597700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a64ab8fd-929c-4a57-92cc-97dc6ac5c066 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:54.598153 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a64ab8fd-929c-4a57-92cc-97dc6ac5c066 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:54.598627 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a64ab8fd-929c-4a57-92cc-97dc6ac5c066 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:54.628272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a64ab8fd-929c-4a57-92cc-97dc6ac5c066 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047403 Aug 28 01:13:54.628802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 615/1459] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:54 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:55.641105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8da6f0ec-2f4b-4558-9b34-f6522ad691a6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:55.643424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8da6f0ec-2f4b-4558-9b34-f6522ad691a6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:55.652593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8da6f0ec-2f4b-4558-9b34-f6522ad691a6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:55.653114 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8da6f0ec-2f4b-4558-9b34-f6522ad691a6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:55.653577 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8da6f0ec-2f4b-4558-9b34-f6522ad691a6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:55.654278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8da6f0ec-2f4b-4558-9b34-f6522ad691a6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:55.654722 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8da6f0ec-2f4b-4558-9b34-f6522ad691a6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:55.655141 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8da6f0ec-2f4b-4558-9b34-f6522ad691a6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:55.684825 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8da6f0ec-2f4b-4558-9b34-f6522ad691a6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047853 Aug 28 01:13:55.685442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 845/1460] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:55 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:56.700128 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b2b0bd3c-1900-4250-a2df-0969b5ba359a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:56.702442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b2b0bd3c-1900-4250-a2df-0969b5ba359a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:56.709538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b2b0bd3c-1900-4250-a2df-0969b5ba359a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:56.709998 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b2b0bd3c-1900-4250-a2df-0969b5ba359a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:56.710440 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b2b0bd3c-1900-4250-a2df-0969b5ba359a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:56.711135 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b2b0bd3c-1900-4250-a2df-0969b5ba359a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:56.711578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b2b0bd3c-1900-4250-a2df-0969b5ba359a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:56.711972 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b2b0bd3c-1900-4250-a2df-0969b5ba359a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:56.745706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b2b0bd3c-1900-4250-a2df-0969b5ba359a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049518 Aug 28 01:13:56.746291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 616/1461] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:56 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:57.759299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0a19d107-1ba5-4849-937e-c4fd3a4db2b2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:57.761428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0a19d107-1ba5-4849-937e-c4fd3a4db2b2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:57.768262 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a19d107-1ba5-4849-937e-c4fd3a4db2b2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:57.768743 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a19d107-1ba5-4849-937e-c4fd3a4db2b2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:57.769213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a19d107-1ba5-4849-937e-c4fd3a4db2b2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:57.769905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a19d107-1ba5-4849-937e-c4fd3a4db2b2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:57.770337 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a19d107-1ba5-4849-937e-c4fd3a4db2b2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:57.770752 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a19d107-1ba5-4849-937e-c4fd3a4db2b2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:57.801781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0a19d107-1ba5-4849-937e-c4fd3a4db2b2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046595 Aug 28 01:13:57.802350 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 846/1462] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:57 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:58.815624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-236712d0-a9af-4482-a317-225abcf5d2b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:58.818282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-236712d0-a9af-4482-a317-225abcf5d2b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:58.825853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-236712d0-a9af-4482-a317-225abcf5d2b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:58.826341 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-236712d0-a9af-4482-a317-225abcf5d2b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:58.826768 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-236712d0-a9af-4482-a317-225abcf5d2b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:58.827469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-236712d0-a9af-4482-a317-225abcf5d2b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:58.827916 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-236712d0-a9af-4482-a317-225abcf5d2b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:58.828428 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-236712d0-a9af-4482-a317-225abcf5d2b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:58.858779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-236712d0-a9af-4482-a317-225abcf5d2b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048250 Aug 28 01:13:58.859342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 617/1463] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:58 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:13:59.873908 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-858d0436-e128-46ab-bf58-858dcb730b3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:13:59.876083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-858d0436-e128-46ab-bf58-858dcb730b3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:13:59.883792 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-858d0436-e128-46ab-bf58-858dcb730b3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:59.884269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-858d0436-e128-46ab-bf58-858dcb730b3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:59.884675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-858d0436-e128-46ab-bf58-858dcb730b3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:59.885416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-858d0436-e128-46ab-bf58-858dcb730b3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:13:59.885856 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-858d0436-e128-46ab-bf58-858dcb730b3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:13:59.886270 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-858d0436-e128-46ab-bf58-858dcb730b3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:13:59.922549 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-858d0436-e128-46ab-bf58-858dcb730b3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052268 Aug 28 01:13:59.923188 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 847/1464] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:13:59 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:00.935908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0039033c-3bba-4159-b58b-d2b27afd6d18 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:00.938265 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0039033c-3bba-4159-b58b-d2b27afd6d18 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:00.946209 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0039033c-3bba-4159-b58b-d2b27afd6d18 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:00.946719 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0039033c-3bba-4159-b58b-d2b27afd6d18 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:00.947148 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0039033c-3bba-4159-b58b-d2b27afd6d18 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:00.947840 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0039033c-3bba-4159-b58b-d2b27afd6d18 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:00.948362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0039033c-3bba-4159-b58b-d2b27afd6d18 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:00.948781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0039033c-3bba-4159-b58b-d2b27afd6d18 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:00.980293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0039033c-3bba-4159-b58b-d2b27afd6d18 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048540 Aug 28 01:14:00.980813 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 618/1465] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:00 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:01.995785 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-92e7d8a6-0da2-4a50-8acb-a0fbf2a582c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:01.998864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-92e7d8a6-0da2-4a50-8acb-a0fbf2a582c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:02.007386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92e7d8a6-0da2-4a50-8acb-a0fbf2a582c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:02.007864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92e7d8a6-0da2-4a50-8acb-a0fbf2a582c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:02.008288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92e7d8a6-0da2-4a50-8acb-a0fbf2a582c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:02.009010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92e7d8a6-0da2-4a50-8acb-a0fbf2a582c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:02.009446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92e7d8a6-0da2-4a50-8acb-a0fbf2a582c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:02.009874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92e7d8a6-0da2-4a50-8acb-a0fbf2a582c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:02.039381 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-92e7d8a6-0da2-4a50-8acb-a0fbf2a582c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050174 Aug 28 01:14:02.039927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 848/1466] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:01 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:03.053804 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6216cf3a-b3b7-496c-8ee9-f485dd682ab7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:03.056107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6216cf3a-b3b7-496c-8ee9-f485dd682ab7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:03.062876 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6216cf3a-b3b7-496c-8ee9-f485dd682ab7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:03.063361 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6216cf3a-b3b7-496c-8ee9-f485dd682ab7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:03.063794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6216cf3a-b3b7-496c-8ee9-f485dd682ab7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:03.064495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6216cf3a-b3b7-496c-8ee9-f485dd682ab7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:03.064944 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6216cf3a-b3b7-496c-8ee9-f485dd682ab7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:03.065410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6216cf3a-b3b7-496c-8ee9-f485dd682ab7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:03.096981 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6216cf3a-b3b7-496c-8ee9-f485dd682ab7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048048 Aug 28 01:14:03.097537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 619/1467] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:03 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:04.111170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5126eff4-349e-4529-a9a2-e32e748d0023 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:04.114106 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5126eff4-349e-4529-a9a2-e32e748d0023 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:04.122488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5126eff4-349e-4529-a9a2-e32e748d0023 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:04.122859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5126eff4-349e-4529-a9a2-e32e748d0023 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:04.123301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5126eff4-349e-4529-a9a2-e32e748d0023 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:04.124459 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5126eff4-349e-4529-a9a2-e32e748d0023 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:04.124459 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5126eff4-349e-4529-a9a2-e32e748d0023 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:04.124841 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5126eff4-349e-4529-a9a2-e32e748d0023 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:04.156677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5126eff4-349e-4529-a9a2-e32e748d0023 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050401 Aug 28 01:14:04.157256 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 849/1468] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:04 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:05.171493 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5411847e-37fb-4de0-9611-d5c63e52e04d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:05.174537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5411847e-37fb-4de0-9611-d5c63e52e04d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:05.181533 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5411847e-37fb-4de0-9611-d5c63e52e04d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:05.181942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5411847e-37fb-4de0-9611-d5c63e52e04d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:05.182367 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5411847e-37fb-4de0-9611-d5c63e52e04d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:05.183063 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5411847e-37fb-4de0-9611-d5c63e52e04d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:05.183508 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5411847e-37fb-4de0-9611-d5c63e52e04d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:05.183923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5411847e-37fb-4de0-9611-d5c63e52e04d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:05.214616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5411847e-37fb-4de0-9611-d5c63e52e04d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048175 Aug 28 01:14:05.215192 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 620/1469] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:05 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:06.227711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-526de78b-ecc1-4fe6-871d-9cf5d703df47 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:06.229924 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-526de78b-ecc1-4fe6-871d-9cf5d703df47 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:06.236718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-526de78b-ecc1-4fe6-871d-9cf5d703df47 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:06.237234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-526de78b-ecc1-4fe6-871d-9cf5d703df47 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:06.237664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-526de78b-ecc1-4fe6-871d-9cf5d703df47 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:06.238357 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-526de78b-ecc1-4fe6-871d-9cf5d703df47 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:06.238874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-526de78b-ecc1-4fe6-871d-9cf5d703df47 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:06.239295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-526de78b-ecc1-4fe6-871d-9cf5d703df47 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:06.241686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=87 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:14:06.270049 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-526de78b-ecc1-4fe6-871d-9cf5d703df47 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046420 Aug 28 01:14:06.270501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 850/1470] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:06 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:07.286344 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d89615da-8a65-4c3f-8314-c6050b1d15fa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:07.289633 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d89615da-8a65-4c3f-8314-c6050b1d15fa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:07.300369 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d89615da-8a65-4c3f-8314-c6050b1d15fa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:07.301197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d89615da-8a65-4c3f-8314-c6050b1d15fa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:07.301782 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d89615da-8a65-4c3f-8314-c6050b1d15fa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:07.302473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d89615da-8a65-4c3f-8314-c6050b1d15fa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:07.302983 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d89615da-8a65-4c3f-8314-c6050b1d15fa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:07.303335 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d89615da-8a65-4c3f-8314-c6050b1d15fa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:07.334108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d89615da-8a65-4c3f-8314-c6050b1d15fa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054573 Aug 28 01:14:07.334649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 621/1471] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:07 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:08.346913 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-63822831-ba26-4335-8200-623a045e31fb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:08.349224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-63822831-ba26-4335-8200-623a045e31fb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:08.356410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63822831-ba26-4335-8200-623a045e31fb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:08.356896 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63822831-ba26-4335-8200-623a045e31fb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:08.357373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63822831-ba26-4335-8200-623a045e31fb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:08.358084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63822831-ba26-4335-8200-623a045e31fb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:08.358536 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63822831-ba26-4335-8200-623a045e31fb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:08.358954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-63822831-ba26-4335-8200-623a045e31fb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:08.391073 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-63822831-ba26-4335-8200-623a045e31fb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048845 Aug 28 01:14:08.391639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 851/1472] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:08 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:09.406271 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-dab680ad-64ec-4064-98c4-dd7c122690a7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:09.409466 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-dab680ad-64ec-4064-98c4-dd7c122690a7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:09.415838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dab680ad-64ec-4064-98c4-dd7c122690a7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:09.416321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dab680ad-64ec-4064-98c4-dd7c122690a7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:09.416733 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dab680ad-64ec-4064-98c4-dd7c122690a7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:09.417579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dab680ad-64ec-4064-98c4-dd7c122690a7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:09.418038 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dab680ad-64ec-4064-98c4-dd7c122690a7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:09.418447 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dab680ad-64ec-4064-98c4-dd7c122690a7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:09.450539 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-dab680ad-64ec-4064-98c4-dd7c122690a7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048285 Aug 28 01:14:09.450995 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 622/1473] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:09 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:10.462630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c20aea09-22dc-452e-9f9e-f935dd6cd8e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:10.465682 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c20aea09-22dc-452e-9f9e-f935dd6cd8e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:10.474050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c20aea09-22dc-452e-9f9e-f935dd6cd8e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:10.474467 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c20aea09-22dc-452e-9f9e-f935dd6cd8e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:10.474944 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c20aea09-22dc-452e-9f9e-f935dd6cd8e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:10.475709 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c20aea09-22dc-452e-9f9e-f935dd6cd8e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:10.476189 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c20aea09-22dc-452e-9f9e-f935dd6cd8e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:10.476643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c20aea09-22dc-452e-9f9e-f935dd6cd8e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:10.515445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c20aea09-22dc-452e-9f9e-f935dd6cd8e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056583 Aug 28 01:14:10.516250 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 852/1474] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:10 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:11.531004 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bec106f0-9519-4754-8941-63209e6fe60e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:11.533416 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bec106f0-9519-4754-8941-63209e6fe60e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:11.540774 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bec106f0-9519-4754-8941-63209e6fe60e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:11.541287 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bec106f0-9519-4754-8941-63209e6fe60e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:11.541722 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bec106f0-9519-4754-8941-63209e6fe60e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:11.542418 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bec106f0-9519-4754-8941-63209e6fe60e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:11.542866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bec106f0-9519-4754-8941-63209e6fe60e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:11.543307 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bec106f0-9519-4754-8941-63209e6fe60e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:11.545831 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=90 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:14:11.575588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bec106f0-9519-4754-8941-63209e6fe60e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049032 Aug 28 01:14:11.576054 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 623/1475] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:11 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:12.588885 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ec5a9204-eb74-47d5-9eb5-5cbe0c61ab40 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:12.591293 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ec5a9204-eb74-47d5-9eb5-5cbe0c61ab40 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:12.597849 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec5a9204-eb74-47d5-9eb5-5cbe0c61ab40 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:12.598331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec5a9204-eb74-47d5-9eb5-5cbe0c61ab40 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:12.598755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec5a9204-eb74-47d5-9eb5-5cbe0c61ab40 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:12.599445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec5a9204-eb74-47d5-9eb5-5cbe0c61ab40 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:12.599892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec5a9204-eb74-47d5-9eb5-5cbe0c61ab40 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:12.600307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ec5a9204-eb74-47d5-9eb5-5cbe0c61ab40 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:12.635539 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ec5a9204-eb74-47d5-9eb5-5cbe0c61ab40 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050911 Aug 28 01:14:12.636293 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 853/1476] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:12 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:13.652105 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fc1c66f5-0980-4e2c-831e-188d465eb869 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:13.655465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fc1c66f5-0980-4e2c-831e-188d465eb869 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:13.664244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc1c66f5-0980-4e2c-831e-188d465eb869 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:13.664721 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc1c66f5-0980-4e2c-831e-188d465eb869 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:13.665187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc1c66f5-0980-4e2c-831e-188d465eb869 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:13.665875 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc1c66f5-0980-4e2c-831e-188d465eb869 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:13.666320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc1c66f5-0980-4e2c-831e-188d465eb869 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:13.666736 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc1c66f5-0980-4e2c-831e-188d465eb869 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:13.696720 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fc1c66f5-0980-4e2c-831e-188d465eb869 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050795 Aug 28 01:14:13.697312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 624/1477] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:13 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:14.711699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3f21fb7a-4dde-4709-bac1-9fdd52a56c44 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:14.713914 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3f21fb7a-4dde-4709-bac1-9fdd52a56c44 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:14.720641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3f21fb7a-4dde-4709-bac1-9fdd52a56c44 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:14.721152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3f21fb7a-4dde-4709-bac1-9fdd52a56c44 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:14.721581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3f21fb7a-4dde-4709-bac1-9fdd52a56c44 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:14.722271 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3f21fb7a-4dde-4709-bac1-9fdd52a56c44 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:14.722713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3f21fb7a-4dde-4709-bac1-9fdd52a56c44 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:14.723126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3f21fb7a-4dde-4709-bac1-9fdd52a56c44 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:14.754713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3f21fb7a-4dde-4709-bac1-9fdd52a56c44 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047841 Aug 28 01:14:14.755187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 854/1478] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:14 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:15.767679 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-042056b2-6f87-4943-a6f7-8d6db696d432 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:15.769890 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-042056b2-6f87-4943-a6f7-8d6db696d432 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:15.776660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-042056b2-6f87-4943-a6f7-8d6db696d432 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:15.777191 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-042056b2-6f87-4943-a6f7-8d6db696d432 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:15.777610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-042056b2-6f87-4943-a6f7-8d6db696d432 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:15.778329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-042056b2-6f87-4943-a6f7-8d6db696d432 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:15.778779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-042056b2-6f87-4943-a6f7-8d6db696d432 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:15.779199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-042056b2-6f87-4943-a6f7-8d6db696d432 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:15.811707 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-042056b2-6f87-4943-a6f7-8d6db696d432 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048046 Aug 28 01:14:15.812199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 625/1479] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:15 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:16.824784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-64a160b8-b029-4b8a-93c3-79efe943a225 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:16.826976 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-64a160b8-b029-4b8a-93c3-79efe943a225 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:16.833818 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64a160b8-b029-4b8a-93c3-79efe943a225 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:16.834327 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64a160b8-b029-4b8a-93c3-79efe943a225 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:16.834761 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64a160b8-b029-4b8a-93c3-79efe943a225 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:16.835465 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64a160b8-b029-4b8a-93c3-79efe943a225 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:16.835911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64a160b8-b029-4b8a-93c3-79efe943a225 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:16.836334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64a160b8-b029-4b8a-93c3-79efe943a225 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:16.866813 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-64a160b8-b029-4b8a-93c3-79efe943a225 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045957 Aug 28 01:14:16.867405 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 855/1480] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:16 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:17.879752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2b5e74ce-3bdb-4bfb-8d31-feeaf0d79b34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:17.881950 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2b5e74ce-3bdb-4bfb-8d31-feeaf0d79b34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:17.889114 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b5e74ce-3bdb-4bfb-8d31-feeaf0d79b34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:17.889619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b5e74ce-3bdb-4bfb-8d31-feeaf0d79b34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:17.890047 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b5e74ce-3bdb-4bfb-8d31-feeaf0d79b34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:17.890738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b5e74ce-3bdb-4bfb-8d31-feeaf0d79b34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:17.891183 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b5e74ce-3bdb-4bfb-8d31-feeaf0d79b34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:17.891600 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b5e74ce-3bdb-4bfb-8d31-feeaf0d79b34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:17.921817 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2b5e74ce-3bdb-4bfb-8d31-feeaf0d79b34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046043 Aug 28 01:14:17.922366 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 626/1481] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:17 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:18.935377 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-78fbab2f-12a6-4681-95d1-b4376cb9ce61 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:18.937560 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-78fbab2f-12a6-4681-95d1-b4376cb9ce61 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:18.944538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-78fbab2f-12a6-4681-95d1-b4376cb9ce61 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:18.945018 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-78fbab2f-12a6-4681-95d1-b4376cb9ce61 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:18.945473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-78fbab2f-12a6-4681-95d1-b4376cb9ce61 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:18.946183 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-78fbab2f-12a6-4681-95d1-b4376cb9ce61 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:18.946627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-78fbab2f-12a6-4681-95d1-b4376cb9ce61 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:18.947040 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-78fbab2f-12a6-4681-95d1-b4376cb9ce61 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:18.977736 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-78fbab2f-12a6-4681-95d1-b4376cb9ce61 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046367 Aug 28 01:14:18.978293 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 856/1482] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:18 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:19.994492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2e0dbf98-a29e-4646-b83f-b2e24432f945 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:19.996653 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2e0dbf98-a29e-4646-b83f-b2e24432f945 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:19.999022 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=30 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:14:20.004523 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e0dbf98-a29e-4646-b83f-b2e24432f945 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:20.004990 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e0dbf98-a29e-4646-b83f-b2e24432f945 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:20.005495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e0dbf98-a29e-4646-b83f-b2e24432f945 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:20.006189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e0dbf98-a29e-4646-b83f-b2e24432f945 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:20.006713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e0dbf98-a29e-4646-b83f-b2e24432f945 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:20.007149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2e0dbf98-a29e-4646-b83f-b2e24432f945 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:20.036923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2e0dbf98-a29e-4646-b83f-b2e24432f945 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047716 Aug 28 01:14:20.037560 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 627/1483] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:19 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:21.048885 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5847f336-8c24-4a3b-8746-a620b8c3f1a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:21.051113 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5847f336-8c24-4a3b-8746-a620b8c3f1a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:21.054157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=30 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:14:21.059638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5847f336-8c24-4a3b-8746-a620b8c3f1a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:21.060118 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5847f336-8c24-4a3b-8746-a620b8c3f1a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:21.060543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5847f336-8c24-4a3b-8746-a620b8c3f1a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:21.061263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5847f336-8c24-4a3b-8746-a620b8c3f1a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:21.061707 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5847f336-8c24-4a3b-8746-a620b8c3f1a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:21.062119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5847f336-8c24-4a3b-8746-a620b8c3f1a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:21.093145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5847f336-8c24-4a3b-8746-a620b8c3f1a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048142 Aug 28 01:14:21.093705 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 857/1484] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:21 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:22.107308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-43adf453-aedb-420e-af15-833ce32dadc0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:22.109620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-43adf453-aedb-420e-af15-833ce32dadc0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:22.117539 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43adf453-aedb-420e-af15-833ce32dadc0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:22.118034 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43adf453-aedb-420e-af15-833ce32dadc0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:22.118471 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43adf453-aedb-420e-af15-833ce32dadc0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:22.119184 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43adf453-aedb-420e-af15-833ce32dadc0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:22.119633 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43adf453-aedb-420e-af15-833ce32dadc0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:22.120053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43adf453-aedb-420e-af15-833ce32dadc0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:22.152127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-43adf453-aedb-420e-af15-833ce32dadc0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050050 Aug 28 01:14:22.152687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 628/1485] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:22 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:23.164757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a6b6acf5-deaf-4c50-9071-78487bb06280 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:23.166953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a6b6acf5-deaf-4c50-9071-78487bb06280 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:23.173886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6b6acf5-deaf-4c50-9071-78487bb06280 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:23.174348 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6b6acf5-deaf-4c50-9071-78487bb06280 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:23.174789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6b6acf5-deaf-4c50-9071-78487bb06280 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:23.175479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6b6acf5-deaf-4c50-9071-78487bb06280 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:23.175919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6b6acf5-deaf-4c50-9071-78487bb06280 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:23.176336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6b6acf5-deaf-4c50-9071-78487bb06280 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:23.207191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a6b6acf5-deaf-4c50-9071-78487bb06280 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046470 Aug 28 01:14:23.207739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 858/1486] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:23 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:24.222449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d80322e6-92e3-4528-98db-5b05fb9907cf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:24.225491 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d80322e6-92e3-4528-98db-5b05fb9907cf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:24.231580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d80322e6-92e3-4528-98db-5b05fb9907cf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:24.232056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d80322e6-92e3-4528-98db-5b05fb9907cf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:24.232483 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d80322e6-92e3-4528-98db-5b05fb9907cf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:24.233526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d80322e6-92e3-4528-98db-5b05fb9907cf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:24.233664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d80322e6-92e3-4528-98db-5b05fb9907cf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:24.234040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d80322e6-92e3-4528-98db-5b05fb9907cf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:24.263617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d80322e6-92e3-4528-98db-5b05fb9907cf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047486 Aug 28 01:14:24.264178 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 629/1487] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:24 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:25.278822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-24c27138-f15a-4794-9ac8-087240c3052c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:25.281333 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-24c27138-f15a-4794-9ac8-087240c3052c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:25.288712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24c27138-f15a-4794-9ac8-087240c3052c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:25.289232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24c27138-f15a-4794-9ac8-087240c3052c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:25.289660 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24c27138-f15a-4794-9ac8-087240c3052c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:25.290358 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24c27138-f15a-4794-9ac8-087240c3052c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:25.290806 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24c27138-f15a-4794-9ac8-087240c3052c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:25.291273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24c27138-f15a-4794-9ac8-087240c3052c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:25.321861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-24c27138-f15a-4794-9ac8-087240c3052c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046946 Aug 28 01:14:25.322450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 859/1488] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:25 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:26.334338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c8500b9c-b5ae-4e1b-bfa7-9dcd171b5ff8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:26.336591 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c8500b9c-b5ae-4e1b-bfa7-9dcd171b5ff8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:26.343991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8500b9c-b5ae-4e1b-bfa7-9dcd171b5ff8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:26.344479 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8500b9c-b5ae-4e1b-bfa7-9dcd171b5ff8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:26.344911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8500b9c-b5ae-4e1b-bfa7-9dcd171b5ff8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:26.345665 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8500b9c-b5ae-4e1b-bfa7-9dcd171b5ff8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:26.346139 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8500b9c-b5ae-4e1b-bfa7-9dcd171b5ff8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:26.346556 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8500b9c-b5ae-4e1b-bfa7-9dcd171b5ff8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:26.378584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c8500b9c-b5ae-4e1b-bfa7-9dcd171b5ff8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048188 Aug 28 01:14:26.379159 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 630/1489] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:26 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:27.391108 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3e7439cc-0670-4057-b28d-3bedbb2073c8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:27.393309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3e7439cc-0670-4057-b28d-3bedbb2073c8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:27.399875 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e7439cc-0670-4057-b28d-3bedbb2073c8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:27.400347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e7439cc-0670-4057-b28d-3bedbb2073c8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:27.400773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e7439cc-0670-4057-b28d-3bedbb2073c8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:27.401496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e7439cc-0670-4057-b28d-3bedbb2073c8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:27.401915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e7439cc-0670-4057-b28d-3bedbb2073c8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:27.402403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e7439cc-0670-4057-b28d-3bedbb2073c8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:27.431860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3e7439cc-0670-4057-b28d-3bedbb2073c8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044732 Aug 28 01:14:27.432406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 860/1490] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:27 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:28.446213 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a30fcd7b-0bc2-4caa-83d5-897741b28720 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:28.448406 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a30fcd7b-0bc2-4caa-83d5-897741b28720 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:28.455724 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a30fcd7b-0bc2-4caa-83d5-897741b28720 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:28.456229 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a30fcd7b-0bc2-4caa-83d5-897741b28720 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:28.456654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a30fcd7b-0bc2-4caa-83d5-897741b28720 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:28.457381 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a30fcd7b-0bc2-4caa-83d5-897741b28720 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:28.457818 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a30fcd7b-0bc2-4caa-83d5-897741b28720 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:28.458233 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a30fcd7b-0bc2-4caa-83d5-897741b28720 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:28.488468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a30fcd7b-0bc2-4caa-83d5-897741b28720 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047469 Aug 28 01:14:28.489060 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 631/1491] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:28 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:29.502347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8f81678d-8c64-4bfa-937a-983f656c2258 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:29.504278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8f81678d-8c64-4bfa-937a-983f656c2258 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:29.511664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f81678d-8c64-4bfa-937a-983f656c2258 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:29.512149 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f81678d-8c64-4bfa-937a-983f656c2258 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:29.512584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f81678d-8c64-4bfa-937a-983f656c2258 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:29.513335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f81678d-8c64-4bfa-937a-983f656c2258 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:29.513778 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f81678d-8c64-4bfa-937a-983f656c2258 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:29.514194 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8f81678d-8c64-4bfa-937a-983f656c2258 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:29.544350 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8f81678d-8c64-4bfa-937a-983f656c2258 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046754 Aug 28 01:14:29.544980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 861/1492] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:29 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:30.557705 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c04b53f7-e257-4f20-b5c8-4357669797c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:30.559887 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c04b53f7-e257-4f20-b5c8-4357669797c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:30.566723 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c04b53f7-e257-4f20-b5c8-4357669797c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:30.571995 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c04b53f7-e257-4f20-b5c8-4357669797c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:30.571995 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c04b53f7-e257-4f20-b5c8-4357669797c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:30.571995 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c04b53f7-e257-4f20-b5c8-4357669797c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:30.571995 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c04b53f7-e257-4f20-b5c8-4357669797c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:30.573032 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c04b53f7-e257-4f20-b5c8-4357669797c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:30.601489 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c04b53f7-e257-4f20-b5c8-4357669797c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048215 Aug 28 01:14:30.602129 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 632/1493] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:30 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:31.615033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-54562d56-6650-4fb6-b2a8-c6e99ed3fef4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:31.617275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-54562d56-6650-4fb6-b2a8-c6e99ed3fef4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:31.625006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54562d56-6650-4fb6-b2a8-c6e99ed3fef4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:31.625489 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54562d56-6650-4fb6-b2a8-c6e99ed3fef4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:31.626079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54562d56-6650-4fb6-b2a8-c6e99ed3fef4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:31.626593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54562d56-6650-4fb6-b2a8-c6e99ed3fef4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:31.627032 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54562d56-6650-4fb6-b2a8-c6e99ed3fef4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:31.627444 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54562d56-6650-4fb6-b2a8-c6e99ed3fef4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:31.657003 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-54562d56-6650-4fb6-b2a8-c6e99ed3fef4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045921 Aug 28 01:14:31.657566 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 862/1494] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:31 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:32.671939 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0ccc6822-736f-42f2-a72c-7b3d74c47e5a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:32.674143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0ccc6822-736f-42f2-a72c-7b3d74c47e5a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:32.681997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0ccc6822-736f-42f2-a72c-7b3d74c47e5a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:32.682495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0ccc6822-736f-42f2-a72c-7b3d74c47e5a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:32.682927 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0ccc6822-736f-42f2-a72c-7b3d74c47e5a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:32.683637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0ccc6822-736f-42f2-a72c-7b3d74c47e5a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:32.684092 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0ccc6822-736f-42f2-a72c-7b3d74c47e5a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:32.684514 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0ccc6822-736f-42f2-a72c-7b3d74c47e5a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:32.721980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0ccc6822-736f-42f2-a72c-7b3d74c47e5a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053960 Aug 28 01:14:32.722561 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 633/1495] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:32 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:33.740193 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f5898f31-216b-4847-97f2-8faf5386a3f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:33.744164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f5898f31-216b-4847-97f2-8faf5386a3f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:33.752336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5898f31-216b-4847-97f2-8faf5386a3f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:33.752829 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5898f31-216b-4847-97f2-8faf5386a3f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:33.753364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5898f31-216b-4847-97f2-8faf5386a3f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:33.754081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5898f31-216b-4847-97f2-8faf5386a3f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:33.754533 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5898f31-216b-4847-97f2-8faf5386a3f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:33.754958 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5898f31-216b-4847-97f2-8faf5386a3f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:33.788240 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f5898f31-216b-4847-97f2-8faf5386a3f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056081 Aug 28 01:14:33.788833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 863/1496] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:33 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:34.800175 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b4b5f2cb-a0dc-4624-9a4d-c333bf6aef69 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:34.802433 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b4b5f2cb-a0dc-4624-9a4d-c333bf6aef69 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:34.809400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4b5f2cb-a0dc-4624-9a4d-c333bf6aef69 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:34.809923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4b5f2cb-a0dc-4624-9a4d-c333bf6aef69 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:34.810359 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4b5f2cb-a0dc-4624-9a4d-c333bf6aef69 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:34.811063 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4b5f2cb-a0dc-4624-9a4d-c333bf6aef69 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:34.811513 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4b5f2cb-a0dc-4624-9a4d-c333bf6aef69 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:34.811902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4b5f2cb-a0dc-4624-9a4d-c333bf6aef69 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:34.842962 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b4b5f2cb-a0dc-4624-9a4d-c333bf6aef69 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046768 Aug 28 01:14:34.845293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 634/1497] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:34 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:35.855506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c55533a4-fc09-4c82-b912-7fee450d06e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:35.857740 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c55533a4-fc09-4c82-b912-7fee450d06e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:35.864497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c55533a4-fc09-4c82-b912-7fee450d06e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:35.865067 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c55533a4-fc09-4c82-b912-7fee450d06e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:35.865447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c55533a4-fc09-4c82-b912-7fee450d06e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:35.866154 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c55533a4-fc09-4c82-b912-7fee450d06e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:35.866649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c55533a4-fc09-4c82-b912-7fee450d06e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:35.867077 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c55533a4-fc09-4c82-b912-7fee450d06e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:35.899411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c55533a4-fc09-4c82-b912-7fee450d06e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047833 Aug 28 01:14:35.900058 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 864/1498] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:35 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:36.911873 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-dc145fef-e364-4b11-a2d3-dfbb06c751fa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:36.914070 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-dc145fef-e364-4b11-a2d3-dfbb06c751fa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:36.920624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dc145fef-e364-4b11-a2d3-dfbb06c751fa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:36.921228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dc145fef-e364-4b11-a2d3-dfbb06c751fa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:36.921566 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dc145fef-e364-4b11-a2d3-dfbb06c751fa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:36.922264 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dc145fef-e364-4b11-a2d3-dfbb06c751fa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:36.922824 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dc145fef-e364-4b11-a2d3-dfbb06c751fa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:36.923362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dc145fef-e364-4b11-a2d3-dfbb06c751fa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:36.954153 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-dc145fef-e364-4b11-a2d3-dfbb06c751fa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046197 Aug 28 01:14:36.954724 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 635/1499] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:36 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:37.966198 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9edeb640-fa62-497d-a6a1-b778611fc609 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:37.968402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9edeb640-fa62-497d-a6a1-b778611fc609 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:37.975193 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9edeb640-fa62-497d-a6a1-b778611fc609 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:37.975683 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9edeb640-fa62-497d-a6a1-b778611fc609 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:37.976105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9edeb640-fa62-497d-a6a1-b778611fc609 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:37.977073 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9edeb640-fa62-497d-a6a1-b778611fc609 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:37.977397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9edeb640-fa62-497d-a6a1-b778611fc609 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:37.977731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9edeb640-fa62-497d-a6a1-b778611fc609 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:37.980775 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=90 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:14:38.015462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9edeb640-fa62-497d-a6a1-b778611fc609 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053601 Aug 28 01:14:38.016283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 865/1500] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:37 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:39.029229 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2a794cfb-213b-431e-a398-000d90fca622 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:39.031415 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2a794cfb-213b-431e-a398-000d90fca622 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:39.038274 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2a794cfb-213b-431e-a398-000d90fca622 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:39.038769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2a794cfb-213b-431e-a398-000d90fca622 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:39.039209 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2a794cfb-213b-431e-a398-000d90fca622 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:39.039925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2a794cfb-213b-431e-a398-000d90fca622 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:39.040379 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2a794cfb-213b-431e-a398-000d90fca622 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:39.040803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2a794cfb-213b-431e-a398-000d90fca622 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:39.075082 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2a794cfb-213b-431e-a398-000d90fca622 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049769 Aug 28 01:14:39.075969 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 636/1501] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:39 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:40.090868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0a7914ae-0697-4a7b-88ee-949915aaad15 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:40.093370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0a7914ae-0697-4a7b-88ee-949915aaad15 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:40.100171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a7914ae-0697-4a7b-88ee-949915aaad15 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:40.100653 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a7914ae-0697-4a7b-88ee-949915aaad15 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:40.101126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a7914ae-0697-4a7b-88ee-949915aaad15 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:40.101820 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a7914ae-0697-4a7b-88ee-949915aaad15 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:40.102268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a7914ae-0697-4a7b-88ee-949915aaad15 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:40.102697 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a7914ae-0697-4a7b-88ee-949915aaad15 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:40.138822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0a7914ae-0697-4a7b-88ee-949915aaad15 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052026 Aug 28 01:14:40.139435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 866/1502] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:40 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:41.151500 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-59bc484e-0995-4ca8-b8c3-165f980fe718 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:41.153710 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-59bc484e-0995-4ca8-b8c3-165f980fe718 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:41.160687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-59bc484e-0995-4ca8-b8c3-165f980fe718 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:41.161208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-59bc484e-0995-4ca8-b8c3-165f980fe718 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:41.161626 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-59bc484e-0995-4ca8-b8c3-165f980fe718 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:41.162336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-59bc484e-0995-4ca8-b8c3-165f980fe718 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:41.162778 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-59bc484e-0995-4ca8-b8c3-165f980fe718 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:41.163191 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-59bc484e-0995-4ca8-b8c3-165f980fe718 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:41.195964 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-59bc484e-0995-4ca8-b8c3-165f980fe718 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048499 Aug 28 01:14:41.200605 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 637/1503] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:41 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:42.209023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-16ea803d-251f-4b38-bd50-f5595f9104fc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:42.214055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-16ea803d-251f-4b38-bd50-f5595f9104fc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:42.219758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16ea803d-251f-4b38-bd50-f5595f9104fc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:42.220237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16ea803d-251f-4b38-bd50-f5595f9104fc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:42.220668 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16ea803d-251f-4b38-bd50-f5595f9104fc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:42.221402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16ea803d-251f-4b38-bd50-f5595f9104fc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:42.221847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16ea803d-251f-4b38-bd50-f5595f9104fc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:42.222267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16ea803d-251f-4b38-bd50-f5595f9104fc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:42.254372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-16ea803d-251f-4b38-bd50-f5595f9104fc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049551 Aug 28 01:14:42.254979 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 867/1504] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:42 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:43.268617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7892c01a-b2af-495a-905f-c8f38dca48c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:43.272117 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7892c01a-b2af-495a-905f-c8f38dca48c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:43.280119 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7892c01a-b2af-495a-905f-c8f38dca48c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:43.280807 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7892c01a-b2af-495a-905f-c8f38dca48c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:43.281279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7892c01a-b2af-495a-905f-c8f38dca48c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:43.282146 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7892c01a-b2af-495a-905f-c8f38dca48c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:43.282599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7892c01a-b2af-495a-905f-c8f38dca48c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:43.283018 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7892c01a-b2af-495a-905f-c8f38dca48c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:43.287160 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=90 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:14:43.315564 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7892c01a-b2af-495a-905f-c8f38dca48c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052445 Aug 28 01:14:43.316108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 638/1505] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:43 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:44.328373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-533deb99-f76b-41d9-b8d8-35a3fdc198ef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:44.330735 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-533deb99-f76b-41d9-b8d8-35a3fdc198ef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:44.338198 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-533deb99-f76b-41d9-b8d8-35a3fdc198ef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:44.338786 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-533deb99-f76b-41d9-b8d8-35a3fdc198ef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:44.339354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-533deb99-f76b-41d9-b8d8-35a3fdc198ef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:44.340095 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-533deb99-f76b-41d9-b8d8-35a3fdc198ef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:44.340771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-533deb99-f76b-41d9-b8d8-35a3fdc198ef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:44.341220 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-533deb99-f76b-41d9-b8d8-35a3fdc198ef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:44.388243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-533deb99-f76b-41d9-b8d8-35a3fdc198ef tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063549 Aug 28 01:14:44.396265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 868/1506] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:44 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:45.404047 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0faaea1f-35f3-4d6b-aa22-d5e09747425c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:45.406474 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0faaea1f-35f3-4d6b-aa22-d5e09747425c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:45.413964 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0faaea1f-35f3-4d6b-aa22-d5e09747425c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:45.414529 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0faaea1f-35f3-4d6b-aa22-d5e09747425c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:45.415036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0faaea1f-35f3-4d6b-aa22-d5e09747425c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:45.415819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0faaea1f-35f3-4d6b-aa22-d5e09747425c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:45.416339 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0faaea1f-35f3-4d6b-aa22-d5e09747425c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:45.416831 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0faaea1f-35f3-4d6b-aa22-d5e09747425c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:45.450199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0faaea1f-35f3-4d6b-aa22-d5e09747425c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050574 Aug 28 01:14:45.451052 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 639/1507] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:45 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:46.464746 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9c3b200e-c0bf-41b7-b53c-db461b843ed7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:46.468124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9c3b200e-c0bf-41b7-b53c-db461b843ed7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:46.476796 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c3b200e-c0bf-41b7-b53c-db461b843ed7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:46.477297 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c3b200e-c0bf-41b7-b53c-db461b843ed7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:46.478052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c3b200e-c0bf-41b7-b53c-db461b843ed7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:46.478802 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c3b200e-c0bf-41b7-b53c-db461b843ed7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:46.479260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c3b200e-c0bf-41b7-b53c-db461b843ed7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:46.479687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c3b200e-c0bf-41b7-b53c-db461b843ed7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:46.519386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9c3b200e-c0bf-41b7-b53c-db461b843ed7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059183 Aug 28 01:14:46.520424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 869/1508] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:46 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:47.532250 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-062451a6-7ffd-4a35-aa93-c05e1485e84b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:47.534651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-062451a6-7ffd-4a35-aa93-c05e1485e84b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:47.542376 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-062451a6-7ffd-4a35-aa93-c05e1485e84b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:47.542376 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-062451a6-7ffd-4a35-aa93-c05e1485e84b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:47.542886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-062451a6-7ffd-4a35-aa93-c05e1485e84b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:47.543579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-062451a6-7ffd-4a35-aa93-c05e1485e84b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:47.544032 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-062451a6-7ffd-4a35-aa93-c05e1485e84b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:47.544461 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-062451a6-7ffd-4a35-aa93-c05e1485e84b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:47.591004 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-062451a6-7ffd-4a35-aa93-c05e1485e84b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062814 Aug 28 01:14:47.591591 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 640/1509] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:47 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:48.604088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f6b6aaab-d095-49df-95a3-10770336a3ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:48.606327 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f6b6aaab-d095-49df-95a3-10770336a3ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:48.612822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6b6aaab-d095-49df-95a3-10770336a3ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:48.613347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6b6aaab-d095-49df-95a3-10770336a3ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:48.613775 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6b6aaab-d095-49df-95a3-10770336a3ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:48.614485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6b6aaab-d095-49df-95a3-10770336a3ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:48.614928 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6b6aaab-d095-49df-95a3-10770336a3ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:48.615346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6b6aaab-d095-49df-95a3-10770336a3ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:48.650110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f6b6aaab-d095-49df-95a3-10770336a3ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049775 Aug 28 01:14:48.650964 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 870/1510] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:48 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:49.665201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-74e707fb-af39-41a4-a99c-5ccff970e870 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:49.667664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-74e707fb-af39-41a4-a99c-5ccff970e870 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:49.674528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-74e707fb-af39-41a4-a99c-5ccff970e870 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:49.675035 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-74e707fb-af39-41a4-a99c-5ccff970e870 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:49.675461 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-74e707fb-af39-41a4-a99c-5ccff970e870 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:49.676158 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-74e707fb-af39-41a4-a99c-5ccff970e870 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:49.676579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-74e707fb-af39-41a4-a99c-5ccff970e870 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:49.677042 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-74e707fb-af39-41a4-a99c-5ccff970e870 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:49.709506 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-74e707fb-af39-41a4-a99c-5ccff970e870 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050170 Aug 28 01:14:49.710094 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 641/1511] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:49 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:50.722403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-00ffe8d4-b1fb-4eee-bfb4-f539e0454c8d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:50.724584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-00ffe8d4-b1fb-4eee-bfb4-f539e0454c8d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:50.732245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-00ffe8d4-b1fb-4eee-bfb4-f539e0454c8d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:50.732726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-00ffe8d4-b1fb-4eee-bfb4-f539e0454c8d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:50.733195 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-00ffe8d4-b1fb-4eee-bfb4-f539e0454c8d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:50.734030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-00ffe8d4-b1fb-4eee-bfb4-f539e0454c8d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:50.734465 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-00ffe8d4-b1fb-4eee-bfb4-f539e0454c8d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:50.734887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-00ffe8d4-b1fb-4eee-bfb4-f539e0454c8d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:50.767621 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-00ffe8d4-b1fb-4eee-bfb4-f539e0454c8d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049328 Aug 28 01:14:50.768206 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 871/1512] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:50 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:51.780431 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b0b76fc2-bbee-4278-b8e5-a23696126f87 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:51.783558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b0b76fc2-bbee-4278-b8e5-a23696126f87 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:51.786552 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=30 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:14:51.793743 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0b76fc2-bbee-4278-b8e5-a23696126f87 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:51.794209 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0b76fc2-bbee-4278-b8e5-a23696126f87 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:51.794637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0b76fc2-bbee-4278-b8e5-a23696126f87 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:51.795303 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0b76fc2-bbee-4278-b8e5-a23696126f87 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:51.795755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0b76fc2-bbee-4278-b8e5-a23696126f87 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:51.796170 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0b76fc2-bbee-4278-b8e5-a23696126f87 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:51.826568 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b0b76fc2-bbee-4278-b8e5-a23696126f87 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050183 Aug 28 01:14:51.827107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 642/1513] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:51 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:52.839784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-795df54f-3cac-4048-b9a0-b628847a0dd4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:52.842034 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-795df54f-3cac-4048-b9a0-b628847a0dd4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:52.845407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=30 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:14:52.851407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-795df54f-3cac-4048-b9a0-b628847a0dd4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:52.851894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-795df54f-3cac-4048-b9a0-b628847a0dd4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:52.852340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-795df54f-3cac-4048-b9a0-b628847a0dd4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:52.853101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-795df54f-3cac-4048-b9a0-b628847a0dd4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:52.853557 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-795df54f-3cac-4048-b9a0-b628847a0dd4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:52.853987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-795df54f-3cac-4048-b9a0-b628847a0dd4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:52.901209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-795df54f-3cac-4048-b9a0-b628847a0dd4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.065284 Aug 28 01:14:52.902325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 872/1514] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:52 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:53.917862 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e4a337b8-410a-4065-b961-d6d81fbd8a21 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:53.920044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e4a337b8-410a-4065-b961-d6d81fbd8a21 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:53.927267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e4a337b8-410a-4065-b961-d6d81fbd8a21 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:53.927750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e4a337b8-410a-4065-b961-d6d81fbd8a21 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:53.928173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e4a337b8-410a-4065-b961-d6d81fbd8a21 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:53.928869 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e4a337b8-410a-4065-b961-d6d81fbd8a21 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:53.929552 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e4a337b8-410a-4065-b961-d6d81fbd8a21 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:53.929967 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e4a337b8-410a-4065-b961-d6d81fbd8a21 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:53.961432 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e4a337b8-410a-4065-b961-d6d81fbd8a21 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047726 Aug 28 01:14:53.962012 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 643/1515] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:53 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:54.974527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-320351df-9f85-460e-b676-53d41e344161 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:54.976791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-320351df-9f85-460e-b676-53d41e344161 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:54.984209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-320351df-9f85-460e-b676-53d41e344161 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:54.985033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-320351df-9f85-460e-b676-53d41e344161 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:54.985148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-320351df-9f85-460e-b676-53d41e344161 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:54.985833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-320351df-9f85-460e-b676-53d41e344161 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:54.986279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-320351df-9f85-460e-b676-53d41e344161 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:54.986762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-320351df-9f85-460e-b676-53d41e344161 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:55.018893 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-320351df-9f85-460e-b676-53d41e344161 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048552 Aug 28 01:14:55.019685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 873/1516] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:54 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:56.034821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9ef7ae4a-39fc-4ebd-9b0e-c1e6f7b01396 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:56.037544 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9ef7ae4a-39fc-4ebd-9b0e-c1e6f7b01396 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:56.045576 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9ef7ae4a-39fc-4ebd-9b0e-c1e6f7b01396 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:56.046071 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9ef7ae4a-39fc-4ebd-9b0e-c1e6f7b01396 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:56.046532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9ef7ae4a-39fc-4ebd-9b0e-c1e6f7b01396 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:56.047265 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9ef7ae4a-39fc-4ebd-9b0e-c1e6f7b01396 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:56.047730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9ef7ae4a-39fc-4ebd-9b0e-c1e6f7b01396 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:56.048175 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9ef7ae4a-39fc-4ebd-9b0e-c1e6f7b01396 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:56.079318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9ef7ae4a-39fc-4ebd-9b0e-c1e6f7b01396 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050872 Aug 28 01:14:56.079897 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 644/1517] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:56 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:57.092133 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2283b28e-9735-4fd2-b4d2-3b872b32d1c3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:57.094329 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2283b28e-9735-4fd2-b4d2-3b872b32d1c3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:57.100858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2283b28e-9735-4fd2-b4d2-3b872b32d1c3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:57.101361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2283b28e-9735-4fd2-b4d2-3b872b32d1c3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:57.101789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2283b28e-9735-4fd2-b4d2-3b872b32d1c3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:57.102467 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2283b28e-9735-4fd2-b4d2-3b872b32d1c3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:57.102912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2283b28e-9735-4fd2-b4d2-3b872b32d1c3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:57.103322 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2283b28e-9735-4fd2-b4d2-3b872b32d1c3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:57.136515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2283b28e-9735-4fd2-b4d2-3b872b32d1c3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048396 Aug 28 01:14:57.137259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 874/1518] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:57 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:58.149184 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e804f82f-0f4d-4acf-9647-ffcded1a3145 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:58.151377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e804f82f-0f4d-4acf-9647-ffcded1a3145 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:58.158265 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e804f82f-0f4d-4acf-9647-ffcded1a3145 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:58.158751 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e804f82f-0f4d-4acf-9647-ffcded1a3145 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:58.159193 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e804f82f-0f4d-4acf-9647-ffcded1a3145 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:58.159890 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e804f82f-0f4d-4acf-9647-ffcded1a3145 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:58.160338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e804f82f-0f4d-4acf-9647-ffcded1a3145 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:58.160757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e804f82f-0f4d-4acf-9647-ffcded1a3145 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:58.192164 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e804f82f-0f4d-4acf-9647-ffcded1a3145 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047053 Aug 28 01:14:58.192879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 645/1519] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:58 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:14:59.205674 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-afb55fd1-5fc9-457b-a00d-e81a2e3b6a4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:14:59.208618 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-afb55fd1-5fc9-457b-a00d-e81a2e3b6a4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:14:59.215680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-afb55fd1-5fc9-457b-a00d-e81a2e3b6a4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:59.216164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-afb55fd1-5fc9-457b-a00d-e81a2e3b6a4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:59.216597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-afb55fd1-5fc9-457b-a00d-e81a2e3b6a4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:59.217347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-afb55fd1-5fc9-457b-a00d-e81a2e3b6a4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:14:59.217824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-afb55fd1-5fc9-457b-a00d-e81a2e3b6a4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:14:59.218244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-afb55fd1-5fc9-457b-a00d-e81a2e3b6a4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:14:59.250761 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-afb55fd1-5fc9-457b-a00d-e81a2e3b6a4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049112 Aug 28 01:14:59.251717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 875/1520] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:14:59 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:00.264678 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4a7e3f78-c498-4b57-a902-3b675aabbd27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:00.266968 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4a7e3f78-c498-4b57-a902-3b675aabbd27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:00.274377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a7e3f78-c498-4b57-a902-3b675aabbd27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:00.274864 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a7e3f78-c498-4b57-a902-3b675aabbd27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:00.275323 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a7e3f78-c498-4b57-a902-3b675aabbd27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:00.276019 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a7e3f78-c498-4b57-a902-3b675aabbd27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:00.276462 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a7e3f78-c498-4b57-a902-3b675aabbd27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:00.276878 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a7e3f78-c498-4b57-a902-3b675aabbd27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:00.308083 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4a7e3f78-c498-4b57-a902-3b675aabbd27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047663 Aug 28 01:15:00.308753 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 646/1521] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:00 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:01.321620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4ab5feb3-969d-40ed-a156-152681f805f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:01.323700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4ab5feb3-969d-40ed-a156-152681f805f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:01.330518 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ab5feb3-969d-40ed-a156-152681f805f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:01.331009 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ab5feb3-969d-40ed-a156-152681f805f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:01.331441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ab5feb3-969d-40ed-a156-152681f805f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:01.332139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ab5feb3-969d-40ed-a156-152681f805f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:01.332590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ab5feb3-969d-40ed-a156-152681f805f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:01.333016 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ab5feb3-969d-40ed-a156-152681f805f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:01.363598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4ab5feb3-969d-40ed-a156-152681f805f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046462 Aug 28 01:15:01.364153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 876/1522] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:01 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:02.378472 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ce6c5631-2381-4b1e-8362-f53b982769fc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:02.380906 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ce6c5631-2381-4b1e-8362-f53b982769fc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:02.387996 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce6c5631-2381-4b1e-8362-f53b982769fc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:02.388569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce6c5631-2381-4b1e-8362-f53b982769fc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:02.389101 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce6c5631-2381-4b1e-8362-f53b982769fc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:02.389927 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce6c5631-2381-4b1e-8362-f53b982769fc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:02.390467 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce6c5631-2381-4b1e-8362-f53b982769fc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:02.391088 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce6c5631-2381-4b1e-8362-f53b982769fc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:02.423208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ce6c5631-2381-4b1e-8362-f53b982769fc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049435 Aug 28 01:15:02.423760 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 647/1523] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:02 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:03.437450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7ab44fe3-2616-4313-9db8-68abf5ec716c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:03.439655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7ab44fe3-2616-4313-9db8-68abf5ec716c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:03.447615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ab44fe3-2616-4313-9db8-68abf5ec716c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:03.448098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ab44fe3-2616-4313-9db8-68abf5ec716c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:03.448530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ab44fe3-2616-4313-9db8-68abf5ec716c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:03.449279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ab44fe3-2616-4313-9db8-68abf5ec716c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:03.449728 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ab44fe3-2616-4313-9db8-68abf5ec716c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:03.450153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ab44fe3-2616-4313-9db8-68abf5ec716c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:03.481322 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7ab44fe3-2616-4313-9db8-68abf5ec716c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048978 Aug 28 01:15:03.481881 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 877/1524] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:03 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:04.495918 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4a8c1106-9768-4550-9ff4-18d5fa9b2a42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:04.498630 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4a8c1106-9768-4550-9ff4-18d5fa9b2a42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:04.507799 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a8c1106-9768-4550-9ff4-18d5fa9b2a42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:04.508419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a8c1106-9768-4550-9ff4-18d5fa9b2a42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:04.509016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a8c1106-9768-4550-9ff4-18d5fa9b2a42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:04.509924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a8c1106-9768-4550-9ff4-18d5fa9b2a42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:04.510513 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a8c1106-9768-4550-9ff4-18d5fa9b2a42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:04.511082 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a8c1106-9768-4550-9ff4-18d5fa9b2a42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:04.543224 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4a8c1106-9768-4550-9ff4-18d5fa9b2a42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052377 Aug 28 01:15:04.543755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 648/1525] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:04 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:05.558828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a4050e5c-3734-4897-b612-9426ee520e61 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:05.561305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a4050e5c-3734-4897-b612-9426ee520e61 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:05.568144 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4050e5c-3734-4897-b612-9426ee520e61 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:05.568625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4050e5c-3734-4897-b612-9426ee520e61 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:05.569440 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4050e5c-3734-4897-b612-9426ee520e61 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:05.570142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4050e5c-3734-4897-b612-9426ee520e61 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:05.570790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4050e5c-3734-4897-b612-9426ee520e61 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:05.571251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4050e5c-3734-4897-b612-9426ee520e61 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:05.602212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a4050e5c-3734-4897-b612-9426ee520e61 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049695 Aug 28 01:15:05.602797 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 878/1526] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:05 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:06.615068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-67fe84cc-a22c-4dbe-ad32-20f5dd7101cb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:06.617296 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-67fe84cc-a22c-4dbe-ad32-20f5dd7101cb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:06.624247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67fe84cc-a22c-4dbe-ad32-20f5dd7101cb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:06.624737 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67fe84cc-a22c-4dbe-ad32-20f5dd7101cb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:06.625220 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67fe84cc-a22c-4dbe-ad32-20f5dd7101cb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:06.625927 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67fe84cc-a22c-4dbe-ad32-20f5dd7101cb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:06.626364 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67fe84cc-a22c-4dbe-ad32-20f5dd7101cb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:06.626787 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67fe84cc-a22c-4dbe-ad32-20f5dd7101cb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:06.658605 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-67fe84cc-a22c-4dbe-ad32-20f5dd7101cb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047471 Aug 28 01:15:06.659240 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 649/1527] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:06 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:07.672946 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7ee48a28-46eb-4f10-9535-a9feeba9e230 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:07.675161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7ee48a28-46eb-4f10-9535-a9feeba9e230 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:07.682265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ee48a28-46eb-4f10-9535-a9feeba9e230 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:07.682752 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ee48a28-46eb-4f10-9535-a9feeba9e230 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:07.683184 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ee48a28-46eb-4f10-9535-a9feeba9e230 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:07.683883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ee48a28-46eb-4f10-9535-a9feeba9e230 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:07.684330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ee48a28-46eb-4f10-9535-a9feeba9e230 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:07.684756 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ee48a28-46eb-4f10-9535-a9feeba9e230 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:07.717085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7ee48a28-46eb-4f10-9535-a9feeba9e230 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048414 Aug 28 01:15:07.717700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 879/1528] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:07 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:08.730069 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f2cd6dc7-8213-4f54-bff9-440114d50b42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:08.732347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f2cd6dc7-8213-4f54-bff9-440114d50b42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:08.739247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2cd6dc7-8213-4f54-bff9-440114d50b42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:08.739744 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2cd6dc7-8213-4f54-bff9-440114d50b42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:08.740160 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2cd6dc7-8213-4f54-bff9-440114d50b42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:08.740879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2cd6dc7-8213-4f54-bff9-440114d50b42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:08.741362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2cd6dc7-8213-4f54-bff9-440114d50b42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:08.741785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2cd6dc7-8213-4f54-bff9-440114d50b42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:08.771490 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f2cd6dc7-8213-4f54-bff9-440114d50b42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045477 Aug 28 01:15:08.772041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 650/1529] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:08 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:09.785355 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-68f1a62f-5522-4cb7-a0f1-494956f8ed8e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:09.787904 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-68f1a62f-5522-4cb7-a0f1-494956f8ed8e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:09.796467 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-68f1a62f-5522-4cb7-a0f1-494956f8ed8e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:09.796945 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-68f1a62f-5522-4cb7-a0f1-494956f8ed8e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:09.797423 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-68f1a62f-5522-4cb7-a0f1-494956f8ed8e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:09.798085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-68f1a62f-5522-4cb7-a0f1-494956f8ed8e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:09.798524 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-68f1a62f-5522-4cb7-a0f1-494956f8ed8e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:09.798939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-68f1a62f-5522-4cb7-a0f1-494956f8ed8e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:09.801280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=90 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:15:09.830702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-68f1a62f-5522-4cb7-a0f1-494956f8ed8e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050305 Aug 28 01:15:09.831244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 880/1530] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:09 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:10.845462 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cf1e8782-315d-4b0d-99d8-4a8a90494c7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:10.847521 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cf1e8782-315d-4b0d-99d8-4a8a90494c7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:10.854634 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cf1e8782-315d-4b0d-99d8-4a8a90494c7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:10.855117 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cf1e8782-315d-4b0d-99d8-4a8a90494c7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:10.855542 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cf1e8782-315d-4b0d-99d8-4a8a90494c7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:10.856239 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cf1e8782-315d-4b0d-99d8-4a8a90494c7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:10.856680 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cf1e8782-315d-4b0d-99d8-4a8a90494c7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:10.857134 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cf1e8782-315d-4b0d-99d8-4a8a90494c7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:10.888615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cf1e8782-315d-4b0d-99d8-4a8a90494c7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048522 Aug 28 01:15:10.889206 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 651/1531] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:10 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:11.901482 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-54e040ae-b5af-40a9-892e-ead549cba23b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:11.903642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-54e040ae-b5af-40a9-892e-ead549cba23b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:11.910678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54e040ae-b5af-40a9-892e-ead549cba23b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:11.911165 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54e040ae-b5af-40a9-892e-ead549cba23b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:11.911600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54e040ae-b5af-40a9-892e-ead549cba23b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:11.912291 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54e040ae-b5af-40a9-892e-ead549cba23b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:11.912713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54e040ae-b5af-40a9-892e-ead549cba23b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:11.913208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54e040ae-b5af-40a9-892e-ead549cba23b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:11.943245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-54e040ae-b5af-40a9-892e-ead549cba23b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045716 Aug 28 01:15:11.943717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 881/1532] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:11 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:12.960165 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-85a09c7a-6fc0-4145-9280-efa4a348b7ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:12.963248 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-85a09c7a-6fc0-4145-9280-efa4a348b7ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:12.971245 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85a09c7a-6fc0-4145-9280-efa4a348b7ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:12.971728 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85a09c7a-6fc0-4145-9280-efa4a348b7ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:12.972155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85a09c7a-6fc0-4145-9280-efa4a348b7ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:12.972839 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85a09c7a-6fc0-4145-9280-efa4a348b7ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:12.973312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85a09c7a-6fc0-4145-9280-efa4a348b7ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:12.973727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85a09c7a-6fc0-4145-9280-efa4a348b7ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:13.004292 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-85a09c7a-6fc0-4145-9280-efa4a348b7ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050333 Aug 28 01:15:13.004814 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 652/1533] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:12 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:14.016835 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0a2b5835-e13b-42af-a6e2-7e877cc39ebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:14.019075 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0a2b5835-e13b-42af-a6e2-7e877cc39ebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:14.026470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a2b5835-e13b-42af-a6e2-7e877cc39ebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:14.026954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a2b5835-e13b-42af-a6e2-7e877cc39ebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:14.027425 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a2b5835-e13b-42af-a6e2-7e877cc39ebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:14.028124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a2b5835-e13b-42af-a6e2-7e877cc39ebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:14.028572 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a2b5835-e13b-42af-a6e2-7e877cc39ebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:14.028986 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a2b5835-e13b-42af-a6e2-7e877cc39ebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:14.060158 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0a2b5835-e13b-42af-a6e2-7e877cc39ebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047302 Aug 28 01:15:14.060734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 882/1534] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:14 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:15.075729 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a689c968-ab34-4d87-b5cf-8a47a45199f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:15.077859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a689c968-ab34-4d87-b5cf-8a47a45199f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:15.085347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a689c968-ab34-4d87-b5cf-8a47a45199f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:15.085827 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a689c968-ab34-4d87-b5cf-8a47a45199f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:15.086263 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a689c968-ab34-4d87-b5cf-8a47a45199f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:15.086968 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a689c968-ab34-4d87-b5cf-8a47a45199f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:15.087416 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a689c968-ab34-4d87-b5cf-8a47a45199f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:15.087847 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a689c968-ab34-4d87-b5cf-8a47a45199f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:15.090269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=90 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:15:15.121624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a689c968-ab34-4d87-b5cf-8a47a45199f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049965 Aug 28 01:15:15.122203 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 653/1535] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:15 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:16.134785 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d6f103f4-a69a-44be-9f18-567ad0fcb019 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:16.136972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d6f103f4-a69a-44be-9f18-567ad0fcb019 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:16.144125 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6f103f4-a69a-44be-9f18-567ad0fcb019 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:16.144622 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6f103f4-a69a-44be-9f18-567ad0fcb019 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:16.145076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6f103f4-a69a-44be-9f18-567ad0fcb019 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:16.145773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6f103f4-a69a-44be-9f18-567ad0fcb019 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:16.146241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6f103f4-a69a-44be-9f18-567ad0fcb019 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:16.146651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6f103f4-a69a-44be-9f18-567ad0fcb019 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:16.178338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d6f103f4-a69a-44be-9f18-567ad0fcb019 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047581 Aug 28 01:15:16.178918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 883/1536] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:16 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:17.191128 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7204102d-0602-4d00-bd20-36eed56004f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:17.194775 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7204102d-0602-4d00-bd20-36eed56004f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:17.203339 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7204102d-0602-4d00-bd20-36eed56004f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:17.203828 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7204102d-0602-4d00-bd20-36eed56004f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:17.204269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7204102d-0602-4d00-bd20-36eed56004f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:17.205007 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7204102d-0602-4d00-bd20-36eed56004f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:17.205478 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7204102d-0602-4d00-bd20-36eed56004f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:17.205946 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7204102d-0602-4d00-bd20-36eed56004f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:17.236347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7204102d-0602-4d00-bd20-36eed56004f0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049356 Aug 28 01:15:17.236912 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 654/1537] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:17 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:18.250174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6c40ecff-60d2-4943-9ff4-e759076d799f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:18.253061 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6c40ecff-60d2-4943-9ff4-e759076d799f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:18.259436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c40ecff-60d2-4943-9ff4-e759076d799f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:18.259923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c40ecff-60d2-4943-9ff4-e759076d799f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:18.260355 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c40ecff-60d2-4943-9ff4-e759076d799f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:18.261087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c40ecff-60d2-4943-9ff4-e759076d799f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:18.261532 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c40ecff-60d2-4943-9ff4-e759076d799f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:18.261949 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c40ecff-60d2-4943-9ff4-e759076d799f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:18.291880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6c40ecff-60d2-4943-9ff4-e759076d799f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046658 Aug 28 01:15:18.292434 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 884/1538] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:18 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:19.308228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9519ebfa-1c28-4fff-aac9-2a640ee87bd6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:19.310904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9519ebfa-1c28-4fff-aac9-2a640ee87bd6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:19.321097 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9519ebfa-1c28-4fff-aac9-2a640ee87bd6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:19.321740 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9519ebfa-1c28-4fff-aac9-2a640ee87bd6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:19.322337 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9519ebfa-1c28-4fff-aac9-2a640ee87bd6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:19.323246 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9519ebfa-1c28-4fff-aac9-2a640ee87bd6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:19.323766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9519ebfa-1c28-4fff-aac9-2a640ee87bd6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:19.324295 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9519ebfa-1c28-4fff-aac9-2a640ee87bd6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:19.362382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9519ebfa-1c28-4fff-aac9-2a640ee87bd6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060249 Aug 28 01:15:19.362943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 655/1539] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:19 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:20.376273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ef2b6726-2494-48b4-acf5-1b5070e09faf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:20.378549 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ef2b6726-2494-48b4-acf5-1b5070e09faf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:20.385561 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef2b6726-2494-48b4-acf5-1b5070e09faf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:20.386037 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef2b6726-2494-48b4-acf5-1b5070e09faf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:20.386466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef2b6726-2494-48b4-acf5-1b5070e09faf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:20.387152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef2b6726-2494-48b4-acf5-1b5070e09faf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:20.387634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef2b6726-2494-48b4-acf5-1b5070e09faf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:20.388045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef2b6726-2494-48b4-acf5-1b5070e09faf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:20.418850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ef2b6726-2494-48b4-acf5-1b5070e09faf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046508 Aug 28 01:15:20.419410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 885/1540] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:20 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:21.431693 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-92db27a7-f97d-48e8-a5c8-b1723c48014f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:21.433864 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-92db27a7-f97d-48e8-a5c8-b1723c48014f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:21.440994 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-92db27a7-f97d-48e8-a5c8-b1723c48014f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:21.441478 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-92db27a7-f97d-48e8-a5c8-b1723c48014f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:21.441903 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-92db27a7-f97d-48e8-a5c8-b1723c48014f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:21.442592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-92db27a7-f97d-48e8-a5c8-b1723c48014f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:21.443036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-92db27a7-f97d-48e8-a5c8-b1723c48014f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:21.443451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-92db27a7-f97d-48e8-a5c8-b1723c48014f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:21.475527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-92db27a7-f97d-48e8-a5c8-b1723c48014f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048009 Aug 28 01:15:21.476080 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 656/1541] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:21 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:22.488474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e1d3f960-0275-4042-b10d-b68dc8269d5c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:22.490680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e1d3f960-0275-4042-b10d-b68dc8269d5c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:22.497514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1d3f960-0275-4042-b10d-b68dc8269d5c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:22.497987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1d3f960-0275-4042-b10d-b68dc8269d5c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:22.498410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1d3f960-0275-4042-b10d-b68dc8269d5c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:22.499098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1d3f960-0275-4042-b10d-b68dc8269d5c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:22.499540 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1d3f960-0275-4042-b10d-b68dc8269d5c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:22.499951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1d3f960-0275-4042-b10d-b68dc8269d5c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:22.529933 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e1d3f960-0275-4042-b10d-b68dc8269d5c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045515 Aug 28 01:15:22.530530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 886/1542] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:22 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:23.542095 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4209d5f1-0816-478a-b4ba-7d133370acb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:23.544437 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4209d5f1-0816-478a-b4ba-7d133370acb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:23.546430 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=30 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:15:23.552306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4209d5f1-0816-478a-b4ba-7d133370acb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:23.552798 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4209d5f1-0816-478a-b4ba-7d133370acb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:23.553261 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4209d5f1-0816-478a-b4ba-7d133370acb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:23.553994 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4209d5f1-0816-478a-b4ba-7d133370acb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:23.554445 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4209d5f1-0816-478a-b4ba-7d133370acb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:23.554873 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4209d5f1-0816-478a-b4ba-7d133370acb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:23.586664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4209d5f1-0816-478a-b4ba-7d133370acb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048902 Aug 28 01:15:23.587548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 657/1543] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:23 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:24.599988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cf3968c0-a1d9-4b78-af4b-c18035714ffd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:24.602441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cf3968c0-a1d9-4b78-af4b-c18035714ffd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:24.605680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=30 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:15:24.611514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf3968c0-a1d9-4b78-af4b-c18035714ffd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:24.612062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf3968c0-a1d9-4b78-af4b-c18035714ffd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:24.612664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf3968c0-a1d9-4b78-af4b-c18035714ffd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:24.613387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf3968c0-a1d9-4b78-af4b-c18035714ffd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:24.613812 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf3968c0-a1d9-4b78-af4b-c18035714ffd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:24.614243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf3968c0-a1d9-4b78-af4b-c18035714ffd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:24.644786 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cf3968c0-a1d9-4b78-af4b-c18035714ffd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049362 Aug 28 01:15:24.645377 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 887/1544] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:24 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:25.657287 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f2ee0556-bfff-4efe-ac2c-3474da65c86c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:25.659519 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f2ee0556-bfff-4efe-ac2c-3474da65c86c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:25.666897 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2ee0556-bfff-4efe-ac2c-3474da65c86c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:25.667382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2ee0556-bfff-4efe-ac2c-3474da65c86c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:25.667819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2ee0556-bfff-4efe-ac2c-3474da65c86c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:25.668514 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2ee0556-bfff-4efe-ac2c-3474da65c86c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:25.668985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2ee0556-bfff-4efe-ac2c-3474da65c86c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:25.669420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2ee0556-bfff-4efe-ac2c-3474da65c86c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:25.702734 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f2ee0556-bfff-4efe-ac2c-3474da65c86c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049523 Aug 28 01:15:25.703284 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 658/1545] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:25 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:26.719363 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2eb66a78-fbca-4849-a48b-0f6e63dcf45c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:26.722435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2eb66a78-fbca-4849-a48b-0f6e63dcf45c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:26.729754 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2eb66a78-fbca-4849-a48b-0f6e63dcf45c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:26.730224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2eb66a78-fbca-4849-a48b-0f6e63dcf45c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:26.730675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2eb66a78-fbca-4849-a48b-0f6e63dcf45c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:26.731376 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2eb66a78-fbca-4849-a48b-0f6e63dcf45c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:26.731818 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2eb66a78-fbca-4849-a48b-0f6e63dcf45c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:26.732231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2eb66a78-fbca-4849-a48b-0f6e63dcf45c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:26.763639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2eb66a78-fbca-4849-a48b-0f6e63dcf45c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050438 Aug 28 01:15:26.768023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 888/1546] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:26 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:27.775878 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f175ec06-69a3-49ba-af32-557f5a3f3462 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:27.777996 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f175ec06-69a3-49ba-af32-557f5a3f3462 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:27.784831 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f175ec06-69a3-49ba-af32-557f5a3f3462 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:27.785351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f175ec06-69a3-49ba-af32-557f5a3f3462 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:27.785786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f175ec06-69a3-49ba-af32-557f5a3f3462 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:27.786475 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f175ec06-69a3-49ba-af32-557f5a3f3462 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:27.786927 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f175ec06-69a3-49ba-af32-557f5a3f3462 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:27.787345 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f175ec06-69a3-49ba-af32-557f5a3f3462 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:27.817119 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f175ec06-69a3-49ba-af32-557f5a3f3462 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045260 Aug 28 01:15:27.817677 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 659/1547] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:27 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:28.831256 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-67aabb73-1e24-4454-afbf-2f73a8f92c8c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:28.833608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-67aabb73-1e24-4454-afbf-2f73a8f92c8c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:28.841100 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67aabb73-1e24-4454-afbf-2f73a8f92c8c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:28.841580 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67aabb73-1e24-4454-afbf-2f73a8f92c8c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:28.842014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67aabb73-1e24-4454-afbf-2f73a8f92c8c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:28.842761 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67aabb73-1e24-4454-afbf-2f73a8f92c8c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:28.843218 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67aabb73-1e24-4454-afbf-2f73a8f92c8c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:28.843632 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67aabb73-1e24-4454-afbf-2f73a8f92c8c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:28.875994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-67aabb73-1e24-4454-afbf-2f73a8f92c8c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049441 Aug 28 01:15:28.876755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 889/1548] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:28 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:29.890916 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ba5c87ab-bff2-4325-8c7e-cbc8b3183422 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:29.893703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ba5c87ab-bff2-4325-8c7e-cbc8b3183422 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:29.900761 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba5c87ab-bff2-4325-8c7e-cbc8b3183422 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:29.901269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba5c87ab-bff2-4325-8c7e-cbc8b3183422 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:29.901701 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba5c87ab-bff2-4325-8c7e-cbc8b3183422 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:29.902405 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba5c87ab-bff2-4325-8c7e-cbc8b3183422 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:29.902855 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba5c87ab-bff2-4325-8c7e-cbc8b3183422 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:29.903315 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba5c87ab-bff2-4325-8c7e-cbc8b3183422 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:29.935743 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ba5c87ab-bff2-4325-8c7e-cbc8b3183422 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050535 Aug 28 01:15:29.936308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 660/1549] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:29 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:30.951205 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-491c4710-ba1c-4216-b3c0-c1ec6af2651d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:30.953560 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-491c4710-ba1c-4216-b3c0-c1ec6af2651d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:30.961367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-491c4710-ba1c-4216-b3c0-c1ec6af2651d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:30.961923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-491c4710-ba1c-4216-b3c0-c1ec6af2651d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:30.962429 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-491c4710-ba1c-4216-b3c0-c1ec6af2651d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:30.963223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-491c4710-ba1c-4216-b3c0-c1ec6af2651d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:30.963745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-491c4710-ba1c-4216-b3c0-c1ec6af2651d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:30.964296 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-491c4710-ba1c-4216-b3c0-c1ec6af2651d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:30.995228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-491c4710-ba1c-4216-b3c0-c1ec6af2651d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048540 Aug 28 01:15:30.995780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 890/1550] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:30 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:32.007341 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3a9c9fbc-c74e-46e5-8cc2-f7e57dc3424d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:32.009480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3a9c9fbc-c74e-46e5-8cc2-f7e57dc3424d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:32.016454 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a9c9fbc-c74e-46e5-8cc2-f7e57dc3424d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:32.016948 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a9c9fbc-c74e-46e5-8cc2-f7e57dc3424d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:32.017416 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a9c9fbc-c74e-46e5-8cc2-f7e57dc3424d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:32.018133 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a9c9fbc-c74e-46e5-8cc2-f7e57dc3424d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:32.018704 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a9c9fbc-c74e-46e5-8cc2-f7e57dc3424d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:32.019116 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a9c9fbc-c74e-46e5-8cc2-f7e57dc3424d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:32.051931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3a9c9fbc-c74e-46e5-8cc2-f7e57dc3424d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048653 Aug 28 01:15:32.052690 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 661/1551] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:32 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:33.064845 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a585ec37-d4c0-48bc-9aa4-4a13c4b3b978 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:33.066978 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a585ec37-d4c0-48bc-9aa4-4a13c4b3b978 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:33.074628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a585ec37-d4c0-48bc-9aa4-4a13c4b3b978 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:33.075116 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a585ec37-d4c0-48bc-9aa4-4a13c4b3b978 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:33.075544 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a585ec37-d4c0-48bc-9aa4-4a13c4b3b978 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:33.076253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a585ec37-d4c0-48bc-9aa4-4a13c4b3b978 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:33.076698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a585ec37-d4c0-48bc-9aa4-4a13c4b3b978 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:33.077145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a585ec37-d4c0-48bc-9aa4-4a13c4b3b978 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:33.112618 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a585ec37-d4c0-48bc-9aa4-4a13c4b3b978 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051744 Aug 28 01:15:33.113273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 891/1552] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:33 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:34.127838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-365c814d-1324-48e1-932c-f90678645844 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:34.130056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-365c814d-1324-48e1-932c-f90678645844 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:34.137046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-365c814d-1324-48e1-932c-f90678645844 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:34.137552 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-365c814d-1324-48e1-932c-f90678645844 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:34.137976 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-365c814d-1324-48e1-932c-f90678645844 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:34.138680 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-365c814d-1324-48e1-932c-f90678645844 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:34.139126 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-365c814d-1324-48e1-932c-f90678645844 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:34.139540 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-365c814d-1324-48e1-932c-f90678645844 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:34.173089 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-365c814d-1324-48e1-932c-f90678645844 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049284 Aug 28 01:15:34.173671 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 662/1553] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:34 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:35.187353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d87a77f0-e1e7-4546-9e9e-cc495734e4b4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:35.189422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d87a77f0-e1e7-4546-9e9e-cc495734e4b4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:35.197216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d87a77f0-e1e7-4546-9e9e-cc495734e4b4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:35.197706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d87a77f0-e1e7-4546-9e9e-cc495734e4b4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:35.198131 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d87a77f0-e1e7-4546-9e9e-cc495734e4b4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:35.198821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d87a77f0-e1e7-4546-9e9e-cc495734e4b4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:35.199266 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d87a77f0-e1e7-4546-9e9e-cc495734e4b4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:35.199679 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d87a77f0-e1e7-4546-9e9e-cc495734e4b4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:35.231878 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d87a77f0-e1e7-4546-9e9e-cc495734e4b4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048738 Aug 28 01:15:35.232453 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 892/1554] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:35 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:36.246333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bcd3c30d-465b-4f90-81c4-5623202106de tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:36.248419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bcd3c30d-465b-4f90-81c4-5623202106de tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:36.256156 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcd3c30d-465b-4f90-81c4-5623202106de tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:36.256639 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcd3c30d-465b-4f90-81c4-5623202106de tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:36.257103 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcd3c30d-465b-4f90-81c4-5623202106de tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:36.257825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcd3c30d-465b-4f90-81c4-5623202106de tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:36.258266 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcd3c30d-465b-4f90-81c4-5623202106de tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:36.258684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcd3c30d-465b-4f90-81c4-5623202106de tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:36.291880 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bcd3c30d-465b-4f90-81c4-5623202106de tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049622 Aug 28 01:15:36.292495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 663/1555] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:36 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:37.305319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4f1b9780-d399-4cae-a10a-fe61b5128d09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:37.307439 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4f1b9780-d399-4cae-a10a-fe61b5128d09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:37.314121 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f1b9780-d399-4cae-a10a-fe61b5128d09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:37.314602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f1b9780-d399-4cae-a10a-fe61b5128d09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:37.315036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f1b9780-d399-4cae-a10a-fe61b5128d09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:37.315914 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f1b9780-d399-4cae-a10a-fe61b5128d09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:37.316369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f1b9780-d399-4cae-a10a-fe61b5128d09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:37.316787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4f1b9780-d399-4cae-a10a-fe61b5128d09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:37.348203 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4f1b9780-d399-4cae-a10a-fe61b5128d09 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047013 Aug 28 01:15:37.348770 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 893/1556] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:37 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:38.362400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6cd19bc8-6b52-4880-99ab-1205a06ed08c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:38.365770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6cd19bc8-6b52-4880-99ab-1205a06ed08c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:38.376395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6cd19bc8-6b52-4880-99ab-1205a06ed08c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:38.377272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6cd19bc8-6b52-4880-99ab-1205a06ed08c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:38.377952 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6cd19bc8-6b52-4880-99ab-1205a06ed08c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:38.378648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6cd19bc8-6b52-4880-99ab-1205a06ed08c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:38.379094 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6cd19bc8-6b52-4880-99ab-1205a06ed08c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:38.379509 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6cd19bc8-6b52-4880-99ab-1205a06ed08c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:38.411051 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6cd19bc8-6b52-4880-99ab-1205a06ed08c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054363 Aug 28 01:15:38.411628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 664/1557] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:38 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:39.424120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4b1e839d-3fc0-4251-aa25-564678b4199f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:39.426526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4b1e839d-3fc0-4251-aa25-564678b4199f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:39.434371 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b1e839d-3fc0-4251-aa25-564678b4199f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:39.434890 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b1e839d-3fc0-4251-aa25-564678b4199f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:39.435327 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b1e839d-3fc0-4251-aa25-564678b4199f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:39.436046 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b1e839d-3fc0-4251-aa25-564678b4199f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:39.436502 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b1e839d-3fc0-4251-aa25-564678b4199f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:39.436927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b1e839d-3fc0-4251-aa25-564678b4199f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:39.470983 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4b1e839d-3fc0-4251-aa25-564678b4199f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051029 Aug 28 01:15:39.471670 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 894/1558] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:39 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:40.487798 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d9266838-9e91-4875-8e17-9a4bef18b57d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:40.489948 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d9266838-9e91-4875-8e17-9a4bef18b57d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:40.496826 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d9266838-9e91-4875-8e17-9a4bef18b57d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:40.497342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d9266838-9e91-4875-8e17-9a4bef18b57d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:40.497776 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d9266838-9e91-4875-8e17-9a4bef18b57d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:40.498508 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d9266838-9e91-4875-8e17-9a4bef18b57d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:40.498958 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d9266838-9e91-4875-8e17-9a4bef18b57d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:40.499369 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d9266838-9e91-4875-8e17-9a4bef18b57d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:40.531473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d9266838-9e91-4875-8e17-9a4bef18b57d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050322 Aug 28 01:15:40.532034 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 665/1559] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:40 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:41.544935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f966e06f-e3c8-4cb5-9445-69b1a538714c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:41.547085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f966e06f-e3c8-4cb5-9445-69b1a538714c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:41.554061 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f966e06f-e3c8-4cb5-9445-69b1a538714c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:41.554519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f966e06f-e3c8-4cb5-9445-69b1a538714c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:41.554952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f966e06f-e3c8-4cb5-9445-69b1a538714c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:41.555655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f966e06f-e3c8-4cb5-9445-69b1a538714c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:41.556100 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f966e06f-e3c8-4cb5-9445-69b1a538714c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:41.556521 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f966e06f-e3c8-4cb5-9445-69b1a538714c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:41.559728 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=90 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:15:41.590020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f966e06f-e3c8-4cb5-9445-69b1a538714c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049472 Aug 28 01:15:41.590582 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 895/1560] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:41 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:42.603166 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cad2e0ca-8a54-4917-87a9-155ebc9bf02e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:42.605712 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cad2e0ca-8a54-4917-87a9-155ebc9bf02e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:42.613089 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cad2e0ca-8a54-4917-87a9-155ebc9bf02e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:42.613627 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cad2e0ca-8a54-4917-87a9-155ebc9bf02e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:42.614101 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cad2e0ca-8a54-4917-87a9-155ebc9bf02e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:42.614802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cad2e0ca-8a54-4917-87a9-155ebc9bf02e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:42.615251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cad2e0ca-8a54-4917-87a9-155ebc9bf02e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:42.615731 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cad2e0ca-8a54-4917-87a9-155ebc9bf02e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:42.650775 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cad2e0ca-8a54-4917-87a9-155ebc9bf02e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051736 Aug 28 01:15:42.651377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 666/1561] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:42 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:43.783846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7f0e1bfd-c628-429f-aa4f-4f854b596d22 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:43.786154 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7f0e1bfd-c628-429f-aa4f-4f854b596d22 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:43.792543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7f0e1bfd-c628-429f-aa4f-4f854b596d22 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:43.793045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7f0e1bfd-c628-429f-aa4f-4f854b596d22 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:43.793481 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7f0e1bfd-c628-429f-aa4f-4f854b596d22 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:43.794180 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7f0e1bfd-c628-429f-aa4f-4f854b596d22 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:43.794620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7f0e1bfd-c628-429f-aa4f-4f854b596d22 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:43.795035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7f0e1bfd-c628-429f-aa4f-4f854b596d22 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:43.828137 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7f0e1bfd-c628-429f-aa4f-4f854b596d22 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.168041 Aug 28 01:15:43.828704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 896/1562] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:43 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:44.841187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0884e90c-3722-47e9-9acd-443def413562 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:44.843790 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0884e90c-3722-47e9-9acd-443def413562 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:44.853222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0884e90c-3722-47e9-9acd-443def413562 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:44.853707 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0884e90c-3722-47e9-9acd-443def413562 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:44.854134 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0884e90c-3722-47e9-9acd-443def413562 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:44.854833 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0884e90c-3722-47e9-9acd-443def413562 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:44.855276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0884e90c-3722-47e9-9acd-443def413562 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:44.855720 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0884e90c-3722-47e9-9acd-443def413562 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:44.886885 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0884e90c-3722-47e9-9acd-443def413562 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049775 Aug 28 01:15:44.887463 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 667/1563] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:44 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:45.899203 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-46be5f2b-8a0b-4afe-a2a3-b12aeab7a369 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:45.901446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-46be5f2b-8a0b-4afe-a2a3-b12aeab7a369 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:45.908818 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46be5f2b-8a0b-4afe-a2a3-b12aeab7a369 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:45.909351 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46be5f2b-8a0b-4afe-a2a3-b12aeab7a369 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:45.909793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46be5f2b-8a0b-4afe-a2a3-b12aeab7a369 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:45.910505 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46be5f2b-8a0b-4afe-a2a3-b12aeab7a369 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:45.911041 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46be5f2b-8a0b-4afe-a2a3-b12aeab7a369 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:45.911484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46be5f2b-8a0b-4afe-a2a3-b12aeab7a369 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:45.945003 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-46be5f2b-8a0b-4afe-a2a3-b12aeab7a369 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049909 Aug 28 01:15:45.945572 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 897/1564] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:45 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:46.958261 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-08f2cfca-a7a1-4a27-8e3a-1156d3043be6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:46.960362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-08f2cfca-a7a1-4a27-8e3a-1156d3043be6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:46.968084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08f2cfca-a7a1-4a27-8e3a-1156d3043be6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:46.968585 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08f2cfca-a7a1-4a27-8e3a-1156d3043be6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:46.969120 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08f2cfca-a7a1-4a27-8e3a-1156d3043be6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:46.969741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08f2cfca-a7a1-4a27-8e3a-1156d3043be6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:46.970214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08f2cfca-a7a1-4a27-8e3a-1156d3043be6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:46.970630 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08f2cfca-a7a1-4a27-8e3a-1156d3043be6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:46.973426 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=90 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:15:47.003038 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-08f2cfca-a7a1-4a27-8e3a-1156d3043be6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048973 Aug 28 01:15:47.003646 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 668/1565] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:46 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:48.015500 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0dac1ea5-1425-483d-a412-584e1efe9586 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:48.017736 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0dac1ea5-1425-483d-a412-584e1efe9586 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:48.024577 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0dac1ea5-1425-483d-a412-584e1efe9586 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:48.025159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0dac1ea5-1425-483d-a412-584e1efe9586 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:48.025588 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0dac1ea5-1425-483d-a412-584e1efe9586 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:48.026286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0dac1ea5-1425-483d-a412-584e1efe9586 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:48.026726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0dac1ea5-1425-483d-a412-584e1efe9586 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:48.027139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0dac1ea5-1425-483d-a412-584e1efe9586 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:48.059232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0dac1ea5-1425-483d-a412-584e1efe9586 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047867 Aug 28 01:15:48.059814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 898/1566] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:48 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:49.071630 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5dcb8f65-cea4-47e9-b227-6d3147ec571b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:49.073771 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5dcb8f65-cea4-47e9-b227-6d3147ec571b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:49.080407 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5dcb8f65-cea4-47e9-b227-6d3147ec571b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:49.080871 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5dcb8f65-cea4-47e9-b227-6d3147ec571b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:49.081348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5dcb8f65-cea4-47e9-b227-6d3147ec571b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:49.082038 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5dcb8f65-cea4-47e9-b227-6d3147ec571b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:49.082552 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5dcb8f65-cea4-47e9-b227-6d3147ec571b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:49.082997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5dcb8f65-cea4-47e9-b227-6d3147ec571b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:49.116857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5dcb8f65-cea4-47e9-b227-6d3147ec571b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049577 Aug 28 01:15:49.117669 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 669/1567] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:49 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:50.129952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e11277ce-9f88-4c9a-9ad1-d6bfa08988de tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:50.132124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e11277ce-9f88-4c9a-9ad1-d6bfa08988de tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:50.139348 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e11277ce-9f88-4c9a-9ad1-d6bfa08988de tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:50.139843 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e11277ce-9f88-4c9a-9ad1-d6bfa08988de tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:50.140271 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e11277ce-9f88-4c9a-9ad1-d6bfa08988de tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:50.141049 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e11277ce-9f88-4c9a-9ad1-d6bfa08988de tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:50.141646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e11277ce-9f88-4c9a-9ad1-d6bfa08988de tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:50.142074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e11277ce-9f88-4c9a-9ad1-d6bfa08988de tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:50.179157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e11277ce-9f88-4c9a-9ad1-d6bfa08988de tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053300 Aug 28 01:15:50.179694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 899/1568] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:50 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:51.194651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-53f11fbd-686a-4023-bf64-d4992bd0792c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:51.196829 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-53f11fbd-686a-4023-bf64-d4992bd0792c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:51.204121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53f11fbd-686a-4023-bf64-d4992bd0792c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:51.204605 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53f11fbd-686a-4023-bf64-d4992bd0792c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:51.205086 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53f11fbd-686a-4023-bf64-d4992bd0792c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:51.205800 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53f11fbd-686a-4023-bf64-d4992bd0792c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:51.206251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53f11fbd-686a-4023-bf64-d4992bd0792c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:51.206654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53f11fbd-686a-4023-bf64-d4992bd0792c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:51.239658 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-53f11fbd-686a-4023-bf64-d4992bd0792c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049520 Aug 28 01:15:51.240247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 670/1569] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:51 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:52.254666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2aabf680-5e22-4faf-96ca-6c06ce52ca62 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:52.257575 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2aabf680-5e22-4faf-96ca-6c06ce52ca62 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:52.266561 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2aabf680-5e22-4faf-96ca-6c06ce52ca62 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:52.267229 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2aabf680-5e22-4faf-96ca-6c06ce52ca62 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:52.267815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2aabf680-5e22-4faf-96ca-6c06ce52ca62 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:52.268760 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2aabf680-5e22-4faf-96ca-6c06ce52ca62 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:52.269426 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2aabf680-5e22-4faf-96ca-6c06ce52ca62 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:52.269987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2aabf680-5e22-4faf-96ca-6c06ce52ca62 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:52.301101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2aabf680-5e22-4faf-96ca-6c06ce52ca62 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051938 Aug 28 01:15:52.301677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 900/1570] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:52 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:53.315491 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d0780378-5b93-423e-8e39-3fb9c164dfd5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:53.317606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d0780378-5b93-423e-8e39-3fb9c164dfd5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:53.324686 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0780378-5b93-423e-8e39-3fb9c164dfd5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:53.325208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0780378-5b93-423e-8e39-3fb9c164dfd5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:53.325645 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0780378-5b93-423e-8e39-3fb9c164dfd5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:53.326350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0780378-5b93-423e-8e39-3fb9c164dfd5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:53.326801 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0780378-5b93-423e-8e39-3fb9c164dfd5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:53.327232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0780378-5b93-423e-8e39-3fb9c164dfd5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:53.357772 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d0780378-5b93-423e-8e39-3fb9c164dfd5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046323 Aug 28 01:15:53.358342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 671/1571] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:53 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:54.370628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-980142ea-682b-4d23-81f4-67343655a442 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:54.372713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-980142ea-682b-4d23-81f4-67343655a442 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:54.379603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-980142ea-682b-4d23-81f4-67343655a442 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:54.380086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-980142ea-682b-4d23-81f4-67343655a442 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:54.380525 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-980142ea-682b-4d23-81f4-67343655a442 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:54.381266 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-980142ea-682b-4d23-81f4-67343655a442 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:54.381723 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-980142ea-682b-4d23-81f4-67343655a442 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:54.382127 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-980142ea-682b-4d23-81f4-67343655a442 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:54.413625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-980142ea-682b-4d23-81f4-67343655a442 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047050 Aug 28 01:15:54.414181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 901/1572] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:54 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:55.427309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cdc8638a-4dbc-4dd4-a066-8054b8e941ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:55.429522 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cdc8638a-4dbc-4dd4-a066-8054b8e941ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:55.431948 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=30 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:15:55.437631 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cdc8638a-4dbc-4dd4-a066-8054b8e941ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:55.438111 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cdc8638a-4dbc-4dd4-a066-8054b8e941ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:55.438535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cdc8638a-4dbc-4dd4-a066-8054b8e941ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:55.439240 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cdc8638a-4dbc-4dd4-a066-8054b8e941ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:55.439681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cdc8638a-4dbc-4dd4-a066-8054b8e941ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:55.440096 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cdc8638a-4dbc-4dd4-a066-8054b8e941ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:55.472194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cdc8638a-4dbc-4dd4-a066-8054b8e941ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049647 Aug 28 01:15:55.472805 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 672/1573] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:55 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:56.485943 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-42bffd0f-b16a-4aa6-a032-f89f2d1d4582 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:56.488026 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-42bffd0f-b16a-4aa6-a032-f89f2d1d4582 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:56.491008 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=30 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:15:56.496414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42bffd0f-b16a-4aa6-a032-f89f2d1d4582 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:56.496894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42bffd0f-b16a-4aa6-a032-f89f2d1d4582 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:56.497364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42bffd0f-b16a-4aa6-a032-f89f2d1d4582 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:56.498072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42bffd0f-b16a-4aa6-a032-f89f2d1d4582 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:56.498517 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42bffd0f-b16a-4aa6-a032-f89f2d1d4582 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:56.498938 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42bffd0f-b16a-4aa6-a032-f89f2d1d4582 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:56.530256 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-42bffd0f-b16a-4aa6-a032-f89f2d1d4582 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048442 Aug 28 01:15:56.530819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 902/1574] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:56 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:57.543435 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-73fe94d7-d5c2-4717-87d5-3a3f5230390a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:57.545504 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-73fe94d7-d5c2-4717-87d5-3a3f5230390a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:57.552996 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73fe94d7-d5c2-4717-87d5-3a3f5230390a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:57.553494 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73fe94d7-d5c2-4717-87d5-3a3f5230390a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:57.553930 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73fe94d7-d5c2-4717-87d5-3a3f5230390a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:57.554621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73fe94d7-d5c2-4717-87d5-3a3f5230390a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:57.555067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73fe94d7-d5c2-4717-87d5-3a3f5230390a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:57.555492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73fe94d7-d5c2-4717-87d5-3a3f5230390a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:57.587832 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-73fe94d7-d5c2-4717-87d5-3a3f5230390a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048494 Aug 28 01:15:57.588372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 673/1575] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:57 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:58.606129 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-71866e79-3e19-49ab-a27c-72a53a0f758d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:58.610384 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-71866e79-3e19-49ab-a27c-72a53a0f758d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:58.617562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-71866e79-3e19-49ab-a27c-72a53a0f758d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:58.618050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-71866e79-3e19-49ab-a27c-72a53a0f758d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:58.618506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-71866e79-3e19-49ab-a27c-72a53a0f758d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:58.620222 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-71866e79-3e19-49ab-a27c-72a53a0f758d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:58.620222 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-71866e79-3e19-49ab-a27c-72a53a0f758d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:58.620614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-71866e79-3e19-49ab-a27c-72a53a0f758d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:58.651131 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-71866e79-3e19-49ab-a27c-72a53a0f758d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053860 Aug 28 01:15:58.651982 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 903/1576] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:58 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:15:59.663478 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8ebe51b0-aeb0-491a-bd42-87c486914f3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:15:59.665629 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8ebe51b0-aeb0-491a-bd42-87c486914f3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:15:59.672444 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ebe51b0-aeb0-491a-bd42-87c486914f3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:59.672924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ebe51b0-aeb0-491a-bd42-87c486914f3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:59.673374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ebe51b0-aeb0-491a-bd42-87c486914f3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:59.674073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ebe51b0-aeb0-491a-bd42-87c486914f3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:15:59.674517 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ebe51b0-aeb0-491a-bd42-87c486914f3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:15:59.674931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ebe51b0-aeb0-491a-bd42-87c486914f3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:15:59.709218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8ebe51b0-aeb0-491a-bd42-87c486914f3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049748 Aug 28 01:15:59.709824 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 674/1577] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:15:59 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:00.723112 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a0b28d5f-e6b3-4bda-819f-ace05e8ead92 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:00.725216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a0b28d5f-e6b3-4bda-819f-ace05e8ead92 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:00.732760 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a0b28d5f-e6b3-4bda-819f-ace05e8ead92 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:00.733243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a0b28d5f-e6b3-4bda-819f-ace05e8ead92 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:00.733702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a0b28d5f-e6b3-4bda-819f-ace05e8ead92 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:00.734401 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a0b28d5f-e6b3-4bda-819f-ace05e8ead92 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:00.734833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a0b28d5f-e6b3-4bda-819f-ace05e8ead92 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:00.735267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a0b28d5f-e6b3-4bda-819f-ace05e8ead92 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:00.766939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a0b28d5f-e6b3-4bda-819f-ace05e8ead92 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047965 Aug 28 01:16:00.767513 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 904/1578] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:00 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:01.779752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fbc8fbed-6961-42b5-870f-fdab1ec3a34b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:01.782758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fbc8fbed-6961-42b5-870f-fdab1ec3a34b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:01.789796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fbc8fbed-6961-42b5-870f-fdab1ec3a34b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:01.790332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fbc8fbed-6961-42b5-870f-fdab1ec3a34b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:01.790776 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fbc8fbed-6961-42b5-870f-fdab1ec3a34b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:01.791487 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fbc8fbed-6961-42b5-870f-fdab1ec3a34b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:01.791943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fbc8fbed-6961-42b5-870f-fdab1ec3a34b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:01.792343 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fbc8fbed-6961-42b5-870f-fdab1ec3a34b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:01.823656 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fbc8fbed-6961-42b5-870f-fdab1ec3a34b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047876 Aug 28 01:16:01.824202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 675/1579] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:01 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:02.838623 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-47c115f1-c578-4ad4-91b3-de8c0451cdc5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:02.840675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-47c115f1-c578-4ad4-91b3-de8c0451cdc5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:02.847540 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47c115f1-c578-4ad4-91b3-de8c0451cdc5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:02.848019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47c115f1-c578-4ad4-91b3-de8c0451cdc5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:02.848445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47c115f1-c578-4ad4-91b3-de8c0451cdc5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:02.849171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47c115f1-c578-4ad4-91b3-de8c0451cdc5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:02.849615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47c115f1-c578-4ad4-91b3-de8c0451cdc5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:02.850034 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47c115f1-c578-4ad4-91b3-de8c0451cdc5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:02.880279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-47c115f1-c578-4ad4-91b3-de8c0451cdc5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047558 Aug 28 01:16:02.880838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 905/1580] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:02 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:03.892248 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6206cd70-0c59-41fb-a007-1fb9bd848fc3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:03.894472 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6206cd70-0c59-41fb-a007-1fb9bd848fc3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:03.901760 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6206cd70-0c59-41fb-a007-1fb9bd848fc3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:03.902156 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6206cd70-0c59-41fb-a007-1fb9bd848fc3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:03.902583 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6206cd70-0c59-41fb-a007-1fb9bd848fc3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:03.903279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6206cd70-0c59-41fb-a007-1fb9bd848fc3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:03.903701 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6206cd70-0c59-41fb-a007-1fb9bd848fc3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:03.904124 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6206cd70-0c59-41fb-a007-1fb9bd848fc3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:03.936489 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6206cd70-0c59-41fb-a007-1fb9bd848fc3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048171 Aug 28 01:16:03.937089 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 676/1581] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:03 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:04.950781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-20dc0acf-d8d7-422e-a988-a25d4202ff8f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:04.952932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-20dc0acf-d8d7-422e-a988-a25d4202ff8f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:04.959375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-20dc0acf-d8d7-422e-a988-a25d4202ff8f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:04.959856 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-20dc0acf-d8d7-422e-a988-a25d4202ff8f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:04.960293 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-20dc0acf-d8d7-422e-a988-a25d4202ff8f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:04.960999 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-20dc0acf-d8d7-422e-a988-a25d4202ff8f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:04.961478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-20dc0acf-d8d7-422e-a988-a25d4202ff8f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:04.961902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-20dc0acf-d8d7-422e-a988-a25d4202ff8f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:04.992295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-20dc0acf-d8d7-422e-a988-a25d4202ff8f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045913 Aug 28 01:16:04.992765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 906/1582] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:04 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:06.005214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-95f37a3a-7272-4fe9-a8ea-a931ca565b39 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:06.007599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-95f37a3a-7272-4fe9-a8ea-a931ca565b39 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:06.015135 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95f37a3a-7272-4fe9-a8ea-a931ca565b39 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:06.015621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95f37a3a-7272-4fe9-a8ea-a931ca565b39 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:06.016053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95f37a3a-7272-4fe9-a8ea-a931ca565b39 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:06.016758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95f37a3a-7272-4fe9-a8ea-a931ca565b39 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:06.017228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95f37a3a-7272-4fe9-a8ea-a931ca565b39 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:06.017652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95f37a3a-7272-4fe9-a8ea-a931ca565b39 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:06.048255 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-95f37a3a-7272-4fe9-a8ea-a931ca565b39 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046910 Aug 28 01:16:06.048713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 677/1583] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:05 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:07.061263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6007ac47-cb78-4728-a7fc-db1ab326a286 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:07.063469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6007ac47-cb78-4728-a7fc-db1ab326a286 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:07.071649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6007ac47-cb78-4728-a7fc-db1ab326a286 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:07.072126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6007ac47-cb78-4728-a7fc-db1ab326a286 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:07.072558 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6007ac47-cb78-4728-a7fc-db1ab326a286 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:07.073305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6007ac47-cb78-4728-a7fc-db1ab326a286 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:07.073745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6007ac47-cb78-4728-a7fc-db1ab326a286 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:07.074166 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6007ac47-cb78-4728-a7fc-db1ab326a286 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:07.104462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6007ac47-cb78-4728-a7fc-db1ab326a286 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047304 Aug 28 01:16:07.104921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 907/1584] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:07 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:08.117988 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-36eb55ad-ab17-4edb-9c4d-aff7178f7080 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:08.120806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-36eb55ad-ab17-4edb-9c4d-aff7178f7080 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:08.128429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36eb55ad-ab17-4edb-9c4d-aff7178f7080 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:08.128910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36eb55ad-ab17-4edb-9c4d-aff7178f7080 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:08.129377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36eb55ad-ab17-4edb-9c4d-aff7178f7080 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:08.130066 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36eb55ad-ab17-4edb-9c4d-aff7178f7080 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:08.130532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36eb55ad-ab17-4edb-9c4d-aff7178f7080 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:08.131007 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36eb55ad-ab17-4edb-9c4d-aff7178f7080 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:08.163382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-36eb55ad-ab17-4edb-9c4d-aff7178f7080 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050246 Aug 28 01:16:08.163853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 678/1585] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:08 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:09.178460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a6713cc3-dbc3-48a4-8f29-5f759d56d2ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:09.180277 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a6713cc3-dbc3-48a4-8f29-5f759d56d2ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:09.189759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6713cc3-dbc3-48a4-8f29-5f759d56d2ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:09.190252 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6713cc3-dbc3-48a4-8f29-5f759d56d2ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:09.190771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6713cc3-dbc3-48a4-8f29-5f759d56d2ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:09.191606 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6713cc3-dbc3-48a4-8f29-5f759d56d2ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:09.192190 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6713cc3-dbc3-48a4-8f29-5f759d56d2ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:09.192713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6713cc3-dbc3-48a4-8f29-5f759d56d2ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:09.224126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a6713cc3-dbc3-48a4-8f29-5f759d56d2ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051749 Aug 28 01:16:09.224600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 908/1586] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:09 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:10.236899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e5c8d361-86e5-4edf-a851-b2148d16614e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:10.239126 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e5c8d361-86e5-4edf-a851-b2148d16614e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:10.248071 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e5c8d361-86e5-4edf-a851-b2148d16614e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:10.248560 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e5c8d361-86e5-4edf-a851-b2148d16614e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:10.248990 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e5c8d361-86e5-4edf-a851-b2148d16614e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:10.249740 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e5c8d361-86e5-4edf-a851-b2148d16614e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:10.250195 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e5c8d361-86e5-4edf-a851-b2148d16614e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:10.250621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e5c8d361-86e5-4edf-a851-b2148d16614e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:10.281590 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e5c8d361-86e5-4edf-a851-b2148d16614e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048713 Aug 28 01:16:10.282146 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 679/1587] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:10 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:11.294439 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5aff3f72-ee21-4c25-8389-902694b3bebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:11.296581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5aff3f72-ee21-4c25-8389-902694b3bebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:11.304883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5aff3f72-ee21-4c25-8389-902694b3bebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:11.305402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5aff3f72-ee21-4c25-8389-902694b3bebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:11.305838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5aff3f72-ee21-4c25-8389-902694b3bebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:11.306527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5aff3f72-ee21-4c25-8389-902694b3bebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:11.306992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5aff3f72-ee21-4c25-8389-902694b3bebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:11.307414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5aff3f72-ee21-4c25-8389-902694b3bebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:11.341478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5aff3f72-ee21-4c25-8389-902694b3bebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051091 Aug 28 01:16:11.342076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 909/1588] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:11 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:12.354268 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6c65a927-23c5-4955-9b0d-fd4df176c40e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:12.356503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6c65a927-23c5-4955-9b0d-fd4df176c40e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:12.363648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6c65a927-23c5-4955-9b0d-fd4df176c40e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:12.364131 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6c65a927-23c5-4955-9b0d-fd4df176c40e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:12.364564 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6c65a927-23c5-4955-9b0d-fd4df176c40e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:12.365301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6c65a927-23c5-4955-9b0d-fd4df176c40e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:12.365787 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6c65a927-23c5-4955-9b0d-fd4df176c40e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:12.366191 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6c65a927-23c5-4955-9b0d-fd4df176c40e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:12.396950 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6c65a927-23c5-4955-9b0d-fd4df176c40e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046630 Aug 28 01:16:12.397544 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 680/1589] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:12 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:13.409808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b8d6ab55-2dad-4ea7-9098-06a1e950fb87 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:13.411964 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b8d6ab55-2dad-4ea7-9098-06a1e950fb87 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:13.418943 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8d6ab55-2dad-4ea7-9098-06a1e950fb87 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:13.419431 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8d6ab55-2dad-4ea7-9098-06a1e950fb87 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:13.419864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8d6ab55-2dad-4ea7-9098-06a1e950fb87 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:13.420565 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8d6ab55-2dad-4ea7-9098-06a1e950fb87 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:13.421079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8d6ab55-2dad-4ea7-9098-06a1e950fb87 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:13.421500 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8d6ab55-2dad-4ea7-9098-06a1e950fb87 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:13.423981 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=90 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:16:13.453758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b8d6ab55-2dad-4ea7-9098-06a1e950fb87 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047891 Aug 28 01:16:13.454329 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 910/1590] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:13 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:14.466386 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-95a05233-f0a2-450f-aa78-839d7a17586c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:14.468557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-95a05233-f0a2-450f-aa78-839d7a17586c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:14.475633 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95a05233-f0a2-450f-aa78-839d7a17586c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:14.476115 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95a05233-f0a2-450f-aa78-839d7a17586c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:14.476576 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95a05233-f0a2-450f-aa78-839d7a17586c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:14.477283 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95a05233-f0a2-450f-aa78-839d7a17586c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:14.477735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95a05233-f0a2-450f-aa78-839d7a17586c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:14.478170 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95a05233-f0a2-450f-aa78-839d7a17586c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:14.511329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-95a05233-f0a2-450f-aa78-839d7a17586c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049013 Aug 28 01:16:14.511922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 681/1591] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:14 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:15.524948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4a747495-3d94-43b5-89cf-dd090fdec6f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:15.527158 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4a747495-3d94-43b5-89cf-dd090fdec6f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:15.535500 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a747495-3d94-43b5-89cf-dd090fdec6f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:15.536001 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a747495-3d94-43b5-89cf-dd090fdec6f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:15.536413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a747495-3d94-43b5-89cf-dd090fdec6f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:15.537160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a747495-3d94-43b5-89cf-dd090fdec6f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:15.537603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a747495-3d94-43b5-89cf-dd090fdec6f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:15.538020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a747495-3d94-43b5-89cf-dd090fdec6f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:15.569658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4a747495-3d94-43b5-89cf-dd090fdec6f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048859 Aug 28 01:16:15.570232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 911/1592] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:15 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:16.582998 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4e8f3514-b8b7-409d-9cb6-a6f0cba41e36 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:16.585443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4e8f3514-b8b7-409d-9cb6-a6f0cba41e36 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:16.592443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e8f3514-b8b7-409d-9cb6-a6f0cba41e36 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:16.592933 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e8f3514-b8b7-409d-9cb6-a6f0cba41e36 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:16.593402 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e8f3514-b8b7-409d-9cb6-a6f0cba41e36 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:16.594107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e8f3514-b8b7-409d-9cb6-a6f0cba41e36 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:16.594561 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e8f3514-b8b7-409d-9cb6-a6f0cba41e36 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:16.594982 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e8f3514-b8b7-409d-9cb6-a6f0cba41e36 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:16.628323 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4e8f3514-b8b7-409d-9cb6-a6f0cba41e36 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049661 Aug 28 01:16:16.628913 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 682/1593] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:16 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:17.642734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ee7f654d-e409-412f-914e-19708b3123c4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:17.645047 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ee7f654d-e409-412f-914e-19708b3123c4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:17.651542 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ee7f654d-e409-412f-914e-19708b3123c4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:17.652012 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ee7f654d-e409-412f-914e-19708b3123c4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:17.652443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ee7f654d-e409-412f-914e-19708b3123c4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:17.653259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ee7f654d-e409-412f-914e-19708b3123c4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:17.653984 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ee7f654d-e409-412f-914e-19708b3123c4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:17.654103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ee7f654d-e409-412f-914e-19708b3123c4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:17.685290 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ee7f654d-e409-412f-914e-19708b3123c4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047839 Aug 28 01:16:17.685849 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 912/1594] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:17 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:18.698264 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-580bfe8e-6f09-435d-8b7e-1ac4fe46fa90 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:18.700501 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-580bfe8e-6f09-435d-8b7e-1ac4fe46fa90 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:18.707895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-580bfe8e-6f09-435d-8b7e-1ac4fe46fa90 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:18.708381 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-580bfe8e-6f09-435d-8b7e-1ac4fe46fa90 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:18.708811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-580bfe8e-6f09-435d-8b7e-1ac4fe46fa90 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:18.709568 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-580bfe8e-6f09-435d-8b7e-1ac4fe46fa90 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:18.710015 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-580bfe8e-6f09-435d-8b7e-1ac4fe46fa90 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:18.710432 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-580bfe8e-6f09-435d-8b7e-1ac4fe46fa90 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:18.712841 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=90 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:16:18.741802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-580bfe8e-6f09-435d-8b7e-1ac4fe46fa90 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047830 Aug 28 01:16:18.742358 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 683/1595] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:18 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:19.754372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-24bf951b-6cdb-4e6f-acf1-b6b0fb956e6b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:19.756521 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-24bf951b-6cdb-4e6f-acf1-b6b0fb956e6b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:19.763265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24bf951b-6cdb-4e6f-acf1-b6b0fb956e6b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:19.763751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24bf951b-6cdb-4e6f-acf1-b6b0fb956e6b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:19.764185 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24bf951b-6cdb-4e6f-acf1-b6b0fb956e6b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:19.764883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24bf951b-6cdb-4e6f-acf1-b6b0fb956e6b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:19.765361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24bf951b-6cdb-4e6f-acf1-b6b0fb956e6b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:19.765766 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24bf951b-6cdb-4e6f-acf1-b6b0fb956e6b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:19.795514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-24bf951b-6cdb-4e6f-acf1-b6b0fb956e6b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045168 Aug 28 01:16:19.796078 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 913/1596] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:19 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:16:20.808008 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6bfb5bec-2eb5-45c1-bbf7-3d5686d227be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:20.810260 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6bfb5bec-2eb5-45c1-bbf7-3d5686d227be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:20.816893 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6bfb5bec-2eb5-45c1-bbf7-3d5686d227be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:20.817448 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6bfb5bec-2eb5-45c1-bbf7-3d5686d227be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:20.817887 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6bfb5bec-2eb5-45c1-bbf7-3d5686d227be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:20.818925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6bfb5bec-2eb5-45c1-bbf7-3d5686d227be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:20.819377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6bfb5bec-2eb5-45c1-bbf7-3d5686d227be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:20.819796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6bfb5bec-2eb5-45c1-bbf7-3d5686d227be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:20.850707 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6bfb5bec-2eb5-45c1-bbf7-3d5686d227be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046649 Aug 28 01:16:20.851285 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 684/1597] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:20 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:21.863570 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-90140d13-6abf-44fa-b44b-0d2bebf55757 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:21.865900 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-90140d13-6abf-44fa-b44b-0d2bebf55757 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:21.872987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90140d13-6abf-44fa-b44b-0d2bebf55757 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:21.873479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90140d13-6abf-44fa-b44b-0d2bebf55757 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:21.873915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90140d13-6abf-44fa-b44b-0d2bebf55757 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:21.874616 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90140d13-6abf-44fa-b44b-0d2bebf55757 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:21.875069 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90140d13-6abf-44fa-b44b-0d2bebf55757 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:21.875486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90140d13-6abf-44fa-b44b-0d2bebf55757 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:21.911205 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-90140d13-6abf-44fa-b44b-0d2bebf55757 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051494 Aug 28 01:16:21.911808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 914/1598] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:21 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:22.923868 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-de2e820e-22f2-434c-b4ba-55c618de5cdb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:22.926145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-de2e820e-22f2-434c-b4ba-55c618de5cdb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:22.932770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de2e820e-22f2-434c-b4ba-55c618de5cdb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:22.933294 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de2e820e-22f2-434c-b4ba-55c618de5cdb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:22.933738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de2e820e-22f2-434c-b4ba-55c618de5cdb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:22.934490 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de2e820e-22f2-434c-b4ba-55c618de5cdb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:22.934943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de2e820e-22f2-434c-b4ba-55c618de5cdb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:22.935369 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de2e820e-22f2-434c-b4ba-55c618de5cdb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:22.967327 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-de2e820e-22f2-434c-b4ba-55c618de5cdb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047456 Aug 28 01:16:22.967892 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 685/1599] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:22 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:23.984391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4ef6694a-1aa0-45bc-83af-1aa318be96d6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:23.987567 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4ef6694a-1aa0-45bc-83af-1aa318be96d6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:23.996015 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ef6694a-1aa0-45bc-83af-1aa318be96d6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:23.996602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ef6694a-1aa0-45bc-83af-1aa318be96d6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:23.997084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ef6694a-1aa0-45bc-83af-1aa318be96d6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:23.997909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ef6694a-1aa0-45bc-83af-1aa318be96d6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:23.998536 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ef6694a-1aa0-45bc-83af-1aa318be96d6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:23.998965 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ef6694a-1aa0-45bc-83af-1aa318be96d6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:24.034484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4ef6694a-1aa0-45bc-83af-1aa318be96d6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054601 Aug 28 01:16:24.035400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 915/1600] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:23 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:25.047215 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-add4b05a-d4cf-4811-be7f-23912c5d7489 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:25.049543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-add4b05a-d4cf-4811-be7f-23912c5d7489 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:25.056770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-add4b05a-d4cf-4811-be7f-23912c5d7489 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:25.057357 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-add4b05a-d4cf-4811-be7f-23912c5d7489 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:25.057797 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-add4b05a-d4cf-4811-be7f-23912c5d7489 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:25.058499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-add4b05a-d4cf-4811-be7f-23912c5d7489 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:25.058977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-add4b05a-d4cf-4811-be7f-23912c5d7489 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:25.059583 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-add4b05a-d4cf-4811-be7f-23912c5d7489 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:25.091588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-add4b05a-d4cf-4811-be7f-23912c5d7489 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048284 Aug 28 01:16:25.092157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 686/1601] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:25 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:26.104976 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b7b1347d-355a-4674-8baf-d6505695a84c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:26.107338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b7b1347d-355a-4674-8baf-d6505695a84c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:26.114518 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7b1347d-355a-4674-8baf-d6505695a84c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:26.115009 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7b1347d-355a-4674-8baf-d6505695a84c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:26.115463 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7b1347d-355a-4674-8baf-d6505695a84c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:26.116197 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7b1347d-355a-4674-8baf-d6505695a84c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:26.116655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7b1347d-355a-4674-8baf-d6505695a84c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:26.117120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7b1347d-355a-4674-8baf-d6505695a84c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:26.150959 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b7b1347d-355a-4674-8baf-d6505695a84c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049921 Aug 28 01:16:26.151550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 916/1602] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:26 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:27.166065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-634d9834-b5e8-4d9d-a5ad-fac6b61bb95a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:27.168352 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-634d9834-b5e8-4d9d-a5ad-fac6b61bb95a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:27.170802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=30 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:16:27.176703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-634d9834-b5e8-4d9d-a5ad-fac6b61bb95a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:27.177217 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-634d9834-b5e8-4d9d-a5ad-fac6b61bb95a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:27.177648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-634d9834-b5e8-4d9d-a5ad-fac6b61bb95a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:27.178357 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-634d9834-b5e8-4d9d-a5ad-fac6b61bb95a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:27.178802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-634d9834-b5e8-4d9d-a5ad-fac6b61bb95a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:27.179220 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-634d9834-b5e8-4d9d-a5ad-fac6b61bb95a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:27.210198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-634d9834-b5e8-4d9d-a5ad-fac6b61bb95a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048646 Aug 28 01:16:27.210761 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 687/1603] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:27 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:28.223376 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-49982858-6713-467a-aa29-cde68f449c7d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:28.225689 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-49982858-6713-467a-aa29-cde68f449c7d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:28.228795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=30 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:16:28.234578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49982858-6713-467a-aa29-cde68f449c7d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:28.235054 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49982858-6713-467a-aa29-cde68f449c7d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:28.235478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49982858-6713-467a-aa29-cde68f449c7d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:28.236172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49982858-6713-467a-aa29-cde68f449c7d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:28.236608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49982858-6713-467a-aa29-cde68f449c7d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:28.237053 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49982858-6713-467a-aa29-cde68f449c7d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:28.269161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-49982858-6713-467a-aa29-cde68f449c7d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049993 Aug 28 01:16:28.269988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 917/1604] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:28 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:29.284077 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cb5a2784-3e53-4070-b6bc-e321fea77e27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:29.287028 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cb5a2784-3e53-4070-b6bc-e321fea77e27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:29.294762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb5a2784-3e53-4070-b6bc-e321fea77e27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:29.295273 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb5a2784-3e53-4070-b6bc-e321fea77e27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:29.295698 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb5a2784-3e53-4070-b6bc-e321fea77e27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:29.296409 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb5a2784-3e53-4070-b6bc-e321fea77e27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:29.296847 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb5a2784-3e53-4070-b6bc-e321fea77e27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:29.297300 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb5a2784-3e53-4070-b6bc-e321fea77e27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:29.327164 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cb5a2784-3e53-4070-b6bc-e321fea77e27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049024 Aug 28 01:16:29.327711 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 688/1605] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:29 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:30.341877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cb80882e-0c86-4582-ad25-906378e78dbd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:30.344033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cb80882e-0c86-4582-ad25-906378e78dbd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:30.351008 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cb80882e-0c86-4582-ad25-906378e78dbd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:30.351487 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cb80882e-0c86-4582-ad25-906378e78dbd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:30.351919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cb80882e-0c86-4582-ad25-906378e78dbd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:30.352607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cb80882e-0c86-4582-ad25-906378e78dbd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:30.353058 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cb80882e-0c86-4582-ad25-906378e78dbd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:30.353491 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cb80882e-0c86-4582-ad25-906378e78dbd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:30.382720 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cb80882e-0c86-4582-ad25-906378e78dbd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044873 Aug 28 01:16:30.383267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 918/1606] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:30 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:31.395073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d06280ba-7c95-4f49-b57e-b9adb08828f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:31.397265 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d06280ba-7c95-4f49-b57e-b9adb08828f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:31.403880 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d06280ba-7c95-4f49-b57e-b9adb08828f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:31.404395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d06280ba-7c95-4f49-b57e-b9adb08828f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:31.404926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d06280ba-7c95-4f49-b57e-b9adb08828f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:31.405577 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d06280ba-7c95-4f49-b57e-b9adb08828f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:31.406014 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d06280ba-7c95-4f49-b57e-b9adb08828f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:31.406489 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d06280ba-7c95-4f49-b57e-b9adb08828f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:31.438216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d06280ba-7c95-4f49-b57e-b9adb08828f1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047014 Aug 28 01:16:31.438827 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 689/1607] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:31 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:32.450805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-881a9656-110c-491c-ab9e-7f2e9fb00d33 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:32.453154 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-881a9656-110c-491c-ab9e-7f2e9fb00d33 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:32.460139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-881a9656-110c-491c-ab9e-7f2e9fb00d33 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:32.460139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-881a9656-110c-491c-ab9e-7f2e9fb00d33 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:32.460702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-881a9656-110c-491c-ab9e-7f2e9fb00d33 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:32.461398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-881a9656-110c-491c-ab9e-7f2e9fb00d33 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:32.461765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-881a9656-110c-491c-ab9e-7f2e9fb00d33 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:32.462186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-881a9656-110c-491c-ab9e-7f2e9fb00d33 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:32.494485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-881a9656-110c-491c-ab9e-7f2e9fb00d33 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047605 Aug 28 01:16:32.495068 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 919/1608] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:32 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:33.513310 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2fed2393-795a-4a21-a202-6bfce00be7bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:33.516106 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2fed2393-795a-4a21-a202-6bfce00be7bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:33.522769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fed2393-795a-4a21-a202-6bfce00be7bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:33.523263 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fed2393-795a-4a21-a202-6bfce00be7bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:33.523694 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fed2393-795a-4a21-a202-6bfce00be7bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:33.524402 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fed2393-795a-4a21-a202-6bfce00be7bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:33.524847 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fed2393-795a-4a21-a202-6bfce00be7bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:33.525295 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fed2393-795a-4a21-a202-6bfce00be7bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:33.556392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2fed2393-795a-4a21-a202-6bfce00be7bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047227 Aug 28 01:16:33.556948 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 690/1609] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:33 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:34.568017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6ebf7471-0134-4e71-bd66-1d6451ba4852 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:34.570607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6ebf7471-0134-4e71-bd66-1d6451ba4852 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:34.578224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ebf7471-0134-4e71-bd66-1d6451ba4852 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:34.579044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ebf7471-0134-4e71-bd66-1d6451ba4852 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:34.579597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ebf7471-0134-4e71-bd66-1d6451ba4852 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:34.580421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ebf7471-0134-4e71-bd66-1d6451ba4852 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:34.581175 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ebf7471-0134-4e71-bd66-1d6451ba4852 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:34.581721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6ebf7471-0134-4e71-bd66-1d6451ba4852 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:34.615074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6ebf7471-0134-4e71-bd66-1d6451ba4852 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051011 Aug 28 01:16:34.615656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 920/1610] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:34 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:35.626845 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-391d982d-2096-4e6b-9c87-66b5ddae90fb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:35.629020 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-391d982d-2096-4e6b-9c87-66b5ddae90fb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:35.635848 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-391d982d-2096-4e6b-9c87-66b5ddae90fb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:35.636333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-391d982d-2096-4e6b-9c87-66b5ddae90fb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:35.636745 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-391d982d-2096-4e6b-9c87-66b5ddae90fb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:35.637462 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-391d982d-2096-4e6b-9c87-66b5ddae90fb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:35.637905 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-391d982d-2096-4e6b-9c87-66b5ddae90fb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:35.638339 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-391d982d-2096-4e6b-9c87-66b5ddae90fb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:35.668676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-391d982d-2096-4e6b-9c87-66b5ddae90fb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045846 Aug 28 01:16:35.669300 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 691/1611] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:35 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:36.680010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-934161aa-e1d1-4c50-b45a-bac4951c505b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:36.682321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-934161aa-e1d1-4c50-b45a-bac4951c505b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:36.695992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-934161aa-e1d1-4c50-b45a-bac4951c505b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:36.696907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-934161aa-e1d1-4c50-b45a-bac4951c505b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:36.697688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-934161aa-e1d1-4c50-b45a-bac4951c505b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:36.698986 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-934161aa-e1d1-4c50-b45a-bac4951c505b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:36.699768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-934161aa-e1d1-4c50-b45a-bac4951c505b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:36.700536 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-934161aa-e1d1-4c50-b45a-bac4951c505b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:36.739413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-934161aa-e1d1-4c50-b45a-bac4951c505b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063255 Aug 28 01:16:36.740040 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 921/1612] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:36 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:37.753121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6f3fb9a0-5579-46ab-9111-a52b5def9a4d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:37.755282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6f3fb9a0-5579-46ab-9111-a52b5def9a4d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:37.762107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f3fb9a0-5579-46ab-9111-a52b5def9a4d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:37.762592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f3fb9a0-5579-46ab-9111-a52b5def9a4d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:37.763025 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f3fb9a0-5579-46ab-9111-a52b5def9a4d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:37.763724 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f3fb9a0-5579-46ab-9111-a52b5def9a4d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:37.764172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f3fb9a0-5579-46ab-9111-a52b5def9a4d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:37.764592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f3fb9a0-5579-46ab-9111-a52b5def9a4d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:37.795163 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6f3fb9a0-5579-46ab-9111-a52b5def9a4d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046037 Aug 28 01:16:37.796069 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 692/1613] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:37 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:38.807822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-305b3934-ab7a-4e71-96a3-5b8fffdb41c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:38.810395 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-305b3934-ab7a-4e71-96a3-5b8fffdb41c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:38.819589 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-305b3934-ab7a-4e71-96a3-5b8fffdb41c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:38.820069 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-305b3934-ab7a-4e71-96a3-5b8fffdb41c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:38.820503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-305b3934-ab7a-4e71-96a3-5b8fffdb41c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:38.821232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-305b3934-ab7a-4e71-96a3-5b8fffdb41c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:38.821675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-305b3934-ab7a-4e71-96a3-5b8fffdb41c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:38.822086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-305b3934-ab7a-4e71-96a3-5b8fffdb41c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:38.857098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-305b3934-ab7a-4e71-96a3-5b8fffdb41c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053327 Aug 28 01:16:38.857742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 922/1614] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:38 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:39.870230 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-51494cd3-ec17-4d20-89ed-fc7967ba33d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:39.873029 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-51494cd3-ec17-4d20-89ed-fc7967ba33d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:39.885354 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-51494cd3-ec17-4d20-89ed-fc7967ba33d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:39.886217 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-51494cd3-ec17-4d20-89ed-fc7967ba33d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:39.887059 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-51494cd3-ec17-4d20-89ed-fc7967ba33d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:39.888296 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-51494cd3-ec17-4d20-89ed-fc7967ba33d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:39.889244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-51494cd3-ec17-4d20-89ed-fc7967ba33d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:39.889997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-51494cd3-ec17-4d20-89ed-fc7967ba33d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:39.923836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-51494cd3-ec17-4d20-89ed-fc7967ba33d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058168 Aug 28 01:16:39.924476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 693/1615] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:39 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:40.937357 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-973350e9-f55d-43f8-af08-3961c76a41c3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:40.939549 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-973350e9-f55d-43f8-af08-3961c76a41c3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:40.946704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-973350e9-f55d-43f8-af08-3961c76a41c3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:40.947192 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-973350e9-f55d-43f8-af08-3961c76a41c3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:40.947623 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-973350e9-f55d-43f8-af08-3961c76a41c3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:40.948313 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-973350e9-f55d-43f8-af08-3961c76a41c3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:40.948778 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-973350e9-f55d-43f8-af08-3961c76a41c3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:40.949223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-973350e9-f55d-43f8-af08-3961c76a41c3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:40.983289 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-973350e9-f55d-43f8-af08-3961c76a41c3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051345 Aug 28 01:16:40.983882 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 923/1616] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:40 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:41.996638 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0ecc151e-21a5-4479-b2b5-7385abdf1035 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:41.998933 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0ecc151e-21a5-4479-b2b5-7385abdf1035 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:42.006068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0ecc151e-21a5-4479-b2b5-7385abdf1035 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:42.006554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0ecc151e-21a5-4479-b2b5-7385abdf1035 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:42.006990 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0ecc151e-21a5-4479-b2b5-7385abdf1035 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:42.007717 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0ecc151e-21a5-4479-b2b5-7385abdf1035 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:42.008150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0ecc151e-21a5-4479-b2b5-7385abdf1035 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:42.008571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0ecc151e-21a5-4479-b2b5-7385abdf1035 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:42.039781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0ecc151e-21a5-4479-b2b5-7385abdf1035 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047854 Aug 28 01:16:42.040549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 694/1617] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:41 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:43.053006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-441fe133-6e3d-43a3-b741-c562bf76ebf9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:43.055306 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-441fe133-6e3d-43a3-b741-c562bf76ebf9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:43.061987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-441fe133-6e3d-43a3-b741-c562bf76ebf9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:43.062477 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-441fe133-6e3d-43a3-b741-c562bf76ebf9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:43.062909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-441fe133-6e3d-43a3-b741-c562bf76ebf9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:43.063620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-441fe133-6e3d-43a3-b741-c562bf76ebf9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:43.064095 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-441fe133-6e3d-43a3-b741-c562bf76ebf9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:43.064510 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-441fe133-6e3d-43a3-b741-c562bf76ebf9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:43.098603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-441fe133-6e3d-43a3-b741-c562bf76ebf9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050411 Aug 28 01:16:43.099389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 924/1618] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:43 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:44.112833 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3a7b6562-c34c-4235-8473-d5a50acd3d14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:44.116861 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3a7b6562-c34c-4235-8473-d5a50acd3d14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:44.127157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a7b6562-c34c-4235-8473-d5a50acd3d14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:44.127621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a7b6562-c34c-4235-8473-d5a50acd3d14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:44.128084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a7b6562-c34c-4235-8473-d5a50acd3d14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:44.128767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a7b6562-c34c-4235-8473-d5a50acd3d14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:44.129391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a7b6562-c34c-4235-8473-d5a50acd3d14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:44.129675 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a7b6562-c34c-4235-8473-d5a50acd3d14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:44.164303 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3a7b6562-c34c-4235-8473-d5a50acd3d14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055732 Aug 28 01:16:44.173370 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 695/1619] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:44 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:45.180313 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b9b12ef7-4433-4911-b5d3-82488e0c5729 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:45.182542 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b9b12ef7-4433-4911-b5d3-82488e0c5729 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:45.189849 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9b12ef7-4433-4911-b5d3-82488e0c5729 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:45.190485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9b12ef7-4433-4911-b5d3-82488e0c5729 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:45.190912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9b12ef7-4433-4911-b5d3-82488e0c5729 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:45.191643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9b12ef7-4433-4911-b5d3-82488e0c5729 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:45.192096 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9b12ef7-4433-4911-b5d3-82488e0c5729 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:45.192527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9b12ef7-4433-4911-b5d3-82488e0c5729 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:45.195483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=90 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:16:45.225382 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b9b12ef7-4433-4911-b5d3-82488e0c5729 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049130 Aug 28 01:16:45.225958 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 925/1620] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:45 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:46.242026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0618986b-1218-45b2-a8cc-0ca9453fb27c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:46.245374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0618986b-1218-45b2-a8cc-0ca9453fb27c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:46.256938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0618986b-1218-45b2-a8cc-0ca9453fb27c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:46.257709 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0618986b-1218-45b2-a8cc-0ca9453fb27c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:46.258487 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0618986b-1218-45b2-a8cc-0ca9453fb27c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:46.258901 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0618986b-1218-45b2-a8cc-0ca9453fb27c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:46.259353 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0618986b-1218-45b2-a8cc-0ca9453fb27c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:46.259781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0618986b-1218-45b2-a8cc-0ca9453fb27c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:46.291755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0618986b-1218-45b2-a8cc-0ca9453fb27c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057024 Aug 28 01:16:46.292370 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 696/1621] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:46 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:47.304340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-dcb82f00-2256-4da0-99d8-635f2506431c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:47.306550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-dcb82f00-2256-4da0-99d8-635f2506431c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:47.313459 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dcb82f00-2256-4da0-99d8-635f2506431c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:47.313946 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dcb82f00-2256-4da0-99d8-635f2506431c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:47.314377 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dcb82f00-2256-4da0-99d8-635f2506431c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:47.315084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dcb82f00-2256-4da0-99d8-635f2506431c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:47.315535 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dcb82f00-2256-4da0-99d8-635f2506431c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:47.315948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dcb82f00-2256-4da0-99d8-635f2506431c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:47.349502 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-dcb82f00-2256-4da0-99d8-635f2506431c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049068 Aug 28 01:16:47.350082 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 926/1622] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:47 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:48.363970 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-98dcdedf-a608-4302-a0ed-a5cc6c759214 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:48.367301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-98dcdedf-a608-4302-a0ed-a5cc6c759214 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:48.376465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98dcdedf-a608-4302-a0ed-a5cc6c759214 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:48.377367 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98dcdedf-a608-4302-a0ed-a5cc6c759214 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:48.377860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98dcdedf-a608-4302-a0ed-a5cc6c759214 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:48.379163 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98dcdedf-a608-4302-a0ed-a5cc6c759214 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:48.379760 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98dcdedf-a608-4302-a0ed-a5cc6c759214 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:48.380228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98dcdedf-a608-4302-a0ed-a5cc6c759214 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:48.415864 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-98dcdedf-a608-4302-a0ed-a5cc6c759214 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056448 Aug 28 01:16:48.416446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 697/1623] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:48 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:49.428158 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-216eddad-f9e4-4b3c-a7f6-6c1e032acf6c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:49.430351 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-216eddad-f9e4-4b3c-a7f6-6c1e032acf6c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:49.437808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-216eddad-f9e4-4b3c-a7f6-6c1e032acf6c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:49.438295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-216eddad-f9e4-4b3c-a7f6-6c1e032acf6c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:49.438722 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-216eddad-f9e4-4b3c-a7f6-6c1e032acf6c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:49.439435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-216eddad-f9e4-4b3c-a7f6-6c1e032acf6c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:49.439891 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-216eddad-f9e4-4b3c-a7f6-6c1e032acf6c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:49.440324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-216eddad-f9e4-4b3c-a7f6-6c1e032acf6c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:49.471575 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-216eddad-f9e4-4b3c-a7f6-6c1e032acf6c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047559 Aug 28 01:16:49.472132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 927/1624] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:49 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:50.484375 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4ad7617e-5365-40f7-ba75-00ff92e22a30 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:50.486592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4ad7617e-5365-40f7-ba75-00ff92e22a30 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:50.496523 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ad7617e-5365-40f7-ba75-00ff92e22a30 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:50.497019 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ad7617e-5365-40f7-ba75-00ff92e22a30 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:50.497509 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ad7617e-5365-40f7-ba75-00ff92e22a30 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:50.498425 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ad7617e-5365-40f7-ba75-00ff92e22a30 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:50.498677 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ad7617e-5365-40f7-ba75-00ff92e22a30 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:50.499102 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ad7617e-5365-40f7-ba75-00ff92e22a30 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:50.501635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=90 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:16:50.529988 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4ad7617e-5365-40f7-ba75-00ff92e22a30 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049951 Aug 28 01:16:50.530543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 698/1625] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:50 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:51.542393 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fecc2fb4-6c71-4fcf-a281-88ceb7211963 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:51.545471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fecc2fb4-6c71-4fcf-a281-88ceb7211963 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:51.552778 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fecc2fb4-6c71-4fcf-a281-88ceb7211963 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:51.553316 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fecc2fb4-6c71-4fcf-a281-88ceb7211963 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:51.553770 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fecc2fb4-6c71-4fcf-a281-88ceb7211963 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:51.554478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fecc2fb4-6c71-4fcf-a281-88ceb7211963 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:51.554950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fecc2fb4-6c71-4fcf-a281-88ceb7211963 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:51.555374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fecc2fb4-6c71-4fcf-a281-88ceb7211963 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:51.588388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fecc2fb4-6c71-4fcf-a281-88ceb7211963 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049861 Aug 28 01:16:51.588986 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 928/1626] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:51 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:52.601383 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2c4d0640-a12f-48d7-b3cc-00959272c28a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:52.604042 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2c4d0640-a12f-48d7-b3cc-00959272c28a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:52.610365 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2c4d0640-a12f-48d7-b3cc-00959272c28a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:52.610770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2c4d0640-a12f-48d7-b3cc-00959272c28a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:52.611205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2c4d0640-a12f-48d7-b3cc-00959272c28a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:52.611908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2c4d0640-a12f-48d7-b3cc-00959272c28a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:52.612788 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2c4d0640-a12f-48d7-b3cc-00959272c28a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:52.612788 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2c4d0640-a12f-48d7-b3cc-00959272c28a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:52.644059 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2c4d0640-a12f-48d7-b3cc-00959272c28a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046591 Aug 28 01:16:52.644642 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 699/1627] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:52 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:53.659780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fe360858-4e69-497f-bddb-46e7bcffaac5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:53.662945 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fe360858-4e69-497f-bddb-46e7bcffaac5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:53.672476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe360858-4e69-497f-bddb-46e7bcffaac5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:53.673260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe360858-4e69-497f-bddb-46e7bcffaac5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:53.673901 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe360858-4e69-497f-bddb-46e7bcffaac5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:53.674976 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe360858-4e69-497f-bddb-46e7bcffaac5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:53.675697 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe360858-4e69-497f-bddb-46e7bcffaac5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:53.676327 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe360858-4e69-497f-bddb-46e7bcffaac5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:53.713108 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fe360858-4e69-497f-bddb-46e7bcffaac5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060136 Aug 28 01:16:53.713675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 929/1628] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:53 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:54.727999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2239dc37-be55-476d-adef-0c0743144759 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:54.731273 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2239dc37-be55-476d-adef-0c0743144759 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:54.738589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2239dc37-be55-476d-adef-0c0743144759 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:54.739078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2239dc37-be55-476d-adef-0c0743144759 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:54.739519 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2239dc37-be55-476d-adef-0c0743144759 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:54.740241 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2239dc37-be55-476d-adef-0c0743144759 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:54.740681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2239dc37-be55-476d-adef-0c0743144759 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:54.741165 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2239dc37-be55-476d-adef-0c0743144759 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:54.772357 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2239dc37-be55-476d-adef-0c0743144759 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049998 Aug 28 01:16:54.772863 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 700/1629] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:54 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:55.785667 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-894cae99-bf46-4c87-9b7d-c06f2cc7a5ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:55.788035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-894cae99-bf46-4c87-9b7d-c06f2cc7a5ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:55.794809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-894cae99-bf46-4c87-9b7d-c06f2cc7a5ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:55.795305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-894cae99-bf46-4c87-9b7d-c06f2cc7a5ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:55.795745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-894cae99-bf46-4c87-9b7d-c06f2cc7a5ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:55.796438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-894cae99-bf46-4c87-9b7d-c06f2cc7a5ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:55.796918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-894cae99-bf46-4c87-9b7d-c06f2cc7a5ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:55.797354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-894cae99-bf46-4c87-9b7d-c06f2cc7a5ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:55.829216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-894cae99-bf46-4c87-9b7d-c06f2cc7a5ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048135 Aug 28 01:16:55.829832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 930/1630] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:55 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:56.844097 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-dcc5cd13-b047-4107-8455-49d411d24701 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:56.846557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-dcc5cd13-b047-4107-8455-49d411d24701 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:56.853736 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dcc5cd13-b047-4107-8455-49d411d24701 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:56.854233 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dcc5cd13-b047-4107-8455-49d411d24701 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:56.854677 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dcc5cd13-b047-4107-8455-49d411d24701 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:56.855393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dcc5cd13-b047-4107-8455-49d411d24701 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:56.855842 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dcc5cd13-b047-4107-8455-49d411d24701 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:56.856267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dcc5cd13-b047-4107-8455-49d411d24701 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:56.887318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-dcc5cd13-b047-4107-8455-49d411d24701 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047063 Aug 28 01:16:56.887857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 701/1631] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:56 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:57.900486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-571824b8-2e81-403d-851b-7647f2a8c4dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:57.903455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-571824b8-2e81-403d-851b-7647f2a8c4dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:57.912023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-571824b8-2e81-403d-851b-7647f2a8c4dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:57.912521 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-571824b8-2e81-403d-851b-7647f2a8c4dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:57.913002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-571824b8-2e81-403d-851b-7647f2a8c4dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:57.913721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-571824b8-2e81-403d-851b-7647f2a8c4dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:57.914199 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-571824b8-2e81-403d-851b-7647f2a8c4dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:57.914621 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-571824b8-2e81-403d-851b-7647f2a8c4dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:57.950042 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-571824b8-2e81-403d-851b-7647f2a8c4dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053742 Aug 28 01:16:57.950650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 931/1632] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:57 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:16:58.964222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0e33c669-918b-49a3-b6b1-cb5250163614 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:16:58.967115 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0e33c669-918b-49a3-b6b1-cb5250163614 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:16:58.969642 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=30 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:16:58.975120 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e33c669-918b-49a3-b6b1-cb5250163614 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:58.975598 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e33c669-918b-49a3-b6b1-cb5250163614 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:58.976026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e33c669-918b-49a3-b6b1-cb5250163614 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:58.976743 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e33c669-918b-49a3-b6b1-cb5250163614 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:16:58.977221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e33c669-918b-49a3-b6b1-cb5250163614 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:16:58.977613 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e33c669-918b-49a3-b6b1-cb5250163614 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:16:59.007534 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0e33c669-918b-49a3-b6b1-cb5250163614 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047957 Aug 28 01:16:59.008084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 702/1633] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:16:58 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:17:00.025160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-052140ff-0c9f-4cdb-a601-911bcf5271d7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:00.028467 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-052140ff-0c9f-4cdb-a601-911bcf5271d7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:00.033596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=30 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:17:00.039472 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-052140ff-0c9f-4cdb-a601-911bcf5271d7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:00.040075 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-052140ff-0c9f-4cdb-a601-911bcf5271d7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:00.040623 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-052140ff-0c9f-4cdb-a601-911bcf5271d7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:00.041525 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-052140ff-0c9f-4cdb-a601-911bcf5271d7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:00.042098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-052140ff-0c9f-4cdb-a601-911bcf5271d7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:00.042617 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-052140ff-0c9f-4cdb-a601-911bcf5271d7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:00.074977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-052140ff-0c9f-4cdb-a601-911bcf5271d7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057501 Aug 28 01:17:00.075892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 932/1634] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:17:00 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:17:01.089230 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2480c7de-2c75-4d65-bbee-5c2486fd8873 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:01.091796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2480c7de-2c75-4d65-bbee-5c2486fd8873 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:01.099702 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2480c7de-2c75-4d65-bbee-5c2486fd8873 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:01.100190 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2480c7de-2c75-4d65-bbee-5c2486fd8873 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:01.100639 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2480c7de-2c75-4d65-bbee-5c2486fd8873 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:01.101362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2480c7de-2c75-4d65-bbee-5c2486fd8873 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:01.101820 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2480c7de-2c75-4d65-bbee-5c2486fd8873 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:01.102238 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2480c7de-2c75-4d65-bbee-5c2486fd8873 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:01.134510 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2480c7de-2c75-4d65-bbee-5c2486fd8873 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050097 Aug 28 01:17:01.135078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 703/1635] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:17:01 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:17:02.147717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4cdaa914-f9ca-4e10-931d-d159ed775760 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:02.149969 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4cdaa914-f9ca-4e10-931d-d159ed775760 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:02.156751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4cdaa914-f9ca-4e10-931d-d159ed775760 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:02.157270 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4cdaa914-f9ca-4e10-931d-d159ed775760 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:02.157701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4cdaa914-f9ca-4e10-931d-d159ed775760 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:02.158402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4cdaa914-f9ca-4e10-931d-d159ed775760 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:02.158857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4cdaa914-f9ca-4e10-931d-d159ed775760 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:02.159279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4cdaa914-f9ca-4e10-931d-d159ed775760 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:02.189495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4cdaa914-f9ca-4e10-931d-d159ed775760 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045743 Aug 28 01:17:02.190083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 933/1636] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:17:02 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:17:03.203442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-42d0ec98-b61a-4f0c-bd9c-40fa73a49cc4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:03.206693 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-42d0ec98-b61a-4f0c-bd9c-40fa73a49cc4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:03.213546 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42d0ec98-b61a-4f0c-bd9c-40fa73a49cc4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:03.214026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42d0ec98-b61a-4f0c-bd9c-40fa73a49cc4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:03.214451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42d0ec98-b61a-4f0c-bd9c-40fa73a49cc4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:03.215249 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42d0ec98-b61a-4f0c-bd9c-40fa73a49cc4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:03.215739 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42d0ec98-b61a-4f0c-bd9c-40fa73a49cc4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:03.216155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42d0ec98-b61a-4f0c-bd9c-40fa73a49cc4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:03.249732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-42d0ec98-b61a-4f0c-bd9c-40fa73a49cc4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050747 Aug 28 01:17:03.250230 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 704/1637] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:17:03 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:17:04.264695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-997026fe-d3cd-445a-ad4d-5a9e08672063 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:04.267000 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-997026fe-d3cd-445a-ad4d-5a9e08672063 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:04.273996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-997026fe-d3cd-445a-ad4d-5a9e08672063 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:04.274484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-997026fe-d3cd-445a-ad4d-5a9e08672063 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:04.274913 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-997026fe-d3cd-445a-ad4d-5a9e08672063 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:04.275622 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-997026fe-d3cd-445a-ad4d-5a9e08672063 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:04.276062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-997026fe-d3cd-445a-ad4d-5a9e08672063 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:04.276485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-997026fe-d3cd-445a-ad4d-5a9e08672063 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:04.306448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-997026fe-d3cd-445a-ad4d-5a9e08672063 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046582 Aug 28 01:17:04.307247 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 934/1638] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:17:04 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:17:05.319752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b61d5ca3-e444-4ee5-a638-ff4f4f3f9d3f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:05.321979 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b61d5ca3-e444-4ee5-a638-ff4f4f3f9d3f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:05.328825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b61d5ca3-e444-4ee5-a638-ff4f4f3f9d3f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:05.329336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b61d5ca3-e444-4ee5-a638-ff4f4f3f9d3f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:05.329765 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b61d5ca3-e444-4ee5-a638-ff4f4f3f9d3f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:05.330480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b61d5ca3-e444-4ee5-a638-ff4f4f3f9d3f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:05.330925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b61d5ca3-e444-4ee5-a638-ff4f4f3f9d3f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:05.331336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b61d5ca3-e444-4ee5-a638-ff4f4f3f9d3f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:05.361967 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b61d5ca3-e444-4ee5-a638-ff4f4f3f9d3f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046532 Aug 28 01:17:05.362519 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 705/1639] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:17:05 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:17:06.375079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-614eaa55-cc94-4679-8791-fd8e88fedd24 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:06.377287 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-614eaa55-cc94-4679-8791-fd8e88fedd24 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:06.384241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-614eaa55-cc94-4679-8791-fd8e88fedd24 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:06.384726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-614eaa55-cc94-4679-8791-fd8e88fedd24 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:06.385191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-614eaa55-cc94-4679-8791-fd8e88fedd24 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:06.385892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-614eaa55-cc94-4679-8791-fd8e88fedd24 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:06.386344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-614eaa55-cc94-4679-8791-fd8e88fedd24 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:06.386765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-614eaa55-cc94-4679-8791-fd8e88fedd24 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:06.418039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-614eaa55-cc94-4679-8791-fd8e88fedd24 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047117 Aug 28 01:17:06.418608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 935/1640] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:17:06 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:17:07.430855 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a6095871-0b9a-4c15-8b8b-2da74312d46a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:07.433112 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a6095871-0b9a-4c15-8b8b-2da74312d46a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:07.440200 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a6095871-0b9a-4c15-8b8b-2da74312d46a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:07.440690 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a6095871-0b9a-4c15-8b8b-2da74312d46a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:07.441165 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a6095871-0b9a-4c15-8b8b-2da74312d46a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:07.441879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a6095871-0b9a-4c15-8b8b-2da74312d46a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:07.442330 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a6095871-0b9a-4c15-8b8b-2da74312d46a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:07.442752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a6095871-0b9a-4c15-8b8b-2da74312d46a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:07.474439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a6095871-0b9a-4c15-8b8b-2da74312d46a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047625 Aug 28 01:17:07.478264 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 706/1641] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:17:07 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:17:08.487112 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f473d69f-5ccb-456d-b260-51c771794b64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:08.489273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f473d69f-5ccb-456d-b260-51c771794b64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:08.495823 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f473d69f-5ccb-456d-b260-51c771794b64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:08.496306 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f473d69f-5ccb-456d-b260-51c771794b64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:08.496823 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f473d69f-5ccb-456d-b260-51c771794b64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:08.497509 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f473d69f-5ccb-456d-b260-51c771794b64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:08.497951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f473d69f-5ccb-456d-b260-51c771794b64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:08.498385 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f473d69f-5ccb-456d-b260-51c771794b64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:08.529193 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f473d69f-5ccb-456d-b260-51c771794b64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045591 Aug 28 01:17:08.529332 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 936/1642] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:17:08 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:17:09.541310 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9f2e65b8-67a2-4518-a3e0-79aedb6620da tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:09.543470 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9f2e65b8-67a2-4518-a3e0-79aedb6620da tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:09.550362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f2e65b8-67a2-4518-a3e0-79aedb6620da tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:09.550845 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f2e65b8-67a2-4518-a3e0-79aedb6620da tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:09.551278 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f2e65b8-67a2-4518-a3e0-79aedb6620da tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:09.551969 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f2e65b8-67a2-4518-a3e0-79aedb6620da tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:09.552423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f2e65b8-67a2-4518-a3e0-79aedb6620da tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:09.552833 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f2e65b8-67a2-4518-a3e0-79aedb6620da tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:09.584457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9f2e65b8-67a2-4518-a3e0-79aedb6620da tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047083 Aug 28 01:17:09.585320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 707/1643] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:17:09 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:17:10.598142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-87c6f32a-7744-4371-9097-9a109c3168ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:10.600286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-87c6f32a-7744-4371-9097-9a109c3168ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:10.607542 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87c6f32a-7744-4371-9097-9a109c3168ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:10.608048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87c6f32a-7744-4371-9097-9a109c3168ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:10.608478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87c6f32a-7744-4371-9097-9a109c3168ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:10.609218 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87c6f32a-7744-4371-9097-9a109c3168ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:10.609699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87c6f32a-7744-4371-9097-9a109c3168ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:10.610162 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87c6f32a-7744-4371-9097-9a109c3168ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:10.640398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-87c6f32a-7744-4371-9097-9a109c3168ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046242 Aug 28 01:17:10.640992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 937/1644] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:17:10 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:17:11.653312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-41e61d83-5137-4cfa-b084-8c3e3332a81d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:11.655352 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-41e61d83-5137-4cfa-b084-8c3e3332a81d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:11.662663 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41e61d83-5137-4cfa-b084-8c3e3332a81d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:11.663128 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41e61d83-5137-4cfa-b084-8c3e3332a81d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:11.663569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41e61d83-5137-4cfa-b084-8c3e3332a81d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:11.664260 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41e61d83-5137-4cfa-b084-8c3e3332a81d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:11.664713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41e61d83-5137-4cfa-b084-8c3e3332a81d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:11.665171 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41e61d83-5137-4cfa-b084-8c3e3332a81d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:11.697141 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-41e61d83-5137-4cfa-b084-8c3e3332a81d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047580 Aug 28 01:17:11.697416 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 708/1645] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:17:11 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:17:12.709935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f9a4ba91-1d9e-42b2-b079-3b1d94eb9f28 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:12.712056 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f9a4ba91-1d9e-42b2-b079-3b1d94eb9f28 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:12.718811 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f9a4ba91-1d9e-42b2-b079-3b1d94eb9f28 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:12.719284 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f9a4ba91-1d9e-42b2-b079-3b1d94eb9f28 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:12.719718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f9a4ba91-1d9e-42b2-b079-3b1d94eb9f28 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:12.720438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f9a4ba91-1d9e-42b2-b079-3b1d94eb9f28 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:12.720881 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f9a4ba91-1d9e-42b2-b079-3b1d94eb9f28 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:12.721343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f9a4ba91-1d9e-42b2-b079-3b1d94eb9f28 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:12.752237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f9a4ba91-1d9e-42b2-b079-3b1d94eb9f28 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046303 Aug 28 01:17:12.752819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 938/1646] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:17:12 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:17:13.765085 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f98129f3-450d-4cce-82a0-89ff493855ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:13.767187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f98129f3-450d-4cce-82a0-89ff493855ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:13.774158 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f98129f3-450d-4cce-82a0-89ff493855ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:13.774642 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f98129f3-450d-4cce-82a0-89ff493855ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:13.775082 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f98129f3-450d-4cce-82a0-89ff493855ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:13.775795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f98129f3-450d-4cce-82a0-89ff493855ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:13.776241 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f98129f3-450d-4cce-82a0-89ff493855ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:13.776656 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f98129f3-450d-4cce-82a0-89ff493855ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:13.807080 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f98129f3-450d-4cce-82a0-89ff493855ec tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046109 Aug 28 01:17:13.807659 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 709/1647] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:17:13 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:17:14.820085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-23c7323d-23eb-465f-adf9-1a951f028ea6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:14.822328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-23c7323d-23eb-465f-adf9-1a951f028ea6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:14.829080 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23c7323d-23eb-465f-adf9-1a951f028ea6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:14.829559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23c7323d-23eb-465f-adf9-1a951f028ea6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:14.829987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23c7323d-23eb-465f-adf9-1a951f028ea6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:14.830782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23c7323d-23eb-465f-adf9-1a951f028ea6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:14.831224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23c7323d-23eb-465f-adf9-1a951f028ea6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:14.831625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23c7323d-23eb-465f-adf9-1a951f028ea6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:14.862686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-23c7323d-23eb-465f-adf9-1a951f028ea6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046939 Aug 28 01:17:14.863279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 939/1648] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:17:14 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:17:15.608175 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-19cd610e-334c-4eda-85c9-d633e510c7d8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:15.608695 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-19cd610e-334c-4eda-85c9-d633e510c7d8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:15.615350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-19cd610e-334c-4eda-85c9-d633e510c7d8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:15.615856 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-19cd610e-334c-4eda-85c9-d633e510c7d8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:15.616289 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-19cd610e-334c-4eda-85c9-d633e510c7d8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:15.616995 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-19cd610e-334c-4eda-85c9-d633e510c7d8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:15.617443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-19cd610e-334c-4eda-85c9-d633e510c7d8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:15.617855 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-19cd610e-334c-4eda-85c9-d633e510c7d8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:15.643975 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-19cd610e-334c-4eda-85c9-d633e510c7d8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Going to try to terminate instance {{(pid=120524) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 28 01:17:15.738701 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-19cd610e-334c-4eda-85c9-d633e510c7d8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "DELETE /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 204 len: 0 microversion: 2.1 time: 0.133912 Aug 28 01:17:15.739116 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 710/1649] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 01:17:15 2022] DELETE /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 0 bytes in 135 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 28 01:17:15.747273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a436c4ae-ef82-40e1-915f-1d7d61e59e8c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:15.747883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a436c4ae-ef82-40e1-915f-1d7d61e59e8c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:15.754327 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a436c4ae-ef82-40e1-915f-1d7d61e59e8c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:15.754804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a436c4ae-ef82-40e1-915f-1d7d61e59e8c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:15.755231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a436c4ae-ef82-40e1-915f-1d7d61e59e8c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:15.755923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a436c4ae-ef82-40e1-915f-1d7d61e59e8c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:15.756367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a436c4ae-ef82-40e1-915f-1d7d61e59e8c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:15.756782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a436c4ae-ef82-40e1-915f-1d7d61e59e8c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:15.760990 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=90 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:17:15.877993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f93280d1-d4e0-47af-82cb-97323b883a95 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:17:15.880531 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f93280d1-d4e0-47af-82cb-97323b883a95 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:15.887190 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f93280d1-d4e0-47af-82cb-97323b883a95 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:15.887759 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f93280d1-d4e0-47af-82cb-97323b883a95 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:15.888290 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f93280d1-d4e0-47af-82cb-97323b883a95 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:15.889133 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f93280d1-d4e0-47af-82cb-97323b883a95 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:15.889673 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f93280d1-d4e0-47af-82cb-97323b883a95 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:15.890079 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f93280d1-d4e0-47af-82cb-97323b883a95 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:15.956621 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a436c4ae-ef82-40e1-915f-1d7d61e59e8c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:15.964985 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a436c4ae-ef82-40e1-915f-1d7d61e59e8c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.221530 Aug 28 01:17:15.965554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 940/1650] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:15 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:15.999584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f93280d1-d4e0-47af-82cb-97323b883a95 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "POST /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/action" status: 200 len: 76038 microversion: 2.1 time: 0.126333 Aug 28 01:17:16.000326 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 711/1651] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Aug 28 01:17:15 2022] POST /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2/action => generated 76038 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:17:16.726751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f67f2ba2-5630-40d3-bf26-191e6d98995b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:16.727251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f67f2ba2-5630-40d3-bf26-191e6d98995b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:16.734605 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f67f2ba2-5630-40d3-bf26-191e6d98995b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:16.735091 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f67f2ba2-5630-40d3-bf26-191e6d98995b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:16.735522 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f67f2ba2-5630-40d3-bf26-191e6d98995b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:16.736226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f67f2ba2-5630-40d3-bf26-191e6d98995b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:16.736668 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f67f2ba2-5630-40d3-bf26-191e6d98995b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:16.737110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f67f2ba2-5630-40d3-bf26-191e6d98995b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:16.768928 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f67f2ba2-5630-40d3-bf26-191e6d98995b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Going to try to terminate instance {{(pid=120525) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 28 01:17:16.872012 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f67f2ba2-5630-40d3-bf26-191e6d98995b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "DELETE /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 204 len: 0 microversion: 2.20 time: 0.154601 Aug 28 01:17:16.872605 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 941/1652] 10.222.0.42 () {60 vars in 1280 bytes} [Sun Aug 28 01:17:16 2022] DELETE /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 0 bytes in 156 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Aug 28 01:17:16.878703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-48973a57-f965-4773-8422-8af4c310463c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:16.879299 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-48973a57-f965-4773-8422-8af4c310463c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:16.880917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8e452c47-ca22-4da9-b2a9-51a3a42fe496 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:16.881695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8e452c47-ca22-4da9-b2a9-51a3a42fe496 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:16.886506 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48973a57-f965-4773-8422-8af4c310463c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:16.886991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48973a57-f965-4773-8422-8af4c310463c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:16.887422 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48973a57-f965-4773-8422-8af4c310463c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:16.887699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e452c47-ca22-4da9-b2a9-51a3a42fe496 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:16.888125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48973a57-f965-4773-8422-8af4c310463c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:16.888261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e452c47-ca22-4da9-b2a9-51a3a42fe496 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:16.888600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e452c47-ca22-4da9-b2a9-51a3a42fe496 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:16.888922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48973a57-f965-4773-8422-8af4c310463c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:16.889309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e452c47-ca22-4da9-b2a9-51a3a42fe496 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:16.889414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-48973a57-f965-4773-8422-8af4c310463c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:16.889753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e452c47-ca22-4da9-b2a9-51a3a42fe496 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:16.890173 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e452c47-ca22-4da9-b2a9-51a3a42fe496 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:16.918277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-48973a57-f965-4773-8422-8af4c310463c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Going to try to terminate instance {{(pid=120524) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 28 01:17:16.947711 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:17:17.015940 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-48973a57-f965-4773-8422-8af4c310463c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "DELETE /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 204 len: 0 microversion: 2.1 time: 0.141105 Aug 28 01:17:17.016731 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 712/1653] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 01:17:16 2022] DELETE /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 28 01:17:17.022962 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cbbea283-9833-4719-bc46-b1fa9e1adef3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:17.023692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cbbea283-9833-4719-bc46-b1fa9e1adef3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:17.030838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cbbea283-9833-4719-bc46-b1fa9e1adef3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:17.033091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cbbea283-9833-4719-bc46-b1fa9e1adef3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:17.033536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cbbea283-9833-4719-bc46-b1fa9e1adef3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:17.034262 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cbbea283-9833-4719-bc46-b1fa9e1adef3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:17.034713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cbbea283-9833-4719-bc46-b1fa9e1adef3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:17.035130 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cbbea283-9833-4719-bc46-b1fa9e1adef3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:17.053183 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8e452c47-ca22-4da9-b2a9-51a3a42fe496 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:17.063638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8e452c47-ca22-4da9-b2a9-51a3a42fe496 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:17.064999 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8e452c47-ca22-4da9-b2a9-51a3a42fe496 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:17.066195 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8e452c47-ca22-4da9-b2a9-51a3a42fe496 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.189377 Aug 28 01:17:17.067120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 942/1654] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:16 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 01:17:17.072757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-33610f2f-0489-4c5c-a47f-7a19fcc6e0a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:17.073980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-33610f2f-0489-4c5c-a47f-7a19fcc6e0a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:17.081429 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-33610f2f-0489-4c5c-a47f-7a19fcc6e0a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:17.081900 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-33610f2f-0489-4c5c-a47f-7a19fcc6e0a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:17.082333 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-33610f2f-0489-4c5c-a47f-7a19fcc6e0a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:17.083041 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-33610f2f-0489-4c5c-a47f-7a19fcc6e0a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:17.083550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-33610f2f-0489-4c5c-a47f-7a19fcc6e0a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:17.083971 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-33610f2f-0489-4c5c-a47f-7a19fcc6e0a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:17.314153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-33610f2f-0489-4c5c-a47f-7a19fcc6e0a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:17.325728 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-33610f2f-0489-4c5c-a47f-7a19fcc6e0a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.257298 Aug 28 01:17:17.326478 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-cbbea283-9833-4719-bc46-b1fa9e1adef3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:17.333167 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 943/1655] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:17 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:17.333167 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-291dd26e-cc86-4605-984c-bd460a6a672f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] Calling method '.version_select at 0x7f0e72060e50>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:17.344839 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cbbea283-9833-4719-bc46-b1fa9e1adef3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.326617 Aug 28 01:17:17.345886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 713/1656] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:17 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:17.352862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-291dd26e-cc86-4605-984c-bd460a6a672f tempest-TestVolumeBootPattern-1331161477 tempest-TestVolumeBootPattern-1331161477-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1578003485" status: 202 len: 0 microversion: 2.1 time: 0.024744 Aug 28 01:17:17.353622 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 944/1657] 10.222.0.42 () {58 vars in 1263 bytes} [Sun Aug 28 01:17:17 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1578003485 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 01:17:18.092226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cc656443-b863-4a1d-84d2-5ff287c1180b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:18.092855 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cc656443-b863-4a1d-84d2-5ff287c1180b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:18.099550 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc656443-b863-4a1d-84d2-5ff287c1180b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:18.100035 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc656443-b863-4a1d-84d2-5ff287c1180b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:18.100464 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc656443-b863-4a1d-84d2-5ff287c1180b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:18.101192 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc656443-b863-4a1d-84d2-5ff287c1180b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:18.101639 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc656443-b863-4a1d-84d2-5ff287c1180b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:18.102059 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc656443-b863-4a1d-84d2-5ff287c1180b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:18.247683 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-cc656443-b863-4a1d-84d2-5ff287c1180b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:18.257284 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-cc656443-b863-4a1d-84d2-5ff287c1180b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:18.259213 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-cc656443-b863-4a1d-84d2-5ff287c1180b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:18.260408 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cc656443-b863-4a1d-84d2-5ff287c1180b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.172199 Aug 28 01:17:18.260989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 714/1658] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:18 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:18.350246 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ac7dca15-6e83-471e-91ea-aaaccc570a75 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:18.352084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ac7dca15-6e83-471e-91ea-aaaccc570a75 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:18.358774 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac7dca15-6e83-471e-91ea-aaaccc570a75 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:18.359300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac7dca15-6e83-471e-91ea-aaaccc570a75 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:18.359927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac7dca15-6e83-471e-91ea-aaaccc570a75 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:18.360644 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac7dca15-6e83-471e-91ea-aaaccc570a75 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:18.361134 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac7dca15-6e83-471e-91ea-aaaccc570a75 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:18.361584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac7dca15-6e83-471e-91ea-aaaccc570a75 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:18.370288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d0505165-5594-48e5-9e27-84730bd32b48 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:18.371494 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d0505165-5594-48e5-9e27-84730bd32b48 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:18.378103 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0505165-5594-48e5-9e27-84730bd32b48 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:18.378575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0505165-5594-48e5-9e27-84730bd32b48 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:18.379255 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0505165-5594-48e5-9e27-84730bd32b48 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:18.380075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0505165-5594-48e5-9e27-84730bd32b48 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:18.381188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0505165-5594-48e5-9e27-84730bd32b48 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:18.381614 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0505165-5594-48e5-9e27-84730bd32b48 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:18.824020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ac7dca15-6e83-471e-91ea-aaaccc570a75 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:18.827386 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d0505165-5594-48e5-9e27-84730bd32b48 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:18.835384 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ac7dca15-6e83-471e-91ea-aaaccc570a75 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.490692 Aug 28 01:17:18.836325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 945/1659] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:18 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 493 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:18.836739 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d0505165-5594-48e5-9e27-84730bd32b48 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.472658 Aug 28 01:17:18.837987 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 715/1660] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:18 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:17:19.295131 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4eae41c0-810d-4046-bcc2-4ad86c84d023 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:19.295131 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4eae41c0-810d-4046-bcc2-4ad86c84d023 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:19.295131 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4eae41c0-810d-4046-bcc2-4ad86c84d023 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:19.295131 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4eae41c0-810d-4046-bcc2-4ad86c84d023 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:19.295535 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4eae41c0-810d-4046-bcc2-4ad86c84d023 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:19.295535 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4eae41c0-810d-4046-bcc2-4ad86c84d023 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:19.295535 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4eae41c0-810d-4046-bcc2-4ad86c84d023 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:19.295535 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4eae41c0-810d-4046-bcc2-4ad86c84d023 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:19.432042 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4eae41c0-810d-4046-bcc2-4ad86c84d023 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:19.441509 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4eae41c0-810d-4046-bcc2-4ad86c84d023 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:19.443147 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4eae41c0-810d-4046-bcc2-4ad86c84d023 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:19.444669 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4eae41c0-810d-4046-bcc2-4ad86c84d023 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.164817 Aug 28 01:17:19.445285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 946/1661] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:19 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:19.858086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3a457bba-18a6-4fae-aaa5-fa4db289687e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:19.858701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3a457bba-18a6-4fae-aaa5-fa4db289687e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:19.859789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4e489a2b-3b2b-414a-875d-ce6448930c20 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:19.861020 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4e489a2b-3b2b-414a-875d-ce6448930c20 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:19.866867 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a457bba-18a6-4fae-aaa5-fa4db289687e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:19.867408 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a457bba-18a6-4fae-aaa5-fa4db289687e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:19.867844 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a457bba-18a6-4fae-aaa5-fa4db289687e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:19.868548 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a457bba-18a6-4fae-aaa5-fa4db289687e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:19.868949 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e489a2b-3b2b-414a-875d-ce6448930c20 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:19.869102 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a457bba-18a6-4fae-aaa5-fa4db289687e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:19.869718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a457bba-18a6-4fae-aaa5-fa4db289687e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:19.870026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e489a2b-3b2b-414a-875d-ce6448930c20 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:19.870670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e489a2b-3b2b-414a-875d-ce6448930c20 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:19.871587 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e489a2b-3b2b-414a-875d-ce6448930c20 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:19.872249 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e489a2b-3b2b-414a-875d-ce6448930c20 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:19.872897 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e489a2b-3b2b-414a-875d-ce6448930c20 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:20.082536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4e489a2b-3b2b-414a-875d-ce6448930c20 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:20.096247 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3a457bba-18a6-4fae-aaa5-fa4db289687e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:20.097869 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4e489a2b-3b2b-414a-875d-ce6448930c20 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.243654 Aug 28 01:17:20.100342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 716/1662] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:19 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:17:20.107140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3a457bba-18a6-4fae-aaa5-fa4db289687e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.252807 Aug 28 01:17:20.107855 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 947/1663] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:19 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:20.471784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-69f74eb4-bed2-46f3-8ae7-8035121d59ca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:20.472557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-69f74eb4-bed2-46f3-8ae7-8035121d59ca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:20.479157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-69f74eb4-bed2-46f3-8ae7-8035121d59ca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:20.479699 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-69f74eb4-bed2-46f3-8ae7-8035121d59ca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:20.480349 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-69f74eb4-bed2-46f3-8ae7-8035121d59ca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:20.481201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-69f74eb4-bed2-46f3-8ae7-8035121d59ca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:20.481739 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-69f74eb4-bed2-46f3-8ae7-8035121d59ca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:20.482225 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-69f74eb4-bed2-46f3-8ae7-8035121d59ca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:20.707182 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-69f74eb4-bed2-46f3-8ae7-8035121d59ca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:20.719553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-69f74eb4-bed2-46f3-8ae7-8035121d59ca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:20.721025 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-69f74eb4-bed2-46f3-8ae7-8035121d59ca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:20.722212 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-69f74eb4-bed2-46f3-8ae7-8035121d59ca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.254982 Aug 28 01:17:20.722811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 717/1664] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:20 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:21.119832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7b60801b-5ded-49ef-908d-030381112e99 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:21.120443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7b60801b-5ded-49ef-908d-030381112e99 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:21.127658 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-679942a5-03bd-4b70-a9d9-58a8506383ab tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:21.128269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-679942a5-03bd-4b70-a9d9-58a8506383ab tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:21.139931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b60801b-5ded-49ef-908d-030381112e99 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:21.140452 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b60801b-5ded-49ef-908d-030381112e99 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:21.140877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b60801b-5ded-49ef-908d-030381112e99 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:21.142673 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-679942a5-03bd-4b70-a9d9-58a8506383ab tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:21.143151 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-679942a5-03bd-4b70-a9d9-58a8506383ab tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:21.143569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-679942a5-03bd-4b70-a9d9-58a8506383ab tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:21.144272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-679942a5-03bd-4b70-a9d9-58a8506383ab tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:21.144711 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-679942a5-03bd-4b70-a9d9-58a8506383ab tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:21.145234 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-679942a5-03bd-4b70-a9d9-58a8506383ab tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:21.148652 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b60801b-5ded-49ef-908d-030381112e99 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:21.149158 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b60801b-5ded-49ef-908d-030381112e99 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:21.149743 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b60801b-5ded-49ef-908d-030381112e99 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:21.514310 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-679942a5-03bd-4b70-a9d9-58a8506383ab tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:21.524349 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-679942a5-03bd-4b70-a9d9-58a8506383ab tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.400418 Aug 28 01:17:21.524911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 718/1665] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:21 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:17:21.527936 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7b60801b-5ded-49ef-908d-030381112e99 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:21.537055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7b60801b-5ded-49ef-908d-030381112e99 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.421009 Aug 28 01:17:21.538217 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 948/1666] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:21 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:21.748163 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-87b3e36a-4414-4ede-a4fb-225920c6de77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:21.748635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-87b3e36a-4414-4ede-a4fb-225920c6de77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:21.755889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-87b3e36a-4414-4ede-a4fb-225920c6de77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:21.756371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-87b3e36a-4414-4ede-a4fb-225920c6de77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:21.756928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-87b3e36a-4414-4ede-a4fb-225920c6de77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:21.757622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-87b3e36a-4414-4ede-a4fb-225920c6de77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:21.758066 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-87b3e36a-4414-4ede-a4fb-225920c6de77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:21.758476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-87b3e36a-4414-4ede-a4fb-225920c6de77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:21.900429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-87b3e36a-4414-4ede-a4fb-225920c6de77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:21.911066 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-87b3e36a-4414-4ede-a4fb-225920c6de77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:21.912654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-87b3e36a-4414-4ede-a4fb-225920c6de77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:21.913945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-87b3e36a-4414-4ede-a4fb-225920c6de77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.169847 Aug 28 01:17:21.914541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 719/1667] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:21 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:22.546043 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-428b637f-e277-4edf-8da1-80198afa4779 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:22.550033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-428b637f-e277-4edf-8da1-80198afa4779 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:22.559997 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-428b637f-e277-4edf-8da1-80198afa4779 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:22.560794 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-428b637f-e277-4edf-8da1-80198afa4779 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:22.561495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-428b637f-e277-4edf-8da1-80198afa4779 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:22.562172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e634bbd1-1746-4313-86e2-671ded9b6f64 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:22.562860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e634bbd1-1746-4313-86e2-671ded9b6f64 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:22.563455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-428b637f-e277-4edf-8da1-80198afa4779 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:22.564491 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-428b637f-e277-4edf-8da1-80198afa4779 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:22.565160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-428b637f-e277-4edf-8da1-80198afa4779 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:22.573889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e634bbd1-1746-4313-86e2-671ded9b6f64 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:22.574644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e634bbd1-1746-4313-86e2-671ded9b6f64 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:22.575695 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e634bbd1-1746-4313-86e2-671ded9b6f64 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:22.577156 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e634bbd1-1746-4313-86e2-671ded9b6f64 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:22.577981 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e634bbd1-1746-4313-86e2-671ded9b6f64 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:22.578565 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e634bbd1-1746-4313-86e2-671ded9b6f64 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:22.790832 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e634bbd1-1746-4313-86e2-671ded9b6f64 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:22.794642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-428b637f-e277-4edf-8da1-80198afa4779 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:22.799993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e634bbd1-1746-4313-86e2-671ded9b6f64 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.245226 Aug 28 01:17:22.800827 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 720/1668] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:22 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:22.807986 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-428b637f-e277-4edf-8da1-80198afa4779 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.265934 Aug 28 01:17:22.808562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 949/1669] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:22 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:22.938022 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-aff829fc-41df-454a-acc1-c56e70afcd07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:22.938633 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-aff829fc-41df-454a-acc1-c56e70afcd07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:22.945990 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aff829fc-41df-454a-acc1-c56e70afcd07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:22.946480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aff829fc-41df-454a-acc1-c56e70afcd07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:22.946899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aff829fc-41df-454a-acc1-c56e70afcd07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:22.947601 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aff829fc-41df-454a-acc1-c56e70afcd07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:22.948050 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aff829fc-41df-454a-acc1-c56e70afcd07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:22.948469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aff829fc-41df-454a-acc1-c56e70afcd07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:23.095191 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-aff829fc-41df-454a-acc1-c56e70afcd07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:23.105276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-aff829fc-41df-454a-acc1-c56e70afcd07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:23.106695 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-aff829fc-41df-454a-acc1-c56e70afcd07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:23.107857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-aff829fc-41df-454a-acc1-c56e70afcd07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.173914 Aug 28 01:17:23.108409 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 721/1670] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:22 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:23.823620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7cf2e9bf-5f52-4fb8-a2e7-6bd05140c253 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:23.823620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7cf2e9bf-5f52-4fb8-a2e7-6bd05140c253 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:23.829562 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-274653e7-4dba-47bd-9ec4-ab175b8e773b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:23.830065 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7cf2e9bf-5f52-4fb8-a2e7-6bd05140c253 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:23.830378 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-274653e7-4dba-47bd-9ec4-ab175b8e773b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:23.830722 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7cf2e9bf-5f52-4fb8-a2e7-6bd05140c253 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:23.831192 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7cf2e9bf-5f52-4fb8-a2e7-6bd05140c253 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:23.831898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7cf2e9bf-5f52-4fb8-a2e7-6bd05140c253 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:23.832338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7cf2e9bf-5f52-4fb8-a2e7-6bd05140c253 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:23.832776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7cf2e9bf-5f52-4fb8-a2e7-6bd05140c253 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:23.836826 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-274653e7-4dba-47bd-9ec4-ab175b8e773b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:23.837368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-274653e7-4dba-47bd-9ec4-ab175b8e773b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:23.837763 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-274653e7-4dba-47bd-9ec4-ab175b8e773b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:23.838628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-274653e7-4dba-47bd-9ec4-ab175b8e773b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:23.839205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-274653e7-4dba-47bd-9ec4-ab175b8e773b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:23.839624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-274653e7-4dba-47bd-9ec4-ab175b8e773b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:24.061677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7cf2e9bf-5f52-4fb8-a2e7-6bd05140c253 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:24.063608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-274653e7-4dba-47bd-9ec4-ab175b8e773b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:24.074939 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-274653e7-4dba-47bd-9ec4-ab175b8e773b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.249595 Aug 28 01:17:24.075639 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 722/1671] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:23 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:24.094452 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7cf2e9bf-5f52-4fb8-a2e7-6bd05140c253 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.276440 Aug 28 01:17:24.109473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 950/1672] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:23 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:24.131501 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-96ff58b9-f53a-4f32-976f-252b0ee7e59b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:24.132225 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-96ff58b9-f53a-4f32-976f-252b0ee7e59b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:24.139570 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96ff58b9-f53a-4f32-976f-252b0ee7e59b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:24.140236 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96ff58b9-f53a-4f32-976f-252b0ee7e59b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:24.140785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96ff58b9-f53a-4f32-976f-252b0ee7e59b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:24.141939 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96ff58b9-f53a-4f32-976f-252b0ee7e59b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:24.142271 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96ff58b9-f53a-4f32-976f-252b0ee7e59b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:24.142786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96ff58b9-f53a-4f32-976f-252b0ee7e59b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:24.332642 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-96ff58b9-f53a-4f32-976f-252b0ee7e59b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:24.341895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-96ff58b9-f53a-4f32-976f-252b0ee7e59b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:24.343642 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-96ff58b9-f53a-4f32-976f-252b0ee7e59b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:24.345185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-96ff58b9-f53a-4f32-976f-252b0ee7e59b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.217944 Aug 28 01:17:24.346809 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 723/1673] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:24 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:25.099116 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-df247b74-41b2-42bb-a4f2-8bc9f313314e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:25.099701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-df247b74-41b2-42bb-a4f2-8bc9f313314e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:25.107212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df247b74-41b2-42bb-a4f2-8bc9f313314e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:25.107705 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df247b74-41b2-42bb-a4f2-8bc9f313314e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:25.108139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df247b74-41b2-42bb-a4f2-8bc9f313314e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:25.108917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df247b74-41b2-42bb-a4f2-8bc9f313314e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:25.113928 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df247b74-41b2-42bb-a4f2-8bc9f313314e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:25.113928 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df247b74-41b2-42bb-a4f2-8bc9f313314e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:25.125021 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-41036a5b-b547-449c-82c6-aff9581dac7a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:25.125021 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-41036a5b-b547-449c-82c6-aff9581dac7a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:25.128661 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41036a5b-b547-449c-82c6-aff9581dac7a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:25.129442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41036a5b-b547-449c-82c6-aff9581dac7a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:25.129885 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41036a5b-b547-449c-82c6-aff9581dac7a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:25.130788 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41036a5b-b547-449c-82c6-aff9581dac7a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:25.131303 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41036a5b-b547-449c-82c6-aff9581dac7a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:25.131781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41036a5b-b547-449c-82c6-aff9581dac7a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:25.346842 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-df247b74-41b2-42bb-a4f2-8bc9f313314e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:25.358347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-41036a5b-b547-449c-82c6-aff9581dac7a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:25.359187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-df247b74-41b2-42bb-a4f2-8bc9f313314e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.264072 Aug 28 01:17:25.359798 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 951/1674] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:25 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:25.373421 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-41036a5b-b547-449c-82c6-aff9581dac7a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.259570 Aug 28 01:17:25.374329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 724/1675] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:25 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:25.375666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2a13ed37-4439-4c3f-a664-f3dd55830246 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:25.376364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2a13ed37-4439-4c3f-a664-f3dd55830246 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:25.387691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a13ed37-4439-4c3f-a664-f3dd55830246 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:25.388699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a13ed37-4439-4c3f-a664-f3dd55830246 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:25.389576 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a13ed37-4439-4c3f-a664-f3dd55830246 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:25.390887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a13ed37-4439-4c3f-a664-f3dd55830246 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:25.391736 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a13ed37-4439-4c3f-a664-f3dd55830246 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:25.392518 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a13ed37-4439-4c3f-a664-f3dd55830246 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:25.540809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2a13ed37-4439-4c3f-a664-f3dd55830246 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:25.549843 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2a13ed37-4439-4c3f-a664-f3dd55830246 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:25.551286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2a13ed37-4439-4c3f-a664-f3dd55830246 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:25.552398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2a13ed37-4439-4c3f-a664-f3dd55830246 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.183192 Aug 28 01:17:25.552954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 952/1676] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:25 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 01:17:26.380261 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3a234541-806b-4fc2-9245-cff5822d9186 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:26.380991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3a234541-806b-4fc2-9245-cff5822d9186 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:26.388335 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a234541-806b-4fc2-9245-cff5822d9186 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:26.388819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a234541-806b-4fc2-9245-cff5822d9186 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:26.389285 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a234541-806b-4fc2-9245-cff5822d9186 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:26.398304 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0f610aee-21db-4340-95ca-fcb8d9aa0cd7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:26.398881 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0f610aee-21db-4340-95ca-fcb8d9aa0cd7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:26.401172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a234541-806b-4fc2-9245-cff5822d9186 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:26.401804 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a234541-806b-4fc2-9245-cff5822d9186 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:26.402196 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a234541-806b-4fc2-9245-cff5822d9186 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:26.408079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f610aee-21db-4340-95ca-fcb8d9aa0cd7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:26.408549 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f610aee-21db-4340-95ca-fcb8d9aa0cd7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:26.409222 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f610aee-21db-4340-95ca-fcb8d9aa0cd7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:26.409915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f610aee-21db-4340-95ca-fcb8d9aa0cd7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:26.410353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f610aee-21db-4340-95ca-fcb8d9aa0cd7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:26.410767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f610aee-21db-4340-95ca-fcb8d9aa0cd7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:26.599315 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3a234541-806b-4fc2-9245-cff5822d9186 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:26.602771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0f610aee-21db-4340-95ca-fcb8d9aa0cd7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:26.610860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3a234541-806b-4fc2-9245-cff5822d9186 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.234451 Aug 28 01:17:26.611447 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 725/1677] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:26 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:26.612121 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0f610aee-21db-4340-95ca-fcb8d9aa0cd7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.218318 Aug 28 01:17:26.613128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 953/1678] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:26 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:26.616322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ba8cb1a4-d6bf-465c-911f-53c87e93ec9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:26.617071 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ba8cb1a4-d6bf-465c-911f-53c87e93ec9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:26.623617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba8cb1a4-d6bf-465c-911f-53c87e93ec9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:26.624359 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba8cb1a4-d6bf-465c-911f-53c87e93ec9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:26.624950 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba8cb1a4-d6bf-465c-911f-53c87e93ec9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:26.625685 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba8cb1a4-d6bf-465c-911f-53c87e93ec9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:26.626808 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba8cb1a4-d6bf-465c-911f-53c87e93ec9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:26.626808 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba8cb1a4-d6bf-465c-911f-53c87e93ec9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:26.766390 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ba8cb1a4-d6bf-465c-911f-53c87e93ec9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:26.775031 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ba8cb1a4-d6bf-465c-911f-53c87e93ec9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:26.776438 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ba8cb1a4-d6bf-465c-911f-53c87e93ec9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:26.777623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ba8cb1a4-d6bf-465c-911f-53c87e93ec9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.165258 Aug 28 01:17:26.778171 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 726/1679] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:26 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:27.633006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-92fcfcb4-2fca-4b3a-bf38-47c17e9254fc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:27.633629 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-92fcfcb4-2fca-4b3a-bf38-47c17e9254fc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:27.637195 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-43355c90-2ed9-4c9c-aa54-e9fd1e096836 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:27.637846 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-43355c90-2ed9-4c9c-aa54-e9fd1e096836 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:27.640881 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92fcfcb4-2fca-4b3a-bf38-47c17e9254fc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:27.641509 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92fcfcb4-2fca-4b3a-bf38-47c17e9254fc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:27.642009 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92fcfcb4-2fca-4b3a-bf38-47c17e9254fc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:27.642742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92fcfcb4-2fca-4b3a-bf38-47c17e9254fc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:27.643200 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92fcfcb4-2fca-4b3a-bf38-47c17e9254fc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:27.643628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92fcfcb4-2fca-4b3a-bf38-47c17e9254fc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:27.644722 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43355c90-2ed9-4c9c-aa54-e9fd1e096836 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:27.645312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43355c90-2ed9-4c9c-aa54-e9fd1e096836 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:27.645766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43355c90-2ed9-4c9c-aa54-e9fd1e096836 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:27.646472 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43355c90-2ed9-4c9c-aa54-e9fd1e096836 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:27.646918 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43355c90-2ed9-4c9c-aa54-e9fd1e096836 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:27.647336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43355c90-2ed9-4c9c-aa54-e9fd1e096836 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:27.862808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-92fcfcb4-2fca-4b3a-bf38-47c17e9254fc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:27.871971 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-92fcfcb4-2fca-4b3a-bf38-47c17e9254fc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.242807 Aug 28 01:17:27.873715 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 954/1680] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:27 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:27.877797 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-43355c90-2ed9-4c9c-aa54-e9fd1e096836 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:27.878562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d0b53e73-64da-4499-8e04-04a01b74e034 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:27.879170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d0b53e73-64da-4499-8e04-04a01b74e034 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:27.885349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d0b53e73-64da-4499-8e04-04a01b74e034 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:27.885831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d0b53e73-64da-4499-8e04-04a01b74e034 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:27.886264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d0b53e73-64da-4499-8e04-04a01b74e034 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:27.886971 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d0b53e73-64da-4499-8e04-04a01b74e034 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:27.887452 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d0b53e73-64da-4499-8e04-04a01b74e034 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:27.887837 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d0b53e73-64da-4499-8e04-04a01b74e034 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:27.890992 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-43355c90-2ed9-4c9c-aa54-e9fd1e096836 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.257470 Aug 28 01:17:27.891689 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 727/1681] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:27 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:17:28.024172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d0b53e73-64da-4499-8e04-04a01b74e034 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:28.035926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d0b53e73-64da-4499-8e04-04a01b74e034 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:28.038301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d0b53e73-64da-4499-8e04-04a01b74e034 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:28.040126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d0b53e73-64da-4499-8e04-04a01b74e034 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.165169 Aug 28 01:17:28.041007 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 955/1682] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:27 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:28.895635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-20460929-4fc3-4d31-ad1e-14b59f251e02 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:28.896251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-20460929-4fc3-4d31-ad1e-14b59f251e02 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:28.903102 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20460929-4fc3-4d31-ad1e-14b59f251e02 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:28.903581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20460929-4fc3-4d31-ad1e-14b59f251e02 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:28.904014 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20460929-4fc3-4d31-ad1e-14b59f251e02 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:28.904686 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20460929-4fc3-4d31-ad1e-14b59f251e02 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:28.905194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20460929-4fc3-4d31-ad1e-14b59f251e02 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:28.905597 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20460929-4fc3-4d31-ad1e-14b59f251e02 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:28.915394 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c25b4c36-83f0-4feb-bd2b-e46e33c980b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:28.916809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c25b4c36-83f0-4feb-bd2b-e46e33c980b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:28.921489 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:17:28.926139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c25b4c36-83f0-4feb-bd2b-e46e33c980b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:28.926819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c25b4c36-83f0-4feb-bd2b-e46e33c980b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:28.927352 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c25b4c36-83f0-4feb-bd2b-e46e33c980b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:28.928129 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c25b4c36-83f0-4feb-bd2b-e46e33c980b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:28.929009 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c25b4c36-83f0-4feb-bd2b-e46e33c980b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:28.929379 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c25b4c36-83f0-4feb-bd2b-e46e33c980b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:28.936522 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:17:29.098592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-20460929-4fc3-4d31-ad1e-14b59f251e02 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:29.107273 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-20460929-4fc3-4d31-ad1e-14b59f251e02 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.215814 Aug 28 01:17:29.107890 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 728/1683] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:28 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:29.113384 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-43f12f50-7510-4104-a5a2-9c6536186e4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:29.113926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-43f12f50-7510-4104-a5a2-9c6536186e4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:29.119105 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=64,nova_api:UPDATE=2 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:17:29.123949 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43f12f50-7510-4104-a5a2-9c6536186e4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:29.124404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43f12f50-7510-4104-a5a2-9c6536186e4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:29.124877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43f12f50-7510-4104-a5a2-9c6536186e4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:29.125602 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43f12f50-7510-4104-a5a2-9c6536186e4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:29.125717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c25b4c36-83f0-4feb-bd2b-e46e33c980b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:29.126047 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43f12f50-7510-4104-a5a2-9c6536186e4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:29.126457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43f12f50-7510-4104-a5a2-9c6536186e4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:29.134824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c25b4c36-83f0-4feb-bd2b-e46e33c980b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.226853 Aug 28 01:17:29.135392 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 956/1684] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:28 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:29.262452 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-43f12f50-7510-4104-a5a2-9c6536186e4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:29.271503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-43f12f50-7510-4104-a5a2-9c6536186e4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:29.272918 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-43f12f50-7510-4104-a5a2-9c6536186e4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:29.274147 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-43f12f50-7510-4104-a5a2-9c6536186e4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.165317 Aug 28 01:17:29.274729 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 729/1685] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:29 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:30.128107 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bf58dd47-e945-4459-8ee7-77b047d9366a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:30.128727 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bf58dd47-e945-4459-8ee7-77b047d9366a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:30.132231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=56,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:17:30.136567 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf58dd47-e945-4459-8ee7-77b047d9366a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:30.137085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf58dd47-e945-4459-8ee7-77b047d9366a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:30.137518 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf58dd47-e945-4459-8ee7-77b047d9366a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:30.138215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf58dd47-e945-4459-8ee7-77b047d9366a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:30.138662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf58dd47-e945-4459-8ee7-77b047d9366a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:30.139086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bf58dd47-e945-4459-8ee7-77b047d9366a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:30.156011 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b4ac6188-da27-4e18-ab32-53ec8cc8ee1c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:30.156614 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b4ac6188-da27-4e18-ab32-53ec8cc8ee1c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:30.163457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4ac6188-da27-4e18-ab32-53ec8cc8ee1c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:30.163934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4ac6188-da27-4e18-ab32-53ec8cc8ee1c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:30.164403 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4ac6188-da27-4e18-ab32-53ec8cc8ee1c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:30.165088 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4ac6188-da27-4e18-ab32-53ec8cc8ee1c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:30.165707 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4ac6188-da27-4e18-ab32-53ec8cc8ee1c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:30.165954 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4ac6188-da27-4e18-ab32-53ec8cc8ee1c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:30.337327 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bf58dd47-e945-4459-8ee7-77b047d9366a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:30.345819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b4ac6188-da27-4e18-ab32-53ec8cc8ee1c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:30.346732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bf58dd47-e945-4459-8ee7-77b047d9366a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.222553 Aug 28 01:17:30.347321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 957/1686] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:30 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:30.351830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c7508b2b-3cde-4800-8e85-46f05fdc7761 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:30.352435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c7508b2b-3cde-4800-8e85-46f05fdc7761 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:30.354741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b4ac6188-da27-4e18-ab32-53ec8cc8ee1c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.202737 Aug 28 01:17:30.355591 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 730/1687] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:30 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:30.358370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7508b2b-3cde-4800-8e85-46f05fdc7761 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:30.358884 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7508b2b-3cde-4800-8e85-46f05fdc7761 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:30.359319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7508b2b-3cde-4800-8e85-46f05fdc7761 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:30.360048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7508b2b-3cde-4800-8e85-46f05fdc7761 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:30.360473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7508b2b-3cde-4800-8e85-46f05fdc7761 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:30.360895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7508b2b-3cde-4800-8e85-46f05fdc7761 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:30.492333 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c7508b2b-3cde-4800-8e85-46f05fdc7761 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:30.504988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c7508b2b-3cde-4800-8e85-46f05fdc7761 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:30.507116 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c7508b2b-3cde-4800-8e85-46f05fdc7761 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:30.508888 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c7508b2b-3cde-4800-8e85-46f05fdc7761 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.160411 Aug 28 01:17:30.509753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 958/1688] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:30 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:31.367280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-878dad5a-79f7-4aaf-b69e-d7226103fcc7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:31.367852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-878dad5a-79f7-4aaf-b69e-d7226103fcc7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:31.374680 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-878dad5a-79f7-4aaf-b69e-d7226103fcc7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:31.375173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-878dad5a-79f7-4aaf-b69e-d7226103fcc7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:31.375603 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-878dad5a-79f7-4aaf-b69e-d7226103fcc7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:31.376299 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-878dad5a-79f7-4aaf-b69e-d7226103fcc7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:31.376584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2f31f1de-c0eb-40b6-ab90-a69fa33d2661 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:31.377091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-878dad5a-79f7-4aaf-b69e-d7226103fcc7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:31.377200 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-878dad5a-79f7-4aaf-b69e-d7226103fcc7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:31.377291 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2f31f1de-c0eb-40b6-ab90-a69fa33d2661 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:31.384505 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f31f1de-c0eb-40b6-ab90-a69fa33d2661 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:31.384992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f31f1de-c0eb-40b6-ab90-a69fa33d2661 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:31.385441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f31f1de-c0eb-40b6-ab90-a69fa33d2661 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:31.386129 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f31f1de-c0eb-40b6-ab90-a69fa33d2661 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:31.386580 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f31f1de-c0eb-40b6-ab90-a69fa33d2661 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:31.386986 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f31f1de-c0eb-40b6-ab90-a69fa33d2661 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:31.578888 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-878dad5a-79f7-4aaf-b69e-d7226103fcc7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:31.586368 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2f31f1de-c0eb-40b6-ab90-a69fa33d2661 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:31.587780 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-878dad5a-79f7-4aaf-b69e-d7226103fcc7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.224495 Aug 28 01:17:31.588370 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 731/1689] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:31 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:31.593375 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-841152dc-56ed-42a7-a46c-a450f070dae2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:31.593985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-841152dc-56ed-42a7-a46c-a450f070dae2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:31.595470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2f31f1de-c0eb-40b6-ab90-a69fa33d2661 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.222646 Aug 28 01:17:31.596699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 959/1690] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:31 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:31.600615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-841152dc-56ed-42a7-a46c-a450f070dae2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:31.601104 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-841152dc-56ed-42a7-a46c-a450f070dae2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:31.601771 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-841152dc-56ed-42a7-a46c-a450f070dae2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:31.602566 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-841152dc-56ed-42a7-a46c-a450f070dae2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:31.603017 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-841152dc-56ed-42a7-a46c-a450f070dae2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:31.603416 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-841152dc-56ed-42a7-a46c-a450f070dae2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:31.749512 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-841152dc-56ed-42a7-a46c-a450f070dae2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:31.758984 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-841152dc-56ed-42a7-a46c-a450f070dae2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:31.760404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-841152dc-56ed-42a7-a46c-a450f070dae2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:31.761614 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-841152dc-56ed-42a7-a46c-a450f070dae2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.172040 Aug 28 01:17:31.762167 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 732/1691] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:31 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:32.608954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-14f819b5-5d81-47f1-a915-54cf703e6244 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:32.609586 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-14f819b5-5d81-47f1-a915-54cf703e6244 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:32.617037 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14f819b5-5d81-47f1-a915-54cf703e6244 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:32.617534 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14f819b5-5d81-47f1-a915-54cf703e6244 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:32.617679 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-688ea28c-e0f0-43cb-9cc5-a7cebbccefff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:32.618108 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14f819b5-5d81-47f1-a915-54cf703e6244 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:32.618279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-688ea28c-e0f0-43cb-9cc5-a7cebbccefff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:32.618680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14f819b5-5d81-47f1-a915-54cf703e6244 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:32.619128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14f819b5-5d81-47f1-a915-54cf703e6244 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:32.619548 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14f819b5-5d81-47f1-a915-54cf703e6244 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:32.625742 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-688ea28c-e0f0-43cb-9cc5-a7cebbccefff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:32.626231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-688ea28c-e0f0-43cb-9cc5-a7cebbccefff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:32.626658 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-688ea28c-e0f0-43cb-9cc5-a7cebbccefff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:32.627368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-688ea28c-e0f0-43cb-9cc5-a7cebbccefff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:32.627811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-688ea28c-e0f0-43cb-9cc5-a7cebbccefff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:32.628228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-688ea28c-e0f0-43cb-9cc5-a7cebbccefff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:32.820417 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-14f819b5-5d81-47f1-a915-54cf703e6244 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:32.824362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-688ea28c-e0f0-43cb-9cc5-a7cebbccefff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:32.829919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-14f819b5-5d81-47f1-a915-54cf703e6244 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.224859 Aug 28 01:17:32.830678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 960/1692] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:32 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:32.835276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-748b202f-aebe-41f7-976c-8f28a559b942 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:32.835859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-748b202f-aebe-41f7-976c-8f28a559b942 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:32.839631 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-688ea28c-e0f0-43cb-9cc5-a7cebbccefff tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.225190 Aug 28 01:17:32.849455 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 733/1693] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:32 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:17:32.849661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-748b202f-aebe-41f7-976c-8f28a559b942 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:32.849661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-748b202f-aebe-41f7-976c-8f28a559b942 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:32.849661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-748b202f-aebe-41f7-976c-8f28a559b942 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:32.849661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-748b202f-aebe-41f7-976c-8f28a559b942 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:32.850053 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-748b202f-aebe-41f7-976c-8f28a559b942 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:32.850053 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-748b202f-aebe-41f7-976c-8f28a559b942 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:32.981951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-748b202f-aebe-41f7-976c-8f28a559b942 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:32.991434 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-748b202f-aebe-41f7-976c-8f28a559b942 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:32.992859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-748b202f-aebe-41f7-976c-8f28a559b942 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:32.994039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-748b202f-aebe-41f7-976c-8f28a559b942 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.162402 Aug 28 01:17:32.994584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 961/1694] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:32 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:33.855363 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-328b767c-8f25-4663-8aef-5198fb7b8d81 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:33.856008 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-328b767c-8f25-4663-8aef-5198fb7b8d81 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:33.863430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c4aeeca2-834a-4858-a88f-00963234dd21 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:33.863970 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-328b767c-8f25-4663-8aef-5198fb7b8d81 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:33.863970 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-328b767c-8f25-4663-8aef-5198fb7b8d81 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:33.864336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c4aeeca2-834a-4858-a88f-00963234dd21 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:33.864469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-328b767c-8f25-4663-8aef-5198fb7b8d81 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:33.865094 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-328b767c-8f25-4663-8aef-5198fb7b8d81 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:33.865539 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-328b767c-8f25-4663-8aef-5198fb7b8d81 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:33.865978 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-328b767c-8f25-4663-8aef-5198fb7b8d81 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:33.871071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c4aeeca2-834a-4858-a88f-00963234dd21 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:33.871563 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c4aeeca2-834a-4858-a88f-00963234dd21 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:33.871994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c4aeeca2-834a-4858-a88f-00963234dd21 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:33.872698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c4aeeca2-834a-4858-a88f-00963234dd21 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:33.873177 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c4aeeca2-834a-4858-a88f-00963234dd21 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:33.873599 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c4aeeca2-834a-4858-a88f-00963234dd21 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:34.055919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c4aeeca2-834a-4858-a88f-00963234dd21 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:34.064773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c4aeeca2-834a-4858-a88f-00963234dd21 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.205100 Aug 28 01:17:34.065407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 962/1695] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:33 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:34.067555 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-328b767c-8f25-4663-8aef-5198fb7b8d81 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:34.070647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f0a6c698-5c79-41a0-b7ef-898c03bf095e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:34.071279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f0a6c698-5c79-41a0-b7ef-898c03bf095e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:34.077103 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-328b767c-8f25-4663-8aef-5198fb7b8d81 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.227972 Aug 28 01:17:34.077518 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0a6c698-5c79-41a0-b7ef-898c03bf095e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:34.078032 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0a6c698-5c79-41a0-b7ef-898c03bf095e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:34.078505 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0a6c698-5c79-41a0-b7ef-898c03bf095e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:34.079222 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0a6c698-5c79-41a0-b7ef-898c03bf095e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:34.079681 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0a6c698-5c79-41a0-b7ef-898c03bf095e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:34.080105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0a6c698-5c79-41a0-b7ef-898c03bf095e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:34.088696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 734/1696] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:33 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:34.216776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f0a6c698-5c79-41a0-b7ef-898c03bf095e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:34.227146 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f0a6c698-5c79-41a0-b7ef-898c03bf095e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:34.228795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f0a6c698-5c79-41a0-b7ef-898c03bf095e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:34.230035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f0a6c698-5c79-41a0-b7ef-898c03bf095e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.163691 Aug 28 01:17:34.230797 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 963/1697] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:34 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:35.086553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e8a15c7f-51f0-4528-a249-844baa43347b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:35.087137 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e8a15c7f-51f0-4528-a249-844baa43347b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:35.093865 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8a15c7f-51f0-4528-a249-844baa43347b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:35.094490 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8a15c7f-51f0-4528-a249-844baa43347b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:35.094778 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8a15c7f-51f0-4528-a249-844baa43347b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:35.095475 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8a15c7f-51f0-4528-a249-844baa43347b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:35.095922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8a15c7f-51f0-4528-a249-844baa43347b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:35.096336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e8a15c7f-51f0-4528-a249-844baa43347b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:35.098573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-67f02298-3650-483d-afbf-5c7c5b474646 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:35.099202 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-67f02298-3650-483d-afbf-5c7c5b474646 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:35.106583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67f02298-3650-483d-afbf-5c7c5b474646 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:35.107133 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67f02298-3650-483d-afbf-5c7c5b474646 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:35.107670 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67f02298-3650-483d-afbf-5c7c5b474646 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:35.108443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67f02298-3650-483d-afbf-5c7c5b474646 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:35.109061 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67f02298-3650-483d-afbf-5c7c5b474646 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:35.109558 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67f02298-3650-483d-afbf-5c7c5b474646 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:35.302696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e8a15c7f-51f0-4528-a249-844baa43347b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:35.314702 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e8a15c7f-51f0-4528-a249-844baa43347b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.232100 Aug 28 01:17:35.315360 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 735/1698] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:35 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:35.315518 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-67f02298-3650-483d-afbf-5c7c5b474646 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:35.320926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ce9fe621-4a63-4fa0-9ba4-07dfdf123cac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:35.321699 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ce9fe621-4a63-4fa0-9ba4-07dfdf123cac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:35.326892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-67f02298-3650-483d-afbf-5c7c5b474646 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.232077 Aug 28 01:17:35.328284 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 964/1699] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:35 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:35.336226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce9fe621-4a63-4fa0-9ba4-07dfdf123cac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:35.336805 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce9fe621-4a63-4fa0-9ba4-07dfdf123cac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:35.337224 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce9fe621-4a63-4fa0-9ba4-07dfdf123cac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:35.338172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce9fe621-4a63-4fa0-9ba4-07dfdf123cac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:35.338645 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce9fe621-4a63-4fa0-9ba4-07dfdf123cac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:35.339198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce9fe621-4a63-4fa0-9ba4-07dfdf123cac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:35.476469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ce9fe621-4a63-4fa0-9ba4-07dfdf123cac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:35.485426 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ce9fe621-4a63-4fa0-9ba4-07dfdf123cac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:35.486844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ce9fe621-4a63-4fa0-9ba4-07dfdf123cac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:35.488001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ce9fe621-4a63-4fa0-9ba4-07dfdf123cac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.171338 Aug 28 01:17:35.488557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 736/1700] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:35 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:36.337438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3a638e3c-84c6-47d8-82af-a83288118351 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:36.344294 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3a638e3c-84c6-47d8-82af-a83288118351 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:36.345340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a638e3c-84c6-47d8-82af-a83288118351 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:36.345836 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a638e3c-84c6-47d8-82af-a83288118351 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:36.346429 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a638e3c-84c6-47d8-82af-a83288118351 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:36.346959 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a638e3c-84c6-47d8-82af-a83288118351 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:36.347399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a638e3c-84c6-47d8-82af-a83288118351 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:36.347818 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a638e3c-84c6-47d8-82af-a83288118351 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:36.353338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a2a13467-5711-4d60-88bf-b3dba3681034 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:36.354447 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a2a13467-5711-4d60-88bf-b3dba3681034 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:36.366205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a2a13467-5711-4d60-88bf-b3dba3681034 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:36.366985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a2a13467-5711-4d60-88bf-b3dba3681034 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:36.367756 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a2a13467-5711-4d60-88bf-b3dba3681034 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:36.369065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a2a13467-5711-4d60-88bf-b3dba3681034 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:36.369866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a2a13467-5711-4d60-88bf-b3dba3681034 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:36.370626 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a2a13467-5711-4d60-88bf-b3dba3681034 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:36.578568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3a638e3c-84c6-47d8-82af-a83288118351 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:36.586935 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a2a13467-5711-4d60-88bf-b3dba3681034 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:36.587136 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3a638e3c-84c6-47d8-82af-a83288118351 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.254402 Aug 28 01:17:36.587698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 965/1701] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:36 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:36.592349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-04a7e5ef-0e94-4e28-a51b-9b155c37336e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:36.592984 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-04a7e5ef-0e94-4e28-a51b-9b155c37336e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:36.598301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a2a13467-5711-4d60-88bf-b3dba3681034 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.251893 Aug 28 01:17:36.599057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-04a7e5ef-0e94-4e28-a51b-9b155c37336e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:36.601661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-04a7e5ef-0e94-4e28-a51b-9b155c37336e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:36.601789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 737/1702] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:36 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:36.601898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-04a7e5ef-0e94-4e28-a51b-9b155c37336e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:36.601898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-04a7e5ef-0e94-4e28-a51b-9b155c37336e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:36.601898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-04a7e5ef-0e94-4e28-a51b-9b155c37336e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:36.601898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-04a7e5ef-0e94-4e28-a51b-9b155c37336e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:36.741335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-04a7e5ef-0e94-4e28-a51b-9b155c37336e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:36.750620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-04a7e5ef-0e94-4e28-a51b-9b155c37336e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:36.752061 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-04a7e5ef-0e94-4e28-a51b-9b155c37336e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:36.753261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-04a7e5ef-0e94-4e28-a51b-9b155c37336e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.164602 Aug 28 01:17:36.753828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 966/1703] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:36 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:37.610628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b406a12c-a98e-4476-bdf6-812655cd36ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:37.611278 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b406a12c-a98e-4476-bdf6-812655cd36ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:37.618085 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b406a12c-a98e-4476-bdf6-812655cd36ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:37.618572 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b406a12c-a98e-4476-bdf6-812655cd36ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:37.619001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b406a12c-a98e-4476-bdf6-812655cd36ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:37.619702 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b406a12c-a98e-4476-bdf6-812655cd36ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:37.620150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b406a12c-a98e-4476-bdf6-812655cd36ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:37.620568 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b406a12c-a98e-4476-bdf6-812655cd36ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:37.620711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-91439712-b889-4345-87f3-57a6744de68f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:37.621353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-91439712-b889-4345-87f3-57a6744de68f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:37.628378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91439712-b889-4345-87f3-57a6744de68f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:37.628853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91439712-b889-4345-87f3-57a6744de68f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:37.629326 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91439712-b889-4345-87f3-57a6744de68f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:37.630024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91439712-b889-4345-87f3-57a6744de68f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:37.630499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91439712-b889-4345-87f3-57a6744de68f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:37.630938 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91439712-b889-4345-87f3-57a6744de68f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:37.823043 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-91439712-b889-4345-87f3-57a6744de68f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:37.826296 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b406a12c-a98e-4476-bdf6-812655cd36ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:37.833142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-91439712-b889-4345-87f3-57a6744de68f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.215794 Aug 28 01:17:37.833873 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 967/1704] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:37 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:37.834805 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b406a12c-a98e-4476-bdf6-812655cd36ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.229002 Aug 28 01:17:37.835380 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 738/1705] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:37 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:37.840352 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5a3e9b11-5cf8-485b-b825-8da46227fd77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:37.841254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5a3e9b11-5cf8-485b-b825-8da46227fd77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:37.848355 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5a3e9b11-5cf8-485b-b825-8da46227fd77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:37.848993 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5a3e9b11-5cf8-485b-b825-8da46227fd77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:37.849422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5a3e9b11-5cf8-485b-b825-8da46227fd77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:37.850126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5a3e9b11-5cf8-485b-b825-8da46227fd77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:37.850566 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5a3e9b11-5cf8-485b-b825-8da46227fd77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:37.850982 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5a3e9b11-5cf8-485b-b825-8da46227fd77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:37.991834 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5a3e9b11-5cf8-485b-b825-8da46227fd77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:38.001336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5a3e9b11-5cf8-485b-b825-8da46227fd77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:38.002788 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5a3e9b11-5cf8-485b-b825-8da46227fd77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:38.003982 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5a3e9b11-5cf8-485b-b825-8da46227fd77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.168651 Aug 28 01:17:38.004593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 968/1706] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:37 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:38.855557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f7134c43-aca8-414e-b412-60e26a68b970 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:38.856132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-dd2b7703-55a4-4349-a76a-1248050271c5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:38.856502 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f7134c43-aca8-414e-b412-60e26a68b970 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:38.856631 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-dd2b7703-55a4-4349-a76a-1248050271c5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:38.863437 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f7134c43-aca8-414e-b412-60e26a68b970 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:38.864309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f7134c43-aca8-414e-b412-60e26a68b970 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:38.865182 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dd2b7703-55a4-4349-a76a-1248050271c5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:38.865345 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f7134c43-aca8-414e-b412-60e26a68b970 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:38.866140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dd2b7703-55a4-4349-a76a-1248050271c5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:38.866140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dd2b7703-55a4-4349-a76a-1248050271c5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:38.866351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f7134c43-aca8-414e-b412-60e26a68b970 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:38.866703 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dd2b7703-55a4-4349-a76a-1248050271c5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:38.867149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f7134c43-aca8-414e-b412-60e26a68b970 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:38.867345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dd2b7703-55a4-4349-a76a-1248050271c5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:38.867569 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dd2b7703-55a4-4349-a76a-1248050271c5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:38.871093 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f7134c43-aca8-414e-b412-60e26a68b970 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:39.071769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f7134c43-aca8-414e-b412-60e26a68b970 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:39.081471 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f7134c43-aca8-414e-b412-60e26a68b970 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.229780 Aug 28 01:17:39.082159 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 739/1707] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:38 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:39.084828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dd2b7703-55a4-4349-a76a-1248050271c5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:39.087052 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b907cf32-41ca-488d-b7cf-7db3f99f6364 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:39.087644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b907cf32-41ca-488d-b7cf-7db3f99f6364 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:39.096186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-dd2b7703-55a4-4349-a76a-1248050271c5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.244122 Aug 28 01:17:39.096873 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 969/1708] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:38 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:39.097518 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b907cf32-41ca-488d-b7cf-7db3f99f6364 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:39.098449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b907cf32-41ca-488d-b7cf-7db3f99f6364 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:39.099234 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b907cf32-41ca-488d-b7cf-7db3f99f6364 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:39.100484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b907cf32-41ca-488d-b7cf-7db3f99f6364 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:39.101337 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b907cf32-41ca-488d-b7cf-7db3f99f6364 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:39.102120 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b907cf32-41ca-488d-b7cf-7db3f99f6364 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:39.249503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b907cf32-41ca-488d-b7cf-7db3f99f6364 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:39.259032 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b907cf32-41ca-488d-b7cf-7db3f99f6364 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:39.260537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b907cf32-41ca-488d-b7cf-7db3f99f6364 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:39.261810 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b907cf32-41ca-488d-b7cf-7db3f99f6364 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.178692 Aug 28 01:17:39.262389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 740/1709] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:39 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:40.103842 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-716858b3-5cf0-4c74-a925-40b6a6504162 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:40.104498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-716858b3-5cf0-4c74-a925-40b6a6504162 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:40.112555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-716858b3-5cf0-4c74-a925-40b6a6504162 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:40.113360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-716858b3-5cf0-4c74-a925-40b6a6504162 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:40.114007 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-716858b3-5cf0-4c74-a925-40b6a6504162 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:40.115060 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-716858b3-5cf0-4c74-a925-40b6a6504162 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:40.115749 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-716858b3-5cf0-4c74-a925-40b6a6504162 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:40.116363 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-716858b3-5cf0-4c74-a925-40b6a6504162 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:40.126508 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5a6123f0-a8e8-431e-aa62-946e2162dee5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:40.127922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5a6123f0-a8e8-431e-aa62-946e2162dee5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:40.138713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a6123f0-a8e8-431e-aa62-946e2162dee5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:40.139453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a6123f0-a8e8-431e-aa62-946e2162dee5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:40.140085 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a6123f0-a8e8-431e-aa62-946e2162dee5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:40.140824 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a6123f0-a8e8-431e-aa62-946e2162dee5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:40.141320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a6123f0-a8e8-431e-aa62-946e2162dee5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:40.141938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a6123f0-a8e8-431e-aa62-946e2162dee5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:40.334927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-716858b3-5cf0-4c74-a925-40b6a6504162 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:40.337103 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5a6123f0-a8e8-431e-aa62-946e2162dee5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:40.346541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5a6123f0-a8e8-431e-aa62-946e2162dee5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.224025 Aug 28 01:17:40.347138 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 741/1710] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:40 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:40.348824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-716858b3-5cf0-4c74-a925-40b6a6504162 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.248701 Aug 28 01:17:40.349778 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 970/1711] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:40 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:40.351942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-86ca90ba-ed65-49d3-ad48-2c0130375d97 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:40.352560 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-86ca90ba-ed65-49d3-ad48-2c0130375d97 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:40.359308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86ca90ba-ed65-49d3-ad48-2c0130375d97 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:40.359851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86ca90ba-ed65-49d3-ad48-2c0130375d97 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:40.360200 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86ca90ba-ed65-49d3-ad48-2c0130375d97 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:40.360914 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86ca90ba-ed65-49d3-ad48-2c0130375d97 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:40.361390 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86ca90ba-ed65-49d3-ad48-2c0130375d97 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:40.361810 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86ca90ba-ed65-49d3-ad48-2c0130375d97 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:40.764734 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-86ca90ba-ed65-49d3-ad48-2c0130375d97 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:40.773815 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-86ca90ba-ed65-49d3-ad48-2c0130375d97 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:40.775241 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-86ca90ba-ed65-49d3-ad48-2c0130375d97 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:40.776427 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-86ca90ba-ed65-49d3-ad48-2c0130375d97 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.428350 Aug 28 01:17:40.776992 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 742/1712] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:40 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:41.373468 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ce593080-13a9-4a34-a4e2-009ec524cea1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:41.374387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ce593080-13a9-4a34-a4e2-009ec524cea1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:41.379798 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7ce81065-2cb0-441a-9944-35954cd25668 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:41.380561 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7ce81065-2cb0-441a-9944-35954cd25668 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:41.381399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce593080-13a9-4a34-a4e2-009ec524cea1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:41.381903 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce593080-13a9-4a34-a4e2-009ec524cea1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:41.382344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce593080-13a9-4a34-a4e2-009ec524cea1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:41.383080 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce593080-13a9-4a34-a4e2-009ec524cea1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:41.384240 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce593080-13a9-4a34-a4e2-009ec524cea1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:41.384240 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce593080-13a9-4a34-a4e2-009ec524cea1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:41.387890 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ce81065-2cb0-441a-9944-35954cd25668 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:41.388382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ce81065-2cb0-441a-9944-35954cd25668 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:41.388909 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ce81065-2cb0-441a-9944-35954cd25668 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:41.389599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ce81065-2cb0-441a-9944-35954cd25668 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:41.390040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ce81065-2cb0-441a-9944-35954cd25668 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:41.390463 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ce81065-2cb0-441a-9944-35954cd25668 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:41.612888 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ce593080-13a9-4a34-a4e2-009ec524cea1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:41.616610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7ce81065-2cb0-441a-9944-35954cd25668 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:41.621878 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ce593080-13a9-4a34-a4e2-009ec524cea1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.252277 Aug 28 01:17:41.622442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 971/1713] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:41 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:41.626098 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7ce81065-2cb0-441a-9944-35954cd25668 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.250079 Aug 28 01:17:41.626973 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 743/1714] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:41 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:41.799504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-db387b09-0a3a-41f6-abe8-87229cb6ef22 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:41.800095 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-db387b09-0a3a-41f6-abe8-87229cb6ef22 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:41.806298 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db387b09-0a3a-41f6-abe8-87229cb6ef22 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:41.806809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db387b09-0a3a-41f6-abe8-87229cb6ef22 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:41.807233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db387b09-0a3a-41f6-abe8-87229cb6ef22 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:41.807934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db387b09-0a3a-41f6-abe8-87229cb6ef22 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:41.808372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db387b09-0a3a-41f6-abe8-87229cb6ef22 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:41.808787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db387b09-0a3a-41f6-abe8-87229cb6ef22 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:41.959371 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-db387b09-0a3a-41f6-abe8-87229cb6ef22 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:41.961927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:17:41.970677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-db387b09-0a3a-41f6-abe8-87229cb6ef22 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:41.972124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-db387b09-0a3a-41f6-abe8-87229cb6ef22 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:41.973372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-db387b09-0a3a-41f6-abe8-87229cb6ef22 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.177910 Aug 28 01:17:41.973952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 972/1715] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:41 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:42.643567 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c37cfcc2-b8c7-4116-a9eb-e7bedec9f8df tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:42.644213 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c37cfcc2-b8c7-4116-a9eb-e7bedec9f8df tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:42.652034 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6b20d6d0-35cc-4b96-a1f9-8e849978ae58 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:42.652684 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6b20d6d0-35cc-4b96-a1f9-8e849978ae58 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:42.656514 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c37cfcc2-b8c7-4116-a9eb-e7bedec9f8df tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:42.657048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c37cfcc2-b8c7-4116-a9eb-e7bedec9f8df tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:42.657797 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c37cfcc2-b8c7-4116-a9eb-e7bedec9f8df tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:42.658994 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c37cfcc2-b8c7-4116-a9eb-e7bedec9f8df tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:42.659669 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b20d6d0-35cc-4b96-a1f9-8e849978ae58 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:42.659862 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c37cfcc2-b8c7-4116-a9eb-e7bedec9f8df tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:42.660159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b20d6d0-35cc-4b96-a1f9-8e849978ae58 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:42.660536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c37cfcc2-b8c7-4116-a9eb-e7bedec9f8df tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:42.660720 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b20d6d0-35cc-4b96-a1f9-8e849978ae58 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:42.662039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b20d6d0-35cc-4b96-a1f9-8e849978ae58 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:42.662479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b20d6d0-35cc-4b96-a1f9-8e849978ae58 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:42.662905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b20d6d0-35cc-4b96-a1f9-8e849978ae58 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:42.887978 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c37cfcc2-b8c7-4116-a9eb-e7bedec9f8df tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:42.896901 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6b20d6d0-35cc-4b96-a1f9-8e849978ae58 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:42.897763 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c37cfcc2-b8c7-4116-a9eb-e7bedec9f8df tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.258119 Aug 28 01:17:42.898490 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 744/1716] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:42 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:42.907501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6b20d6d0-35cc-4b96-a1f9-8e849978ae58 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.259357 Aug 28 01:17:42.908105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 973/1717] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:42 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:42.997152 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5d0c509d-655d-44ba-93dd-f3fd06543d62 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:42.997816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5d0c509d-655d-44ba-93dd-f3fd06543d62 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:43.004736 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5d0c509d-655d-44ba-93dd-f3fd06543d62 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:43.005279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5d0c509d-655d-44ba-93dd-f3fd06543d62 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:43.005720 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5d0c509d-655d-44ba-93dd-f3fd06543d62 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:43.006447 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5d0c509d-655d-44ba-93dd-f3fd06543d62 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:43.006895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5d0c509d-655d-44ba-93dd-f3fd06543d62 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:43.007324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5d0c509d-655d-44ba-93dd-f3fd06543d62 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:43.010434 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:17:43.159382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5d0c509d-655d-44ba-93dd-f3fd06543d62 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:43.171281 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5d0c509d-655d-44ba-93dd-f3fd06543d62 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:43.172871 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5d0c509d-655d-44ba-93dd-f3fd06543d62 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:43.174655 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5d0c509d-655d-44ba-93dd-f3fd06543d62 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.181187 Aug 28 01:17:43.174895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 745/1718] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:42 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:43.922592 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-61d644f0-0bc2-4fa1-b7c4-d25c98daab8e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:43.923204 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-61d644f0-0bc2-4fa1-b7c4-d25c98daab8e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:43.929959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b28ec39d-2178-4877-9836-82a0dcaadc9e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:43.930504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61d644f0-0bc2-4fa1-b7c4-d25c98daab8e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:43.930618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b28ec39d-2178-4877-9836-82a0dcaadc9e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:43.930777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61d644f0-0bc2-4fa1-b7c4-d25c98daab8e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:43.931202 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61d644f0-0bc2-4fa1-b7c4-d25c98daab8e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:43.931882 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61d644f0-0bc2-4fa1-b7c4-d25c98daab8e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:43.932325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61d644f0-0bc2-4fa1-b7c4-d25c98daab8e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:43.932739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-61d644f0-0bc2-4fa1-b7c4-d25c98daab8e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:43.937401 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b28ec39d-2178-4877-9836-82a0dcaadc9e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:43.937930 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b28ec39d-2178-4877-9836-82a0dcaadc9e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:43.938353 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b28ec39d-2178-4877-9836-82a0dcaadc9e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:43.939066 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b28ec39d-2178-4877-9836-82a0dcaadc9e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:43.939511 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b28ec39d-2178-4877-9836-82a0dcaadc9e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:43.939932 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b28ec39d-2178-4877-9836-82a0dcaadc9e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:44.139911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-61d644f0-0bc2-4fa1-b7c4-d25c98daab8e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:44.143452 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b28ec39d-2178-4877-9836-82a0dcaadc9e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:44.149448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-61d644f0-0bc2-4fa1-b7c4-d25c98daab8e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.232444 Aug 28 01:17:44.151250 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 974/1719] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:43 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:44.153579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b28ec39d-2178-4877-9836-82a0dcaadc9e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.227754 Aug 28 01:17:44.154272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 746/1720] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:43 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:44.198350 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a58807c9-f1b4-4e2b-9447-1df28c12dde2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:44.198967 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a58807c9-f1b4-4e2b-9447-1df28c12dde2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:44.205781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a58807c9-f1b4-4e2b-9447-1df28c12dde2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:44.206299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a58807c9-f1b4-4e2b-9447-1df28c12dde2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:44.206736 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a58807c9-f1b4-4e2b-9447-1df28c12dde2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:44.207466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a58807c9-f1b4-4e2b-9447-1df28c12dde2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:44.207925 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a58807c9-f1b4-4e2b-9447-1df28c12dde2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:44.208501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a58807c9-f1b4-4e2b-9447-1df28c12dde2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:44.357529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a58807c9-f1b4-4e2b-9447-1df28c12dde2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:44.367353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a58807c9-f1b4-4e2b-9447-1df28c12dde2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:44.369320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a58807c9-f1b4-4e2b-9447-1df28c12dde2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:44.370675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a58807c9-f1b4-4e2b-9447-1df28c12dde2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.176458 Aug 28 01:17:44.371317 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 975/1721] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:44 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:45.172185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9c1d21f1-c072-48bb-8db0-4767bc250910 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:45.172809 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9c1d21f1-c072-48bb-8db0-4767bc250910 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:45.175485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-46af4892-06a0-4d2f-bf03-d100de9d25ce tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:45.176450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-46af4892-06a0-4d2f-bf03-d100de9d25ce tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:45.180151 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9c1d21f1-c072-48bb-8db0-4767bc250910 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:45.180651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9c1d21f1-c072-48bb-8db0-4767bc250910 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:45.181115 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9c1d21f1-c072-48bb-8db0-4767bc250910 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:45.181845 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9c1d21f1-c072-48bb-8db0-4767bc250910 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:45.182296 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9c1d21f1-c072-48bb-8db0-4767bc250910 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:45.182721 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9c1d21f1-c072-48bb-8db0-4767bc250910 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:45.185461 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46af4892-06a0-4d2f-bf03-d100de9d25ce tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:45.186196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46af4892-06a0-4d2f-bf03-d100de9d25ce tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:45.186857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46af4892-06a0-4d2f-bf03-d100de9d25ce tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:45.187807 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46af4892-06a0-4d2f-bf03-d100de9d25ce tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:45.188290 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46af4892-06a0-4d2f-bf03-d100de9d25ce tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:45.188738 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46af4892-06a0-4d2f-bf03-d100de9d25ce tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:45.394322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9c1d21f1-c072-48bb-8db0-4767bc250910 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:45.407785 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-46af4892-06a0-4d2f-bf03-d100de9d25ce tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:45.408033 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9c1d21f1-c072-48bb-8db0-4767bc250910 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.239755 Aug 28 01:17:45.408698 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 747/1722] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:45 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:45.416584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0c56956a-a601-4c32-bd07-6e3b8bec7a27 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:45.417402 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0c56956a-a601-4c32-bd07-6e3b8bec7a27 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:45.417595 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-46af4892-06a0-4d2f-bf03-d100de9d25ce tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.246461 Aug 28 01:17:45.418362 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 976/1723] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:45 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:17:45.428523 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0c56956a-a601-4c32-bd07-6e3b8bec7a27 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:45.429416 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0c56956a-a601-4c32-bd07-6e3b8bec7a27 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:45.430168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0c56956a-a601-4c32-bd07-6e3b8bec7a27 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:45.431443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0c56956a-a601-4c32-bd07-6e3b8bec7a27 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:45.432242 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0c56956a-a601-4c32-bd07-6e3b8bec7a27 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:45.433048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0c56956a-a601-4c32-bd07-6e3b8bec7a27 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:45.598345 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0c56956a-a601-4c32-bd07-6e3b8bec7a27 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:45.607925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0c56956a-a601-4c32-bd07-6e3b8bec7a27 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:45.609446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0c56956a-a601-4c32-bd07-6e3b8bec7a27 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:45.610709 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0c56956a-a601-4c32-bd07-6e3b8bec7a27 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.201013 Aug 28 01:17:45.611275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 748/1724] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:45 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:46.434360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9804f69e-1ecd-4148-9988-063dd8d3ac60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:46.434967 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9804f69e-1ecd-4148-9988-063dd8d3ac60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:46.439306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-34f950f6-2258-48be-bb3c-77e4566a6d91 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:46.439925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-34f950f6-2258-48be-bb3c-77e4566a6d91 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:46.442226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9804f69e-1ecd-4148-9988-063dd8d3ac60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:46.442817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9804f69e-1ecd-4148-9988-063dd8d3ac60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:46.443598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9804f69e-1ecd-4148-9988-063dd8d3ac60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:46.444307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9804f69e-1ecd-4148-9988-063dd8d3ac60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:46.444779 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9804f69e-1ecd-4148-9988-063dd8d3ac60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:46.445222 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9804f69e-1ecd-4148-9988-063dd8d3ac60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:46.446955 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-34f950f6-2258-48be-bb3c-77e4566a6d91 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:46.447442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-34f950f6-2258-48be-bb3c-77e4566a6d91 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:46.447867 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-34f950f6-2258-48be-bb3c-77e4566a6d91 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:46.448559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-34f950f6-2258-48be-bb3c-77e4566a6d91 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:46.449035 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-34f950f6-2258-48be-bb3c-77e4566a6d91 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:46.449439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-34f950f6-2258-48be-bb3c-77e4566a6d91 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:46.650828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9804f69e-1ecd-4148-9988-063dd8d3ac60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:46.653969 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-34f950f6-2258-48be-bb3c-77e4566a6d91 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:46.660628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9804f69e-1ecd-4148-9988-063dd8d3ac60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.230150 Aug 28 01:17:46.661297 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 977/1725] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:46 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:46.663097 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-34f950f6-2258-48be-bb3c-77e4566a6d91 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.227720 Aug 28 01:17:46.663666 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 749/1726] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:46 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:17:46.666017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f5403204-2726-4574-94d5-0f9aab109c17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:46.666614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f5403204-2726-4574-94d5-0f9aab109c17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:46.672889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5403204-2726-4574-94d5-0f9aab109c17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:46.673603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5403204-2726-4574-94d5-0f9aab109c17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:46.674120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5403204-2726-4574-94d5-0f9aab109c17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:46.674759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5403204-2726-4574-94d5-0f9aab109c17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:46.675201 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5403204-2726-4574-94d5-0f9aab109c17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:46.675613 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f5403204-2726-4574-94d5-0f9aab109c17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:46.826145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f5403204-2726-4574-94d5-0f9aab109c17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:46.834981 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f5403204-2726-4574-94d5-0f9aab109c17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:46.836418 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f5403204-2726-4574-94d5-0f9aab109c17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:46.837596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f5403204-2726-4574-94d5-0f9aab109c17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.175381 Aug 28 01:17:46.838188 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 978/1727] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:46 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:47.684092 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-db50af91-cc2d-4681-8d63-b67f934847bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:47.684683 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-db50af91-cc2d-4681-8d63-b67f934847bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:47.692997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-db50af91-cc2d-4681-8d63-b67f934847bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:47.693884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-db50af91-cc2d-4681-8d63-b67f934847bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:47.694309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-db50af91-cc2d-4681-8d63-b67f934847bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:47.695015 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-db50af91-cc2d-4681-8d63-b67f934847bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:47.695453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-db50af91-cc2d-4681-8d63-b67f934847bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:47.695847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f0c467c6-0c8a-43e0-8bf3-4df5f169f28f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:47.696336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-db50af91-cc2d-4681-8d63-b67f934847bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:47.696598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f0c467c6-0c8a-43e0-8bf3-4df5f169f28f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:47.704030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0c467c6-0c8a-43e0-8bf3-4df5f169f28f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:47.704634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0c467c6-0c8a-43e0-8bf3-4df5f169f28f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:47.705205 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0c467c6-0c8a-43e0-8bf3-4df5f169f28f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:47.706092 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0c467c6-0c8a-43e0-8bf3-4df5f169f28f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:47.706715 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0c467c6-0c8a-43e0-8bf3-4df5f169f28f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:47.707309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0c467c6-0c8a-43e0-8bf3-4df5f169f28f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:47.902196 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-db50af91-cc2d-4681-8d63-b67f934847bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:47.910292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f0c467c6-0c8a-43e0-8bf3-4df5f169f28f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:47.911173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-db50af91-cc2d-4681-8d63-b67f934847bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.230914 Aug 28 01:17:47.911745 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 750/1728] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:47 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:47.916393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7331a95c-b633-42e6-af95-4d065cde8ee3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:47.917027 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7331a95c-b633-42e6-af95-4d065cde8ee3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:47.920342 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f0c467c6-0c8a-43e0-8bf3-4df5f169f28f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.229393 Aug 28 01:17:47.921209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 979/1729] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:47 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:47.923818 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7331a95c-b633-42e6-af95-4d065cde8ee3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:47.924306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7331a95c-b633-42e6-af95-4d065cde8ee3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:47.924913 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7331a95c-b633-42e6-af95-4d065cde8ee3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:47.925840 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7331a95c-b633-42e6-af95-4d065cde8ee3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:47.926285 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7331a95c-b633-42e6-af95-4d065cde8ee3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:47.926699 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7331a95c-b633-42e6-af95-4d065cde8ee3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:48.067708 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7331a95c-b633-42e6-af95-4d065cde8ee3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:48.077605 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7331a95c-b633-42e6-af95-4d065cde8ee3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:48.079109 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7331a95c-b633-42e6-af95-4d065cde8ee3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:48.080316 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7331a95c-b633-42e6-af95-4d065cde8ee3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.167649 Aug 28 01:17:48.080875 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 751/1730] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:47 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:48.932641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-51aaa2d8-91f2-4f99-8c5b-ca633d375e3a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:48.933313 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-51aaa2d8-91f2-4f99-8c5b-ca633d375e3a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:48.940226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-51aaa2d8-91f2-4f99-8c5b-ca633d375e3a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:48.940716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-51aaa2d8-91f2-4f99-8c5b-ca633d375e3a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:48.941186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-51aaa2d8-91f2-4f99-8c5b-ca633d375e3a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:48.941888 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-51aaa2d8-91f2-4f99-8c5b-ca633d375e3a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:48.942033 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9b26047e-1ecf-4b88-81ec-d0da74f15bcf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:48.942447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-51aaa2d8-91f2-4f99-8c5b-ca633d375e3a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:48.942556 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9b26047e-1ecf-4b88-81ec-d0da74f15bcf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:48.942720 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-51aaa2d8-91f2-4f99-8c5b-ca633d375e3a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:48.949893 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b26047e-1ecf-4b88-81ec-d0da74f15bcf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:48.950389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b26047e-1ecf-4b88-81ec-d0da74f15bcf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:48.950813 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b26047e-1ecf-4b88-81ec-d0da74f15bcf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:48.951507 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b26047e-1ecf-4b88-81ec-d0da74f15bcf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:48.951938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b26047e-1ecf-4b88-81ec-d0da74f15bcf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:48.952357 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b26047e-1ecf-4b88-81ec-d0da74f15bcf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:49.147474 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9b26047e-1ecf-4b88-81ec-d0da74f15bcf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:49.155061 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-51aaa2d8-91f2-4f99-8c5b-ca633d375e3a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:49.156256 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9b26047e-1ecf-4b88-81ec-d0da74f15bcf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.217976 Aug 28 01:17:49.157269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 752/1731] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:48 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:49.161763 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b84077f5-ddae-4de9-a8fb-6aa8b97f8a2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:49.162360 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b84077f5-ddae-4de9-a8fb-6aa8b97f8a2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:49.168202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b84077f5-ddae-4de9-a8fb-6aa8b97f8a2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:49.168676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b84077f5-ddae-4de9-a8fb-6aa8b97f8a2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:49.169130 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b84077f5-ddae-4de9-a8fb-6aa8b97f8a2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:49.169831 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b84077f5-ddae-4de9-a8fb-6aa8b97f8a2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:49.170270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b84077f5-ddae-4de9-a8fb-6aa8b97f8a2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:49.170684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b84077f5-ddae-4de9-a8fb-6aa8b97f8a2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:49.170795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-51aaa2d8-91f2-4f99-8c5b-ca633d375e3a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.240597 Aug 28 01:17:49.171383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 980/1732] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:48 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:49.312930 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b84077f5-ddae-4de9-a8fb-6aa8b97f8a2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:49.326836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b84077f5-ddae-4de9-a8fb-6aa8b97f8a2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:49.328846 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b84077f5-ddae-4de9-a8fb-6aa8b97f8a2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:49.330020 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b84077f5-ddae-4de9-a8fb-6aa8b97f8a2c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.171805 Aug 28 01:17:49.330657 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 753/1733] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:49 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:50.179055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ad194f9b-d516-49cb-8888-c60eb2d7b477 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:50.179649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ad194f9b-d516-49cb-8888-c60eb2d7b477 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:50.186512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ad194f9b-d516-49cb-8888-c60eb2d7b477 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:50.188362 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ad194f9b-d516-49cb-8888-c60eb2d7b477 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:50.188362 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ad194f9b-d516-49cb-8888-c60eb2d7b477 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:50.188362 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ad194f9b-d516-49cb-8888-c60eb2d7b477 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:50.188631 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ad194f9b-d516-49cb-8888-c60eb2d7b477 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:50.188899 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ad194f9b-d516-49cb-8888-c60eb2d7b477 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:50.194397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-796a70fc-be9c-42a0-9b55-3df26454305f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:50.201000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-796a70fc-be9c-42a0-9b55-3df26454305f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:50.208393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-796a70fc-be9c-42a0-9b55-3df26454305f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:50.208867 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-796a70fc-be9c-42a0-9b55-3df26454305f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:50.210257 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-796a70fc-be9c-42a0-9b55-3df26454305f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:50.211038 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-796a70fc-be9c-42a0-9b55-3df26454305f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:50.211530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-796a70fc-be9c-42a0-9b55-3df26454305f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:50.212099 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-796a70fc-be9c-42a0-9b55-3df26454305f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:50.382820 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ad194f9b-d516-49cb-8888-c60eb2d7b477 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:50.391831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ad194f9b-d516-49cb-8888-c60eb2d7b477 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.216573 Aug 28 01:17:50.392436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 981/1734] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:50 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:50.397282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9745e959-cc84-4833-8ba8-9d58ee3ec0b2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:50.397884 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9745e959-cc84-4833-8ba8-9d58ee3ec0b2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:50.406880 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-796a70fc-be9c-42a0-9b55-3df26454305f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:50.408211 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9745e959-cc84-4833-8ba8-9d58ee3ec0b2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:50.408704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9745e959-cc84-4833-8ba8-9d58ee3ec0b2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:50.409169 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9745e959-cc84-4833-8ba8-9d58ee3ec0b2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:50.409938 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9745e959-cc84-4833-8ba8-9d58ee3ec0b2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:50.410314 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9745e959-cc84-4833-8ba8-9d58ee3ec0b2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:50.410709 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9745e959-cc84-4833-8ba8-9d58ee3ec0b2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:50.415556 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-796a70fc-be9c-42a0-9b55-3df26454305f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.225625 Aug 28 01:17:50.416244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 754/1735] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:50 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:50.549185 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9745e959-cc84-4833-8ba8-9d58ee3ec0b2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:50.562853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9745e959-cc84-4833-8ba8-9d58ee3ec0b2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:50.564304 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9745e959-cc84-4833-8ba8-9d58ee3ec0b2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:50.565503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9745e959-cc84-4833-8ba8-9d58ee3ec0b2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.172122 Aug 28 01:17:50.566045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 982/1736] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:50 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:51.414731 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1a7152ff-3589-4d59-a9cb-53b527c93872 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:51.415356 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1a7152ff-3589-4d59-a9cb-53b527c93872 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:51.422146 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1a7152ff-3589-4d59-a9cb-53b527c93872 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:51.422637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1a7152ff-3589-4d59-a9cb-53b527c93872 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:51.423065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1a7152ff-3589-4d59-a9cb-53b527c93872 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:51.423848 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1a7152ff-3589-4d59-a9cb-53b527c93872 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:51.424299 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1a7152ff-3589-4d59-a9cb-53b527c93872 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:51.424715 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1a7152ff-3589-4d59-a9cb-53b527c93872 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:51.440138 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fd1d8d09-df5b-4bbf-8a5a-b1b79fdb0664 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:51.440813 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fd1d8d09-df5b-4bbf-8a5a-b1b79fdb0664 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:51.449250 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd1d8d09-df5b-4bbf-8a5a-b1b79fdb0664 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:51.449733 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd1d8d09-df5b-4bbf-8a5a-b1b79fdb0664 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:51.450170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd1d8d09-df5b-4bbf-8a5a-b1b79fdb0664 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:51.450859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd1d8d09-df5b-4bbf-8a5a-b1b79fdb0664 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:51.451311 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd1d8d09-df5b-4bbf-8a5a-b1b79fdb0664 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:51.451723 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fd1d8d09-df5b-4bbf-8a5a-b1b79fdb0664 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:51.619222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1a7152ff-3589-4d59-a9cb-53b527c93872 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:51.628763 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1a7152ff-3589-4d59-a9cb-53b527c93872 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.218083 Aug 28 01:17:51.629455 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 755/1737] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:51 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:51.634426 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0cccdb0e-a40f-4ce5-bb88-f19750fb8d9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:51.635160 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0cccdb0e-a40f-4ce5-bb88-f19750fb8d9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:51.641994 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0cccdb0e-a40f-4ce5-bb88-f19750fb8d9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:51.642584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0cccdb0e-a40f-4ce5-bb88-f19750fb8d9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:51.642946 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fd1d8d09-df5b-4bbf-8a5a-b1b79fdb0664 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:51.643075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0cccdb0e-a40f-4ce5-bb88-f19750fb8d9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:51.643767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0cccdb0e-a40f-4ce5-bb88-f19750fb8d9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:51.644210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0cccdb0e-a40f-4ce5-bb88-f19750fb8d9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:51.644630 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0cccdb0e-a40f-4ce5-bb88-f19750fb8d9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:51.652572 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fd1d8d09-df5b-4bbf-8a5a-b1b79fdb0664 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.216456 Aug 28 01:17:51.653519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 983/1738] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:51 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:51.786807 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0cccdb0e-a40f-4ce5-bb88-f19750fb8d9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:51.799791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0cccdb0e-a40f-4ce5-bb88-f19750fb8d9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:51.801229 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0cccdb0e-a40f-4ce5-bb88-f19750fb8d9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:51.802519 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0cccdb0e-a40f-4ce5-bb88-f19750fb8d9f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.172081 Aug 28 01:17:51.803120 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 756/1739] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:51 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:52.651608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-65eb4594-8bb0-4ef3-86fb-5b77f2e1b3c4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:52.652333 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-65eb4594-8bb0-4ef3-86fb-5b77f2e1b3c4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:52.659345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65eb4594-8bb0-4ef3-86fb-5b77f2e1b3c4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:52.659825 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65eb4594-8bb0-4ef3-86fb-5b77f2e1b3c4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:52.660255 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65eb4594-8bb0-4ef3-86fb-5b77f2e1b3c4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:52.660935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65eb4594-8bb0-4ef3-86fb-5b77f2e1b3c4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:52.661447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65eb4594-8bb0-4ef3-86fb-5b77f2e1b3c4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:52.661869 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65eb4594-8bb0-4ef3-86fb-5b77f2e1b3c4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:52.674485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7a0f92e6-e3d6-4a8a-9f6b-7c2549d0127b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:52.675190 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7a0f92e6-e3d6-4a8a-9f6b-7c2549d0127b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:52.682186 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7a0f92e6-e3d6-4a8a-9f6b-7c2549d0127b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:52.682663 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7a0f92e6-e3d6-4a8a-9f6b-7c2549d0127b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:52.683087 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7a0f92e6-e3d6-4a8a-9f6b-7c2549d0127b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:52.683811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7a0f92e6-e3d6-4a8a-9f6b-7c2549d0127b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:52.684280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7a0f92e6-e3d6-4a8a-9f6b-7c2549d0127b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:52.684730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7a0f92e6-e3d6-4a8a-9f6b-7c2549d0127b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:52.873894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-65eb4594-8bb0-4ef3-86fb-5b77f2e1b3c4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:52.882467 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7a0f92e6-e3d6-4a8a-9f6b-7c2549d0127b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:52.883014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-65eb4594-8bb0-4ef3-86fb-5b77f2e1b3c4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.235479 Aug 28 01:17:52.883597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 984/1740] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:52 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:52.892836 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-620abbfb-4c5e-4ae9-8522-d6eafb2326d4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:52.892836 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-620abbfb-4c5e-4ae9-8522-d6eafb2326d4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:52.899338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-620abbfb-4c5e-4ae9-8522-d6eafb2326d4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:52.899833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-620abbfb-4c5e-4ae9-8522-d6eafb2326d4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:52.900271 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-620abbfb-4c5e-4ae9-8522-d6eafb2326d4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:52.901029 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-620abbfb-4c5e-4ae9-8522-d6eafb2326d4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:52.901484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-620abbfb-4c5e-4ae9-8522-d6eafb2326d4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:52.901770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7a0f92e6-e3d6-4a8a-9f6b-7c2549d0127b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.231045 Aug 28 01:17:52.901956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-620abbfb-4c5e-4ae9-8522-d6eafb2326d4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:52.902306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 757/1741] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:52 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:53.079023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-620abbfb-4c5e-4ae9-8522-d6eafb2326d4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:53.089855 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-620abbfb-4c5e-4ae9-8522-d6eafb2326d4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:53.091331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-620abbfb-4c5e-4ae9-8522-d6eafb2326d4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:53.092527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-620abbfb-4c5e-4ae9-8522-d6eafb2326d4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.207987 Aug 28 01:17:53.093121 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 985/1742] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:52 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:53.909495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4d253412-31bd-4295-87ef-f18442f60d0f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:53.910151 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4d253412-31bd-4295-87ef-f18442f60d0f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:53.917378 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d253412-31bd-4295-87ef-f18442f60d0f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:53.917869 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d253412-31bd-4295-87ef-f18442f60d0f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:53.918629 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d253412-31bd-4295-87ef-f18442f60d0f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:53.918954 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d253412-31bd-4295-87ef-f18442f60d0f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:53.919402 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d253412-31bd-4295-87ef-f18442f60d0f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:53.919817 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d253412-31bd-4295-87ef-f18442f60d0f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:53.924295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cd6192c6-1743-4135-9251-84da2c9e020f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:53.925056 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cd6192c6-1743-4135-9251-84da2c9e020f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:53.931941 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cd6192c6-1743-4135-9251-84da2c9e020f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:53.932425 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cd6192c6-1743-4135-9251-84da2c9e020f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:53.932848 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cd6192c6-1743-4135-9251-84da2c9e020f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:53.933560 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cd6192c6-1743-4135-9251-84da2c9e020f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:53.934020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cd6192c6-1743-4135-9251-84da2c9e020f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:53.934442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cd6192c6-1743-4135-9251-84da2c9e020f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:54.120043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4d253412-31bd-4295-87ef-f18442f60d0f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:54.129458 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4d253412-31bd-4295-87ef-f18442f60d0f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.223594 Aug 28 01:17:54.130117 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 758/1743] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:53 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:54.132105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-cd6192c6-1743-4135-9251-84da2c9e020f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:54.135723 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bd7c0dc0-e22f-4319-99ac-8c84c489ec02 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:54.136426 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bd7c0dc0-e22f-4319-99ac-8c84c489ec02 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:54.141289 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cd6192c6-1743-4135-9251-84da2c9e020f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.221049 Aug 28 01:17:54.142138 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 986/1744] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:53 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:17:54.142980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd7c0dc0-e22f-4319-99ac-8c84c489ec02 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:54.143473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd7c0dc0-e22f-4319-99ac-8c84c489ec02 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:54.143896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd7c0dc0-e22f-4319-99ac-8c84c489ec02 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:54.145205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd7c0dc0-e22f-4319-99ac-8c84c489ec02 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:54.145205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd7c0dc0-e22f-4319-99ac-8c84c489ec02 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:54.145459 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd7c0dc0-e22f-4319-99ac-8c84c489ec02 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:54.283681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bd7c0dc0-e22f-4319-99ac-8c84c489ec02 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:54.292529 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bd7c0dc0-e22f-4319-99ac-8c84c489ec02 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:54.293983 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bd7c0dc0-e22f-4319-99ac-8c84c489ec02 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:54.295141 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bd7c0dc0-e22f-4319-99ac-8c84c489ec02 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.163913 Aug 28 01:17:54.295692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 759/1745] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:54 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:55.151923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d015de9a-531b-4e14-b0f9-fea110f73b60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:55.152495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d015de9a-531b-4e14-b0f9-fea110f73b60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:55.159460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d015de9a-531b-4e14-b0f9-fea110f73b60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:55.159944 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d015de9a-531b-4e14-b0f9-fea110f73b60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:55.160370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d015de9a-531b-4e14-b0f9-fea110f73b60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:55.161279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d015de9a-531b-4e14-b0f9-fea110f73b60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:55.161698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d015de9a-531b-4e14-b0f9-fea110f73b60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:55.162146 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d015de9a-531b-4e14-b0f9-fea110f73b60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:55.163983 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-31aeeb27-2651-4da7-a545-5d2313646f54 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:55.164625 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-31aeeb27-2651-4da7-a545-5d2313646f54 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:55.171257 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31aeeb27-2651-4da7-a545-5d2313646f54 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:55.171740 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31aeeb27-2651-4da7-a545-5d2313646f54 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:55.172179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31aeeb27-2651-4da7-a545-5d2313646f54 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:55.172872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31aeeb27-2651-4da7-a545-5d2313646f54 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:55.173347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31aeeb27-2651-4da7-a545-5d2313646f54 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:55.173844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31aeeb27-2651-4da7-a545-5d2313646f54 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:55.362955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d015de9a-531b-4e14-b0f9-fea110f73b60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:55.370997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-31aeeb27-2651-4da7-a545-5d2313646f54 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:55.371600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d015de9a-531b-4e14-b0f9-fea110f73b60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.223579 Aug 28 01:17:55.372170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 987/1746] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:55 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:55.376667 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8c72ee30-0dcd-44b3-9522-02ac60f7d075 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:55.377319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8c72ee30-0dcd-44b3-9522-02ac60f7d075 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:55.379735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-31aeeb27-2651-4da7-a545-5d2313646f54 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.219586 Aug 28 01:17:55.380517 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 760/1747] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:55 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:55.383137 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8c72ee30-0dcd-44b3-9522-02ac60f7d075 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:55.383611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8c72ee30-0dcd-44b3-9522-02ac60f7d075 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:55.384019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8c72ee30-0dcd-44b3-9522-02ac60f7d075 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:55.384705 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8c72ee30-0dcd-44b3-9522-02ac60f7d075 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:55.385256 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8c72ee30-0dcd-44b3-9522-02ac60f7d075 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:55.385673 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8c72ee30-0dcd-44b3-9522-02ac60f7d075 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:55.536364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8c72ee30-0dcd-44b3-9522-02ac60f7d075 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:55.545285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8c72ee30-0dcd-44b3-9522-02ac60f7d075 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:55.546797 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8c72ee30-0dcd-44b3-9522-02ac60f7d075 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:55.547996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8c72ee30-0dcd-44b3-9522-02ac60f7d075 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.174950 Aug 28 01:17:55.548619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 988/1748] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:55 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:56.402585 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7dd14540-2aa9-4c10-bb53-bbc9c3a56464 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:56.403312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2c14d647-35fe-44fe-9da4-00eac620bb60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:56.403729 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7dd14540-2aa9-4c10-bb53-bbc9c3a56464 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:56.403837 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2c14d647-35fe-44fe-9da4-00eac620bb60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:56.411497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c14d647-35fe-44fe-9da4-00eac620bb60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:56.411698 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7dd14540-2aa9-4c10-bb53-bbc9c3a56464 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:56.411907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c14d647-35fe-44fe-9da4-00eac620bb60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:56.412002 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7dd14540-2aa9-4c10-bb53-bbc9c3a56464 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:56.412090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c14d647-35fe-44fe-9da4-00eac620bb60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:56.412177 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7dd14540-2aa9-4c10-bb53-bbc9c3a56464 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:56.412474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c14d647-35fe-44fe-9da4-00eac620bb60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:56.412636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7dd14540-2aa9-4c10-bb53-bbc9c3a56464 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:56.412915 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7dd14540-2aa9-4c10-bb53-bbc9c3a56464 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:56.413095 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c14d647-35fe-44fe-9da4-00eac620bb60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:56.413372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7dd14540-2aa9-4c10-bb53-bbc9c3a56464 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:56.413511 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c14d647-35fe-44fe-9da4-00eac620bb60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:56.428920 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:17:56.609153 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7dd14540-2aa9-4c10-bb53-bbc9c3a56464 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:56.612075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:17:56.612620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2c14d647-35fe-44fe-9da4-00eac620bb60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:56.619923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7dd14540-2aa9-4c10-bb53-bbc9c3a56464 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.221802 Aug 28 01:17:56.620710 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 761/1749] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:56 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:56.625350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-00bdb176-c80c-4688-93da-312841e49ce2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:56.625961 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-00bdb176-c80c-4688-93da-312841e49ce2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:56.632219 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00bdb176-c80c-4688-93da-312841e49ce2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:56.632698 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00bdb176-c80c-4688-93da-312841e49ce2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:56.633066 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2c14d647-35fe-44fe-9da4-00eac620bb60 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.222366 Aug 28 01:17:56.633198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00bdb176-c80c-4688-93da-312841e49ce2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:56.639790 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00bdb176-c80c-4688-93da-312841e49ce2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:56.639790 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00bdb176-c80c-4688-93da-312841e49ce2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:56.639790 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00bdb176-c80c-4688-93da-312841e49ce2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:56.640149 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 989/1750] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:56 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:17:56.768899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-00bdb176-c80c-4688-93da-312841e49ce2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:56.778511 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-00bdb176-c80c-4688-93da-312841e49ce2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:56.779941 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-00bdb176-c80c-4688-93da-312841e49ce2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:56.781139 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-00bdb176-c80c-4688-93da-312841e49ce2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.159623 Aug 28 01:17:56.781698 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 762/1751] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:56 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:57.643164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9550e565-accc-4f1c-8944-5cb2991595e5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:57.643867 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9550e565-accc-4f1c-8944-5cb2991595e5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:57.651804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9550e565-accc-4f1c-8944-5cb2991595e5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:57.652704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9550e565-accc-4f1c-8944-5cb2991595e5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:57.653594 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9550e565-accc-4f1c-8944-5cb2991595e5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:57.654553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9550e565-accc-4f1c-8944-5cb2991595e5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:57.654674 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f044e8cf-c7d5-45b7-a507-6060a50e9c88 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:57.655283 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f044e8cf-c7d5-45b7-a507-6060a50e9c88 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:57.656043 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9550e565-accc-4f1c-8944-5cb2991595e5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:57.656833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9550e565-accc-4f1c-8944-5cb2991595e5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:57.661759 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f044e8cf-c7d5-45b7-a507-6060a50e9c88 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:57.662191 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f044e8cf-c7d5-45b7-a507-6060a50e9c88 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:57.662900 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f044e8cf-c7d5-45b7-a507-6060a50e9c88 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:57.663573 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f044e8cf-c7d5-45b7-a507-6060a50e9c88 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:57.664238 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f044e8cf-c7d5-45b7-a507-6060a50e9c88 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:57.664649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f044e8cf-c7d5-45b7-a507-6060a50e9c88 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:57.851722 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f044e8cf-c7d5-45b7-a507-6060a50e9c88 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:57.854943 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9550e565-accc-4f1c-8944-5cb2991595e5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:57.860418 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f044e8cf-c7d5-45b7-a507-6060a50e9c88 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.209315 Aug 28 01:17:57.861178 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 763/1752] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:57 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:57.863803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9550e565-accc-4f1c-8944-5cb2991595e5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.224426 Aug 28 01:17:57.864379 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 990/1753] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:57 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:57.865999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6e756afa-b274-40d8-82d6-a75b72b150f9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:57.866603 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6e756afa-b274-40d8-82d6-a75b72b150f9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:57.872628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6e756afa-b274-40d8-82d6-a75b72b150f9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:57.873100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6e756afa-b274-40d8-82d6-a75b72b150f9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:57.873819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6e756afa-b274-40d8-82d6-a75b72b150f9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:57.874584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6e756afa-b274-40d8-82d6-a75b72b150f9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:57.875038 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6e756afa-b274-40d8-82d6-a75b72b150f9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:57.875459 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6e756afa-b274-40d8-82d6-a75b72b150f9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:58.009321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6e756afa-b274-40d8-82d6-a75b72b150f9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:58.021795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6e756afa-b274-40d8-82d6-a75b72b150f9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:58.023667 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6e756afa-b274-40d8-82d6-a75b72b150f9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:58.025353 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6e756afa-b274-40d8-82d6-a75b72b150f9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.162744 Aug 28 01:17:58.026151 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 764/1754] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:57 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:17:58.884567 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-07a24bfc-23d4-4d5a-b9d7-c241b8bca1cf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:58.885427 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-07a24bfc-23d4-4d5a-b9d7-c241b8bca1cf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:58.890340 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-90f8bb47-ba71-4cc7-8e16-3e643ed6cbe2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:58.891219 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-90f8bb47-ba71-4cc7-8e16-3e643ed6cbe2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:58.895216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07a24bfc-23d4-4d5a-b9d7-c241b8bca1cf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:58.895902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07a24bfc-23d4-4d5a-b9d7-c241b8bca1cf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:58.896495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07a24bfc-23d4-4d5a-b9d7-c241b8bca1cf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:58.897568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07a24bfc-23d4-4d5a-b9d7-c241b8bca1cf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:58.898215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07a24bfc-23d4-4d5a-b9d7-c241b8bca1cf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:58.898796 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07a24bfc-23d4-4d5a-b9d7-c241b8bca1cf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:58.900009 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90f8bb47-ba71-4cc7-8e16-3e643ed6cbe2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:58.900494 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90f8bb47-ba71-4cc7-8e16-3e643ed6cbe2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:58.900925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90f8bb47-ba71-4cc7-8e16-3e643ed6cbe2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:58.901847 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90f8bb47-ba71-4cc7-8e16-3e643ed6cbe2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:58.902302 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90f8bb47-ba71-4cc7-8e16-3e643ed6cbe2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:58.902725 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90f8bb47-ba71-4cc7-8e16-3e643ed6cbe2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:59.091557 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-07a24bfc-23d4-4d5a-b9d7-c241b8bca1cf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:59.094390 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-90f8bb47-ba71-4cc7-8e16-3e643ed6cbe2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:59.100399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-07a24bfc-23d4-4d5a-b9d7-c241b8bca1cf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.221312 Aug 28 01:17:59.101091 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 991/1755] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:58 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:59.102721 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-90f8bb47-ba71-4cc7-8e16-3e643ed6cbe2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.217886 Aug 28 01:17:59.103247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 765/1756] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:17:58 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:17:59.105685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-37be2a63-99cd-432e-9f78-a0463f13a27a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:17:59.106262 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-37be2a63-99cd-432e-9f78-a0463f13a27a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:17:59.112447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-37be2a63-99cd-432e-9f78-a0463f13a27a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:59.112992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-37be2a63-99cd-432e-9f78-a0463f13a27a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:59.115546 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-37be2a63-99cd-432e-9f78-a0463f13a27a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:59.116541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-37be2a63-99cd-432e-9f78-a0463f13a27a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:17:59.117171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-37be2a63-99cd-432e-9f78-a0463f13a27a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:17:59.117634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-37be2a63-99cd-432e-9f78-a0463f13a27a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:17:59.251509 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-37be2a63-99cd-432e-9f78-a0463f13a27a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:59.262791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-37be2a63-99cd-432e-9f78-a0463f13a27a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:59.264146 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-37be2a63-99cd-432e-9f78-a0463f13a27a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:17:59.265343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-37be2a63-99cd-432e-9f78-a0463f13a27a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.163299 Aug 28 01:17:59.265873 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 992/1757] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:17:59 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:00.123701 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0b4a600a-3bb4-4ab0-8dab-2a3292660563 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:00.124384 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0b4a600a-3bb4-4ab0-8dab-2a3292660563 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:00.124569 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-db400b2a-3781-4ee9-8ed3-3fdab0e94c59 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:00.125159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-db400b2a-3781-4ee9-8ed3-3fdab0e94c59 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:00.127685 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=74 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:18:00.133816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db400b2a-3781-4ee9-8ed3-3fdab0e94c59 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:00.134170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db400b2a-3781-4ee9-8ed3-3fdab0e94c59 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:00.134599 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db400b2a-3781-4ee9-8ed3-3fdab0e94c59 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:00.135019 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0b4a600a-3bb4-4ab0-8dab-2a3292660563 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:00.135275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db400b2a-3781-4ee9-8ed3-3fdab0e94c59 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:00.135499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0b4a600a-3bb4-4ab0-8dab-2a3292660563 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:00.135738 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db400b2a-3781-4ee9-8ed3-3fdab0e94c59 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:00.135903 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0b4a600a-3bb4-4ab0-8dab-2a3292660563 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:00.136161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db400b2a-3781-4ee9-8ed3-3fdab0e94c59 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:00.136642 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0b4a600a-3bb4-4ab0-8dab-2a3292660563 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:00.137052 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0b4a600a-3bb4-4ab0-8dab-2a3292660563 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:00.137469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0b4a600a-3bb4-4ab0-8dab-2a3292660563 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:00.342777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-db400b2a-3781-4ee9-8ed3-3fdab0e94c59 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:00.346925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0b4a600a-3bb4-4ab0-8dab-2a3292660563 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:00.351074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-db400b2a-3781-4ee9-8ed3-3fdab0e94c59 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.231099 Aug 28 01:18:00.351628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 993/1758] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:00 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:00.355573 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0b4a600a-3bb4-4ab0-8dab-2a3292660563 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.236371 Aug 28 01:18:00.356076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-dabea85b-23b8-4d63-adeb-7d553f9b01e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:00.356433 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 766/1759] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:00 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:00.361000 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-dabea85b-23b8-4d63-adeb-7d553f9b01e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:00.362895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=74 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:18:00.368445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dabea85b-23b8-4d63-adeb-7d553f9b01e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:00.368953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dabea85b-23b8-4d63-adeb-7d553f9b01e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:00.369403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dabea85b-23b8-4d63-adeb-7d553f9b01e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:00.370091 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dabea85b-23b8-4d63-adeb-7d553f9b01e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:00.370536 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dabea85b-23b8-4d63-adeb-7d553f9b01e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:00.370950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dabea85b-23b8-4d63-adeb-7d553f9b01e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:00.516928 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dabea85b-23b8-4d63-adeb-7d553f9b01e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:00.525629 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dabea85b-23b8-4d63-adeb-7d553f9b01e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:00.527047 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dabea85b-23b8-4d63-adeb-7d553f9b01e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:00.529557 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-dabea85b-23b8-4d63-adeb-7d553f9b01e3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.176651 Aug 28 01:18:00.530381 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 994/1760] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:00 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:01.373585 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-489d077a-c75a-4138-93e0-4ed2df5040c3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:01.374154 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-489d077a-c75a-4138-93e0-4ed2df5040c3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:01.377072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-40856b13-561a-496d-9928-741bd4132f6e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:01.377831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-40856b13-561a-496d-9928-741bd4132f6e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:01.381096 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-489d077a-c75a-4138-93e0-4ed2df5040c3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:01.381595 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-489d077a-c75a-4138-93e0-4ed2df5040c3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:01.382468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-489d077a-c75a-4138-93e0-4ed2df5040c3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:01.382912 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-489d077a-c75a-4138-93e0-4ed2df5040c3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:01.383371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-489d077a-c75a-4138-93e0-4ed2df5040c3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:01.383805 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-489d077a-c75a-4138-93e0-4ed2df5040c3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:01.384424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-40856b13-561a-496d-9928-741bd4132f6e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:01.384900 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-40856b13-561a-496d-9928-741bd4132f6e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:01.385347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-40856b13-561a-496d-9928-741bd4132f6e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:01.386038 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-40856b13-561a-496d-9928-741bd4132f6e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:01.386475 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-40856b13-561a-496d-9928-741bd4132f6e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:01.386885 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-40856b13-561a-496d-9928-741bd4132f6e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:01.576234 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-489d077a-c75a-4138-93e0-4ed2df5040c3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:01.580356 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-40856b13-561a-496d-9928-741bd4132f6e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:01.584549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-489d077a-c75a-4138-93e0-4ed2df5040c3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.214985 Aug 28 01:18:01.585149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 767/1761] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:01 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:01.589076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-40856b13-561a-496d-9928-741bd4132f6e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.215830 Aug 28 01:18:01.589636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-966adf4e-6bbd-4140-8dc2-ad1fecf31bde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:01.589830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 995/1762] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:01 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:01.590665 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-966adf4e-6bbd-4140-8dc2-ad1fecf31bde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:01.596668 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-966adf4e-6bbd-4140-8dc2-ad1fecf31bde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:01.598445 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-966adf4e-6bbd-4140-8dc2-ad1fecf31bde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:01.598445 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-966adf4e-6bbd-4140-8dc2-ad1fecf31bde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:01.600769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-966adf4e-6bbd-4140-8dc2-ad1fecf31bde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:01.601230 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-966adf4e-6bbd-4140-8dc2-ad1fecf31bde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:01.601658 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-966adf4e-6bbd-4140-8dc2-ad1fecf31bde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:01.739177 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-966adf4e-6bbd-4140-8dc2-ad1fecf31bde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:01.749277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-966adf4e-6bbd-4140-8dc2-ad1fecf31bde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:01.750684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-966adf4e-6bbd-4140-8dc2-ad1fecf31bde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:01.751844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-966adf4e-6bbd-4140-8dc2-ad1fecf31bde tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.165753 Aug 28 01:18:01.752376 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 768/1763] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:01 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:02.607599 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1034979d-78ee-43b7-9d7c-89c41db189f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:02.608220 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1034979d-78ee-43b7-9d7c-89c41db189f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:02.609977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3cc050bd-c3ea-4d42-9e23-b3bf08d3b8f7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:02.610587 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3cc050bd-c3ea-4d42-9e23-b3bf08d3b8f7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:02.615150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1034979d-78ee-43b7-9d7c-89c41db189f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:02.615698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1034979d-78ee-43b7-9d7c-89c41db189f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:02.617315 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cc050bd-c3ea-4d42-9e23-b3bf08d3b8f7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:02.617795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cc050bd-c3ea-4d42-9e23-b3bf08d3b8f7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:02.618214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cc050bd-c3ea-4d42-9e23-b3bf08d3b8f7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:02.618714 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1034979d-78ee-43b7-9d7c-89c41db189f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:02.618910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cc050bd-c3ea-4d42-9e23-b3bf08d3b8f7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:02.619439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cc050bd-c3ea-4d42-9e23-b3bf08d3b8f7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:02.619557 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1034979d-78ee-43b7-9d7c-89c41db189f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:02.619849 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cc050bd-c3ea-4d42-9e23-b3bf08d3b8f7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:02.620131 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1034979d-78ee-43b7-9d7c-89c41db189f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:02.620575 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1034979d-78ee-43b7-9d7c-89c41db189f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:02.826252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3cc050bd-c3ea-4d42-9e23-b3bf08d3b8f7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:02.829093 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1034979d-78ee-43b7-9d7c-89c41db189f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:02.835176 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3cc050bd-c3ea-4d42-9e23-b3bf08d3b8f7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.228958 Aug 28 01:18:02.835779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 769/1764] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:02 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:02.838192 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1034979d-78ee-43b7-9d7c-89c41db189f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.234637 Aug 28 01:18:02.838848 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 996/1765] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:02 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:02.845210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-be31cdb3-bf45-4910-beee-b80454cbc09e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:02.846111 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-be31cdb3-bf45-4910-beee-b80454cbc09e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:02.852932 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-be31cdb3-bf45-4910-beee-b80454cbc09e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:02.853604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-be31cdb3-bf45-4910-beee-b80454cbc09e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:02.854054 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-be31cdb3-bf45-4910-beee-b80454cbc09e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:02.854752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-be31cdb3-bf45-4910-beee-b80454cbc09e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:02.855206 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-be31cdb3-bf45-4910-beee-b80454cbc09e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:02.855622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-be31cdb3-bf45-4910-beee-b80454cbc09e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:03.002509 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-be31cdb3-bf45-4910-beee-b80454cbc09e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:03.011334 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-be31cdb3-bf45-4910-beee-b80454cbc09e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:03.012759 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-be31cdb3-bf45-4910-beee-b80454cbc09e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:03.013948 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-be31cdb3-bf45-4910-beee-b80454cbc09e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.173009 Aug 28 01:18:03.014504 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 770/1766] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:02 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:03.858391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0590b432-dbf6-4541-9eca-906185471bc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:03.859017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0590b432-dbf6-4541-9eca-906185471bc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:03.859378 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fae808b2-03e3-4a04-a57b-ac7261e08f84 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:03.859992 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fae808b2-03e3-4a04-a57b-ac7261e08f84 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:03.865833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0590b432-dbf6-4541-9eca-906185471bc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:03.866312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0590b432-dbf6-4541-9eca-906185471bc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:03.866749 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0590b432-dbf6-4541-9eca-906185471bc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:03.866910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fae808b2-03e3-4a04-a57b-ac7261e08f84 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:03.867390 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fae808b2-03e3-4a04-a57b-ac7261e08f84 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:03.867559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0590b432-dbf6-4541-9eca-906185471bc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:03.867802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fae808b2-03e3-4a04-a57b-ac7261e08f84 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:03.867912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0590b432-dbf6-4541-9eca-906185471bc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:03.868258 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0590b432-dbf6-4541-9eca-906185471bc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:03.868497 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fae808b2-03e3-4a04-a57b-ac7261e08f84 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:03.868945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fae808b2-03e3-4a04-a57b-ac7261e08f84 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:03.869394 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fae808b2-03e3-4a04-a57b-ac7261e08f84 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:04.067438 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fae808b2-03e3-4a04-a57b-ac7261e08f84 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:04.072756 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0590b432-dbf6-4541-9eca-906185471bc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:04.076144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fae808b2-03e3-4a04-a57b-ac7261e08f84 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.220574 Aug 28 01:18:04.076707 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 771/1767] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:03 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:04.081144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e9e05bc1-88e5-4fd4-9c5e-9235392a25fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:04.081694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0590b432-dbf6-4541-9eca-906185471bc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.227167 Aug 28 01:18:04.081836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e9e05bc1-88e5-4fd4-9c5e-9235392a25fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:04.082520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 997/1768] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:03 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:04.087530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e9e05bc1-88e5-4fd4-9c5e-9235392a25fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:04.088065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e9e05bc1-88e5-4fd4-9c5e-9235392a25fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:04.088488 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e9e05bc1-88e5-4fd4-9c5e-9235392a25fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:04.092149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e9e05bc1-88e5-4fd4-9c5e-9235392a25fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:04.092149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e9e05bc1-88e5-4fd4-9c5e-9235392a25fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:04.092149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e9e05bc1-88e5-4fd4-9c5e-9235392a25fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:04.232581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e9e05bc1-88e5-4fd4-9c5e-9235392a25fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:04.244950 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e9e05bc1-88e5-4fd4-9c5e-9235392a25fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:04.246393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e9e05bc1-88e5-4fd4-9c5e-9235392a25fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:04.247648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e9e05bc1-88e5-4fd4-9c5e-9235392a25fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.170019 Aug 28 01:18:04.248210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 772/1769] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:04 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:05.096742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5ebdaa41-3adc-4aea-8fcb-fd3e66615c0d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:05.097399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5ebdaa41-3adc-4aea-8fcb-fd3e66615c0d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:05.104522 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ebdaa41-3adc-4aea-8fcb-fd3e66615c0d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:05.105071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ebdaa41-3adc-4aea-8fcb-fd3e66615c0d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:05.105889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ebdaa41-3adc-4aea-8fcb-fd3e66615c0d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:05.106601 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ebdaa41-3adc-4aea-8fcb-fd3e66615c0d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:05.107445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ebdaa41-3adc-4aea-8fcb-fd3e66615c0d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:05.107767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-da30c7b0-c6b9-41dd-bd8c-7cc7d2a7e3ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:05.108211 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ebdaa41-3adc-4aea-8fcb-fd3e66615c0d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:05.109123 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-da30c7b0-c6b9-41dd-bd8c-7cc7d2a7e3ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:05.131033 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da30c7b0-c6b9-41dd-bd8c-7cc7d2a7e3ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:05.131525 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da30c7b0-c6b9-41dd-bd8c-7cc7d2a7e3ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:05.131931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da30c7b0-c6b9-41dd-bd8c-7cc7d2a7e3ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:05.132665 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da30c7b0-c6b9-41dd-bd8c-7cc7d2a7e3ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:05.133137 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da30c7b0-c6b9-41dd-bd8c-7cc7d2a7e3ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:05.133573 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da30c7b0-c6b9-41dd-bd8c-7cc7d2a7e3ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:05.318257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5ebdaa41-3adc-4aea-8fcb-fd3e66615c0d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:05.322378 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-da30c7b0-c6b9-41dd-bd8c-7cc7d2a7e3ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:05.327213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5ebdaa41-3adc-4aea-8fcb-fd3e66615c0d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.234341 Aug 28 01:18:05.327992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 998/1770] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:05 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:05.331276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-da30c7b0-c6b9-41dd-bd8c-7cc7d2a7e3ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.227327 Aug 28 01:18:05.332101 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 773/1771] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:05 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:05.340417 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-05fafb7e-9b43-4a99-847b-9c27743c38b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:05.341062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-05fafb7e-9b43-4a99-847b-9c27743c38b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:05.347346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05fafb7e-9b43-4a99-847b-9c27743c38b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:05.347868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05fafb7e-9b43-4a99-847b-9c27743c38b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:05.348293 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05fafb7e-9b43-4a99-847b-9c27743c38b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:05.349023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05fafb7e-9b43-4a99-847b-9c27743c38b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:05.349538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05fafb7e-9b43-4a99-847b-9c27743c38b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:05.349977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05fafb7e-9b43-4a99-847b-9c27743c38b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:05.489600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-05fafb7e-9b43-4a99-847b-9c27743c38b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:05.503135 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-05fafb7e-9b43-4a99-847b-9c27743c38b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:05.505399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-05fafb7e-9b43-4a99-847b-9c27743c38b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:05.506791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-05fafb7e-9b43-4a99-847b-9c27743c38b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.177841 Aug 28 01:18:05.507533 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 999/1772] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:05 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:06.349953 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8207332a-9447-468f-b892-3250398509de tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:06.350621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8207332a-9447-468f-b892-3250398509de tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:06.352228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-62a6153c-ff78-4f91-9a19-aae5d0b158b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:06.352890 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-62a6153c-ff78-4f91-9a19-aae5d0b158b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:06.358627 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8207332a-9447-468f-b892-3250398509de tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:06.359123 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8207332a-9447-468f-b892-3250398509de tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:06.359580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8207332a-9447-468f-b892-3250398509de tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:06.360024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62a6153c-ff78-4f91-9a19-aae5d0b158b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:06.360293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8207332a-9447-468f-b892-3250398509de tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:06.360515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62a6153c-ff78-4f91-9a19-aae5d0b158b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:06.360715 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8207332a-9447-468f-b892-3250398509de tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:06.360929 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62a6153c-ff78-4f91-9a19-aae5d0b158b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:06.361293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8207332a-9447-468f-b892-3250398509de tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:06.361661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62a6153c-ff78-4f91-9a19-aae5d0b158b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:06.362096 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62a6153c-ff78-4f91-9a19-aae5d0b158b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:06.362525 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62a6153c-ff78-4f91-9a19-aae5d0b158b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:06.556100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8207332a-9447-468f-b892-3250398509de tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:06.560477 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-62a6153c-ff78-4f91-9a19-aae5d0b158b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:06.568279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8207332a-9447-468f-b892-3250398509de tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.222028 Aug 28 01:18:06.568902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 774/1773] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:06 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:06.573640 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-69b054b4-ce36-43e5-97dc-c624a282a482 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:06.574364 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-69b054b4-ce36-43e5-97dc-c624a282a482 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:06.576243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-62a6153c-ff78-4f91-9a19-aae5d0b158b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.228237 Aug 28 01:18:06.577148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1000/1774] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:06 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:06.580828 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-69b054b4-ce36-43e5-97dc-c624a282a482 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:06.581421 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-69b054b4-ce36-43e5-97dc-c624a282a482 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:06.581935 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-69b054b4-ce36-43e5-97dc-c624a282a482 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:06.582748 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-69b054b4-ce36-43e5-97dc-c624a282a482 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:06.583197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-69b054b4-ce36-43e5-97dc-c624a282a482 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:06.583645 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-69b054b4-ce36-43e5-97dc-c624a282a482 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:06.718744 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-69b054b4-ce36-43e5-97dc-c624a282a482 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:06.728028 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-69b054b4-ce36-43e5-97dc-c624a282a482 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:06.729413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-69b054b4-ce36-43e5-97dc-c624a282a482 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:06.730689 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-69b054b4-ce36-43e5-97dc-c624a282a482 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.160821 Aug 28 01:18:06.731220 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 775/1775] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:06 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:07.592036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9083dcd0-bbb8-4b27-af7b-c8fab8d94aef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:07.592713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9083dcd0-bbb8-4b27-af7b-c8fab8d94aef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:07.597462 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-12a83696-afe5-4c00-81a9-2db91fab99f0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:07.598066 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-12a83696-afe5-4c00-81a9-2db91fab99f0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:07.599421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9083dcd0-bbb8-4b27-af7b-c8fab8d94aef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:07.599915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9083dcd0-bbb8-4b27-af7b-c8fab8d94aef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:07.600345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9083dcd0-bbb8-4b27-af7b-c8fab8d94aef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:07.601063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9083dcd0-bbb8-4b27-af7b-c8fab8d94aef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:07.601510 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9083dcd0-bbb8-4b27-af7b-c8fab8d94aef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:07.601926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9083dcd0-bbb8-4b27-af7b-c8fab8d94aef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:07.604873 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-12a83696-afe5-4c00-81a9-2db91fab99f0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:07.605485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-12a83696-afe5-4c00-81a9-2db91fab99f0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:07.605857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-12a83696-afe5-4c00-81a9-2db91fab99f0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:07.606568 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-12a83696-afe5-4c00-81a9-2db91fab99f0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:07.607009 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-12a83696-afe5-4c00-81a9-2db91fab99f0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:07.607427 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-12a83696-afe5-4c00-81a9-2db91fab99f0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:07.792294 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9083dcd0-bbb8-4b27-af7b-c8fab8d94aef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:07.795614 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-12a83696-afe5-4c00-81a9-2db91fab99f0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:07.801104 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9083dcd0-bbb8-4b27-af7b-c8fab8d94aef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.213801 Aug 28 01:18:07.801787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1001/1776] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:07 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:07.803968 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-12a83696-afe5-4c00-81a9-2db91fab99f0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.210277 Aug 28 01:18:07.804530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 776/1777] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:07 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:07.806777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-48049d2a-4acd-431c-9be9-6ad38ee7a30b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:07.807541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-48049d2a-4acd-431c-9be9-6ad38ee7a30b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:07.813847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48049d2a-4acd-431c-9be9-6ad38ee7a30b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:07.814464 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48049d2a-4acd-431c-9be9-6ad38ee7a30b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:07.815001 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48049d2a-4acd-431c-9be9-6ad38ee7a30b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:07.815700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48049d2a-4acd-431c-9be9-6ad38ee7a30b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:07.816152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48049d2a-4acd-431c-9be9-6ad38ee7a30b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:07.816574 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48049d2a-4acd-431c-9be9-6ad38ee7a30b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:07.966075 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-48049d2a-4acd-431c-9be9-6ad38ee7a30b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:07.978998 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-48049d2a-4acd-431c-9be9-6ad38ee7a30b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:07.980836 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-48049d2a-4acd-431c-9be9-6ad38ee7a30b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:07.982019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-48049d2a-4acd-431c-9be9-6ad38ee7a30b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.179208 Aug 28 01:18:07.982576 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1002/1778] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:07 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:08.824176 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1322cdd7-dab7-4755-baa4-371155c19c5d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:08.825118 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-22d2b422-d91d-49f7-8f40-fb73e9a2a6c7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:08.825742 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1322cdd7-dab7-4755-baa4-371155c19c5d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:08.825916 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-22d2b422-d91d-49f7-8f40-fb73e9a2a6c7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:08.832374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-22d2b422-d91d-49f7-8f40-fb73e9a2a6c7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:08.832722 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1322cdd7-dab7-4755-baa4-371155c19c5d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:08.832904 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-22d2b422-d91d-49f7-8f40-fb73e9a2a6c7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:08.833246 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1322cdd7-dab7-4755-baa4-371155c19c5d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:08.833424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-22d2b422-d91d-49f7-8f40-fb73e9a2a6c7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:08.833638 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1322cdd7-dab7-4755-baa4-371155c19c5d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:08.833974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-22d2b422-d91d-49f7-8f40-fb73e9a2a6c7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:08.834344 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1322cdd7-dab7-4755-baa4-371155c19c5d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:08.834492 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-22d2b422-d91d-49f7-8f40-fb73e9a2a6c7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:08.834782 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1322cdd7-dab7-4755-baa4-371155c19c5d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:08.834941 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-22d2b422-d91d-49f7-8f40-fb73e9a2a6c7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:08.835200 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1322cdd7-dab7-4755-baa4-371155c19c5d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:09.031296 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1322cdd7-dab7-4755-baa4-371155c19c5d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:09.035215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-22d2b422-d91d-49f7-8f40-fb73e9a2a6c7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:09.040057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1322cdd7-dab7-4755-baa4-371155c19c5d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.220401 Aug 28 01:18:09.040643 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 777/1779] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:08 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:09.043951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-22d2b422-d91d-49f7-8f40-fb73e9a2a6c7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.222801 Aug 28 01:18:09.044567 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1003/1780] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:08 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:09.046276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8c84dd66-b773-431f-92a8-a3e2208bdae1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:09.046854 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8c84dd66-b773-431f-92a8-a3e2208bdae1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:09.053248 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8c84dd66-b773-431f-92a8-a3e2208bdae1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:09.053732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8c84dd66-b773-431f-92a8-a3e2208bdae1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:09.054154 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8c84dd66-b773-431f-92a8-a3e2208bdae1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:09.054875 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8c84dd66-b773-431f-92a8-a3e2208bdae1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:09.055315 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8c84dd66-b773-431f-92a8-a3e2208bdae1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:09.055744 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8c84dd66-b773-431f-92a8-a3e2208bdae1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:09.070963 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:18:09.191330 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8c84dd66-b773-431f-92a8-a3e2208bdae1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:09.200586 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8c84dd66-b773-431f-92a8-a3e2208bdae1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:09.202035 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8c84dd66-b773-431f-92a8-a3e2208bdae1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:09.203190 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8c84dd66-b773-431f-92a8-a3e2208bdae1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.160796 Aug 28 01:18:09.203737 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 778/1781] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:09 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:10.064156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ac34edfc-3403-4508-ae39-d510152f9612 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:10.064801 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-96058430-722d-4c7d-82ab-a23e929f29a7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:10.065234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ac34edfc-3403-4508-ae39-d510152f9612 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:10.065361 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-96058430-722d-4c7d-82ab-a23e929f29a7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:10.072209 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96058430-722d-4c7d-82ab-a23e929f29a7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:10.072691 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96058430-722d-4c7d-82ab-a23e929f29a7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:10.073045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac34edfc-3403-4508-ae39-d510152f9612 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:10.073181 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96058430-722d-4c7d-82ab-a23e929f29a7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:10.073539 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac34edfc-3403-4508-ae39-d510152f9612 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:10.073921 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96058430-722d-4c7d-82ab-a23e929f29a7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:10.074109 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac34edfc-3403-4508-ae39-d510152f9612 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:10.074308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96058430-722d-4c7d-82ab-a23e929f29a7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:10.074725 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96058430-722d-4c7d-82ab-a23e929f29a7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:10.074830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac34edfc-3403-4508-ae39-d510152f9612 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:10.075260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac34edfc-3403-4508-ae39-d510152f9612 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:10.075714 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac34edfc-3403-4508-ae39-d510152f9612 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:10.575814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ac34edfc-3403-4508-ae39-d510152f9612 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:10.583547 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-96058430-722d-4c7d-82ab-a23e929f29a7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:10.584349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ac34edfc-3403-4508-ae39-d510152f9612 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.526518 Aug 28 01:18:10.585320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1004/1782] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:10 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:10.590073 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6b27ed44-3f9d-44c9-ae56-60204e6e17c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:10.590608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6b27ed44-3f9d-44c9-ae56-60204e6e17c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:10.592831 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-96058430-722d-4c7d-82ab-a23e929f29a7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.532196 Aug 28 01:18:10.593637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 779/1783] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:10 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 533 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:10.596581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b27ed44-3f9d-44c9-ae56-60204e6e17c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:10.597029 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b27ed44-3f9d-44c9-ae56-60204e6e17c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:10.597460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b27ed44-3f9d-44c9-ae56-60204e6e17c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:10.598178 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b27ed44-3f9d-44c9-ae56-60204e6e17c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:10.598618 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b27ed44-3f9d-44c9-ae56-60204e6e17c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:10.599102 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b27ed44-3f9d-44c9-ae56-60204e6e17c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:10.830419 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6b27ed44-3f9d-44c9-ae56-60204e6e17c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:10.839717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6b27ed44-3f9d-44c9-ae56-60204e6e17c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:10.841172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6b27ed44-3f9d-44c9-ae56-60204e6e17c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:10.842354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6b27ed44-3f9d-44c9-ae56-60204e6e17c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.256006 Aug 28 01:18:10.842981 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1005/1784] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:10 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:11.607276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4c8051ef-ed42-4768-a3dc-ba2c6e4f82a5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:11.607881 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4c8051ef-ed42-4768-a3dc-ba2c6e4f82a5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:11.614271 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7213b581-d91a-42a6-87e8-3423ec22068f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:11.614790 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c8051ef-ed42-4768-a3dc-ba2c6e4f82a5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:11.614903 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7213b581-d91a-42a6-87e8-3423ec22068f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:11.615166 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c8051ef-ed42-4768-a3dc-ba2c6e4f82a5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:11.615597 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c8051ef-ed42-4768-a3dc-ba2c6e4f82a5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:11.616292 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c8051ef-ed42-4768-a3dc-ba2c6e4f82a5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:11.616724 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c8051ef-ed42-4768-a3dc-ba2c6e4f82a5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:11.617186 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c8051ef-ed42-4768-a3dc-ba2c6e4f82a5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:11.621931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7213b581-d91a-42a6-87e8-3423ec22068f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:11.622416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7213b581-d91a-42a6-87e8-3423ec22068f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:11.622833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7213b581-d91a-42a6-87e8-3423ec22068f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:11.623526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7213b581-d91a-42a6-87e8-3423ec22068f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:11.623974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7213b581-d91a-42a6-87e8-3423ec22068f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:11.624396 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7213b581-d91a-42a6-87e8-3423ec22068f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:11.627025 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:18:11.814776 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4c8051ef-ed42-4768-a3dc-ba2c6e4f82a5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:11.817276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7213b581-d91a-42a6-87e8-3423ec22068f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:11.824074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4c8051ef-ed42-4768-a3dc-ba2c6e4f82a5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.221690 Aug 28 01:18:11.824662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 780/1785] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:11 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:11.825822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7213b581-d91a-42a6-87e8-3423ec22068f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.215379 Aug 28 01:18:11.826387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1006/1786] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:11 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:11.867776 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fd598c80-8246-496e-82ec-a2c8e49a272a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:11.868374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fd598c80-8246-496e-82ec-a2c8e49a272a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:11.875219 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd598c80-8246-496e-82ec-a2c8e49a272a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:11.875703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd598c80-8246-496e-82ec-a2c8e49a272a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:11.876138 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd598c80-8246-496e-82ec-a2c8e49a272a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:11.876869 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd598c80-8246-496e-82ec-a2c8e49a272a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:11.877346 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd598c80-8246-496e-82ec-a2c8e49a272a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:11.877769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd598c80-8246-496e-82ec-a2c8e49a272a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:12.012210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fd598c80-8246-496e-82ec-a2c8e49a272a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:12.021976 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fd598c80-8246-496e-82ec-a2c8e49a272a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:12.024164 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fd598c80-8246-496e-82ec-a2c8e49a272a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:12.025889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fd598c80-8246-496e-82ec-a2c8e49a272a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.161930 Aug 28 01:18:12.026657 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 781/1787] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:11 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:12.851122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-909f4ad7-c5f7-4f3d-b411-36dff79b1b6a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:12.851697 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1c58a29d-21b3-418d-9abf-72daab4434c9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:12.852084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-909f4ad7-c5f7-4f3d-b411-36dff79b1b6a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:12.852194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1c58a29d-21b3-418d-9abf-72daab4434c9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:12.858803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-909f4ad7-c5f7-4f3d-b411-36dff79b1b6a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:12.859281 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-909f4ad7-c5f7-4f3d-b411-36dff79b1b6a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:12.859690 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-909f4ad7-c5f7-4f3d-b411-36dff79b1b6a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:12.860383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-909f4ad7-c5f7-4f3d-b411-36dff79b1b6a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:12.860501 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c58a29d-21b3-418d-9abf-72daab4434c9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:12.860826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-909f4ad7-c5f7-4f3d-b411-36dff79b1b6a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:12.861040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c58a29d-21b3-418d-9abf-72daab4434c9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:12.861275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-909f4ad7-c5f7-4f3d-b411-36dff79b1b6a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:12.861403 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c58a29d-21b3-418d-9abf-72daab4434c9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:12.862101 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c58a29d-21b3-418d-9abf-72daab4434c9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:12.862530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c58a29d-21b3-418d-9abf-72daab4434c9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:12.862945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c58a29d-21b3-418d-9abf-72daab4434c9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:13.059435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-909f4ad7-c5f7-4f3d-b411-36dff79b1b6a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:13.063865 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1c58a29d-21b3-418d-9abf-72daab4434c9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:13.068662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-909f4ad7-c5f7-4f3d-b411-36dff79b1b6a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.221141 Aug 28 01:18:13.069282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1007/1788] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:12 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:13.073977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-802afee7-5189-4269-9158-94dbfc2f0b17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:13.074574 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-802afee7-5189-4269-9158-94dbfc2f0b17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:13.077483 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1c58a29d-21b3-418d-9abf-72daab4434c9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.229126 Aug 28 01:18:13.079465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 782/1789] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:12 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:13.081100 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-802afee7-5189-4269-9158-94dbfc2f0b17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:13.081850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-802afee7-5189-4269-9158-94dbfc2f0b17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:13.082455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-802afee7-5189-4269-9158-94dbfc2f0b17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:13.083817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-802afee7-5189-4269-9158-94dbfc2f0b17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:13.084272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-802afee7-5189-4269-9158-94dbfc2f0b17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:13.084692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-802afee7-5189-4269-9158-94dbfc2f0b17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:13.226353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-802afee7-5189-4269-9158-94dbfc2f0b17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:13.235445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-802afee7-5189-4269-9158-94dbfc2f0b17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:13.236877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-802afee7-5189-4269-9158-94dbfc2f0b17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:13.238066 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-802afee7-5189-4269-9158-94dbfc2f0b17 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.167869 Aug 28 01:18:13.238619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1008/1790] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:13 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:14.095371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-91d166e8-70e5-4bf1-bf71-4ccadf793b72 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:14.096004 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-91d166e8-70e5-4bf1-bf71-4ccadf793b72 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:14.103990 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-91d166e8-70e5-4bf1-bf71-4ccadf793b72 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:14.104478 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-91d166e8-70e5-4bf1-bf71-4ccadf793b72 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:14.104663 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d913dda2-a97c-4a35-87e2-0fff6efb768d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:14.105190 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-91d166e8-70e5-4bf1-bf71-4ccadf793b72 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:14.105529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d913dda2-a97c-4a35-87e2-0fff6efb768d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:14.105711 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-91d166e8-70e5-4bf1-bf71-4ccadf793b72 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:14.106047 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-91d166e8-70e5-4bf1-bf71-4ccadf793b72 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:14.106482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-91d166e8-70e5-4bf1-bf71-4ccadf793b72 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:14.114975 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d913dda2-a97c-4a35-87e2-0fff6efb768d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:14.114975 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d913dda2-a97c-4a35-87e2-0fff6efb768d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:14.114975 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d913dda2-a97c-4a35-87e2-0fff6efb768d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:14.115373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d913dda2-a97c-4a35-87e2-0fff6efb768d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:14.115610 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d913dda2-a97c-4a35-87e2-0fff6efb768d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:14.116014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d913dda2-a97c-4a35-87e2-0fff6efb768d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:14.311028 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-91d166e8-70e5-4bf1-bf71-4ccadf793b72 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:14.314137 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d913dda2-a97c-4a35-87e2-0fff6efb768d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:14.320272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-91d166e8-70e5-4bf1-bf71-4ccadf793b72 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.228767 Aug 28 01:18:14.320824 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 783/1791] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:14 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:14.323195 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d913dda2-a97c-4a35-87e2-0fff6efb768d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.226136 Aug 28 01:18:14.323927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1009/1792] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:14 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:14.325507 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-59cf5e8d-ba2c-4724-a16b-d18ff44b1a0c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:14.326304 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-59cf5e8d-ba2c-4724-a16b-d18ff44b1a0c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:14.332777 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-59cf5e8d-ba2c-4724-a16b-d18ff44b1a0c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:14.333296 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-59cf5e8d-ba2c-4724-a16b-d18ff44b1a0c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:14.333930 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-59cf5e8d-ba2c-4724-a16b-d18ff44b1a0c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:14.334709 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-59cf5e8d-ba2c-4724-a16b-d18ff44b1a0c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:14.335272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-59cf5e8d-ba2c-4724-a16b-d18ff44b1a0c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:14.335687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-59cf5e8d-ba2c-4724-a16b-d18ff44b1a0c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:14.472620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-59cf5e8d-ba2c-4724-a16b-d18ff44b1a0c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:14.482388 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-59cf5e8d-ba2c-4724-a16b-d18ff44b1a0c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:14.483843 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-59cf5e8d-ba2c-4724-a16b-d18ff44b1a0c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:14.485150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-59cf5e8d-ba2c-4724-a16b-d18ff44b1a0c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.163326 Aug 28 01:18:14.485720 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 784/1793] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:14 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:15.342063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-855260f1-d0f8-42a4-a533-587c17c4bfa1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:15.342663 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-855260f1-d0f8-42a4-a533-587c17c4bfa1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:15.343604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-217f8c12-74c6-4b32-a6d1-d7ac30e13ef9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:15.344218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-217f8c12-74c6-4b32-a6d1-d7ac30e13ef9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:15.349013 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-855260f1-d0f8-42a4-a533-587c17c4bfa1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:15.349488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-855260f1-d0f8-42a4-a533-587c17c4bfa1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:15.349914 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-855260f1-d0f8-42a4-a533-587c17c4bfa1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:15.350599 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-855260f1-d0f8-42a4-a533-587c17c4bfa1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:15.350840 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-217f8c12-74c6-4b32-a6d1-d7ac30e13ef9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:15.350998 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-855260f1-d0f8-42a4-a533-587c17c4bfa1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:15.351331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-217f8c12-74c6-4b32-a6d1-d7ac30e13ef9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:15.351482 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-855260f1-d0f8-42a4-a533-587c17c4bfa1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:15.351764 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-217f8c12-74c6-4b32-a6d1-d7ac30e13ef9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:15.352453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-217f8c12-74c6-4b32-a6d1-d7ac30e13ef9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:15.352899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-217f8c12-74c6-4b32-a6d1-d7ac30e13ef9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:15.353340 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-217f8c12-74c6-4b32-a6d1-d7ac30e13ef9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:15.542282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-855260f1-d0f8-42a4-a533-587c17c4bfa1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:15.550770 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-855260f1-d0f8-42a4-a533-587c17c4bfa1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.212865 Aug 28 01:18:15.551353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1010/1794] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:15 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:15.555525 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-217f8c12-74c6-4b32-a6d1-d7ac30e13ef9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:15.555968 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-89be8e70-bf9a-49c7-be7a-0903872be051 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:15.556573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-89be8e70-bf9a-49c7-be7a-0903872be051 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:15.562698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-89be8e70-bf9a-49c7-be7a-0903872be051 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:15.563187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-89be8e70-bf9a-49c7-be7a-0903872be051 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:15.563602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-89be8e70-bf9a-49c7-be7a-0903872be051 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:15.564292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-89be8e70-bf9a-49c7-be7a-0903872be051 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:15.564733 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-89be8e70-bf9a-49c7-be7a-0903872be051 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:15.565169 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-89be8e70-bf9a-49c7-be7a-0903872be051 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:15.565291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-217f8c12-74c6-4b32-a6d1-d7ac30e13ef9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.225312 Aug 28 01:18:15.565790 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 785/1795] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:15 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:15.704694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-89be8e70-bf9a-49c7-be7a-0903872be051 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:15.714769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-89be8e70-bf9a-49c7-be7a-0903872be051 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:15.716360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-89be8e70-bf9a-49c7-be7a-0903872be051 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:15.721453 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-89be8e70-bf9a-49c7-be7a-0903872be051 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.165284 Aug 28 01:18:15.721453 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1011/1796] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:15 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:16.574380 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3c2c5226-e185-4a70-8bfa-6a381fc7c5a6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:16.575139 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3c2c5226-e185-4a70-8bfa-6a381fc7c5a6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:16.583917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c2c5226-e185-4a70-8bfa-6a381fc7c5a6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:16.584452 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c2c5226-e185-4a70-8bfa-6a381fc7c5a6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:16.585067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c2c5226-e185-4a70-8bfa-6a381fc7c5a6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:16.585775 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c2c5226-e185-4a70-8bfa-6a381fc7c5a6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:16.586426 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c2c5226-e185-4a70-8bfa-6a381fc7c5a6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:16.586938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c2c5226-e185-4a70-8bfa-6a381fc7c5a6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:16.589046 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f6693b0d-edb3-4347-96aa-14c46fa697a3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:16.589772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f6693b0d-edb3-4347-96aa-14c46fa697a3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:16.596818 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6693b0d-edb3-4347-96aa-14c46fa697a3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:16.597914 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6693b0d-edb3-4347-96aa-14c46fa697a3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:16.597914 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6693b0d-edb3-4347-96aa-14c46fa697a3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:16.598452 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6693b0d-edb3-4347-96aa-14c46fa697a3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:16.598910 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6693b0d-edb3-4347-96aa-14c46fa697a3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:16.599329 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6693b0d-edb3-4347-96aa-14c46fa697a3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:16.790315 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f6693b0d-edb3-4347-96aa-14c46fa697a3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:16.794002 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3c2c5226-e185-4a70-8bfa-6a381fc7c5a6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:16.799146 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f6693b0d-edb3-4347-96aa-14c46fa697a3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.215362 Aug 28 01:18:16.801143 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1012/1797] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:16 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:16.804294 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3c2c5226-e185-4a70-8bfa-6a381fc7c5a6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.235956 Aug 28 01:18:16.805425 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e093e616-da9f-4295-bf5d-4dc01c886a89 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:16.809062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e093e616-da9f-4295-bf5d-4dc01c886a89 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:16.809671 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 786/1798] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:16 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:16.821409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e093e616-da9f-4295-bf5d-4dc01c886a89 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:16.821746 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e093e616-da9f-4295-bf5d-4dc01c886a89 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:16.822063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e093e616-da9f-4295-bf5d-4dc01c886a89 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:16.822844 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e093e616-da9f-4295-bf5d-4dc01c886a89 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:16.823212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e093e616-da9f-4295-bf5d-4dc01c886a89 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:16.823624 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e093e616-da9f-4295-bf5d-4dc01c886a89 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:16.959817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e093e616-da9f-4295-bf5d-4dc01c886a89 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:16.971975 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e093e616-da9f-4295-bf5d-4dc01c886a89 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:16.973459 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e093e616-da9f-4295-bf5d-4dc01c886a89 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:16.974573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e093e616-da9f-4295-bf5d-4dc01c886a89 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.173929 Aug 28 01:18:16.975129 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1013/1799] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:16 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:17.823482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a07ceead-70fd-400e-8f31-55d50cde2f73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:17.824067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a07ceead-70fd-400e-8f31-55d50cde2f73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:17.828115 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-246490f6-c950-46ca-b622-eb4968078f42 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:17.828709 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-246490f6-c950-46ca-b622-eb4968078f42 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:17.831113 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a07ceead-70fd-400e-8f31-55d50cde2f73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:17.831611 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a07ceead-70fd-400e-8f31-55d50cde2f73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:17.832065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a07ceead-70fd-400e-8f31-55d50cde2f73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:17.832760 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a07ceead-70fd-400e-8f31-55d50cde2f73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:17.833263 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a07ceead-70fd-400e-8f31-55d50cde2f73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:17.833648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a07ceead-70fd-400e-8f31-55d50cde2f73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:17.835126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-246490f6-c950-46ca-b622-eb4968078f42 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:17.835608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-246490f6-c950-46ca-b622-eb4968078f42 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:17.836037 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-246490f6-c950-46ca-b622-eb4968078f42 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:17.836733 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-246490f6-c950-46ca-b622-eb4968078f42 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:17.837218 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-246490f6-c950-46ca-b622-eb4968078f42 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:17.837618 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-246490f6-c950-46ca-b622-eb4968078f42 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:18.040450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a07ceead-70fd-400e-8f31-55d50cde2f73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:18.045714 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-246490f6-c950-46ca-b622-eb4968078f42 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:18.052904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a07ceead-70fd-400e-8f31-55d50cde2f73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.234684 Aug 28 01:18:18.065062 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 787/1800] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:17 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:18.065280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-246490f6-c950-46ca-b622-eb4968078f42 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.231992 Aug 28 01:18:18.065280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1014/1801] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:17 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:18.065487 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-670155f5-1d2d-49a0-8bd1-3f19b016550e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:18.065487 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-670155f5-1d2d-49a0-8bd1-3f19b016550e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:18.065487 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-670155f5-1d2d-49a0-8bd1-3f19b016550e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:18.065487 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-670155f5-1d2d-49a0-8bd1-3f19b016550e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:18.065897 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-670155f5-1d2d-49a0-8bd1-3f19b016550e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:18.066735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-670155f5-1d2d-49a0-8bd1-3f19b016550e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:18.067155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-670155f5-1d2d-49a0-8bd1-3f19b016550e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:18.067633 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-670155f5-1d2d-49a0-8bd1-3f19b016550e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:18.207107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-670155f5-1d2d-49a0-8bd1-3f19b016550e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:18.218948 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-670155f5-1d2d-49a0-8bd1-3f19b016550e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:18.221208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-670155f5-1d2d-49a0-8bd1-3f19b016550e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:18.222816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-670155f5-1d2d-49a0-8bd1-3f19b016550e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.168080 Aug 28 01:18:18.223696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 788/1802] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:18 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:19.077189 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-80dcb3a5-cf69-4773-b98b-d9860849004d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:19.078316 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-80dcb3a5-cf69-4773-b98b-d9860849004d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:19.078579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b9d21926-0e06-47f0-b85e-95a21db168f2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:19.078579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b9d21926-0e06-47f0-b85e-95a21db168f2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:19.084213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-80dcb3a5-cf69-4773-b98b-d9860849004d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:19.084702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-80dcb3a5-cf69-4773-b98b-d9860849004d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:19.085170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-80dcb3a5-cf69-4773-b98b-d9860849004d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:19.085886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b9d21926-0e06-47f0-b85e-95a21db168f2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:19.086065 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-80dcb3a5-cf69-4773-b98b-d9860849004d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:19.086331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-80dcb3a5-cf69-4773-b98b-d9860849004d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:19.086500 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b9d21926-0e06-47f0-b85e-95a21db168f2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:19.086751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-80dcb3a5-cf69-4773-b98b-d9860849004d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:19.086908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b9d21926-0e06-47f0-b85e-95a21db168f2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:19.087453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b9d21926-0e06-47f0-b85e-95a21db168f2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:19.087892 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b9d21926-0e06-47f0-b85e-95a21db168f2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:19.088305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b9d21926-0e06-47f0-b85e-95a21db168f2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:19.293635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b9d21926-0e06-47f0-b85e-95a21db168f2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:19.299016 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-80dcb3a5-cf69-4773-b98b-d9860849004d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:19.305152 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b9d21926-0e06-47f0-b85e-95a21db168f2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.230701 Aug 28 01:18:19.306158 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 789/1803] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:19 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:18:19.309724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-80dcb3a5-cf69-4773-b98b-d9860849004d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.236867 Aug 28 01:18:19.310451 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1015/1804] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:19 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:19.310594 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-afc1aab9-e75b-40e8-b070-f7f3ff7c1a5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:19.311020 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-afc1aab9-e75b-40e8-b070-f7f3ff7c1a5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:19.316850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-afc1aab9-e75b-40e8-b070-f7f3ff7c1a5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:19.317341 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-afc1aab9-e75b-40e8-b070-f7f3ff7c1a5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:19.317774 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-afc1aab9-e75b-40e8-b070-f7f3ff7c1a5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:19.318802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-afc1aab9-e75b-40e8-b070-f7f3ff7c1a5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:19.319241 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-afc1aab9-e75b-40e8-b070-f7f3ff7c1a5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:19.319640 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-afc1aab9-e75b-40e8-b070-f7f3ff7c1a5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:19.461913 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-afc1aab9-e75b-40e8-b070-f7f3ff7c1a5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:19.472377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-afc1aab9-e75b-40e8-b070-f7f3ff7c1a5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:19.473855 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-afc1aab9-e75b-40e8-b070-f7f3ff7c1a5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:19.475031 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-afc1aab9-e75b-40e8-b070-f7f3ff7c1a5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.168164 Aug 28 01:18:19.475589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 790/1805] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:19 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:20.328067 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-eecc0b45-e5cc-4476-bee5-f119a201a697 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:20.328753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-eecc0b45-e5cc-4476-bee5-f119a201a697 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:20.332212 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3ef28b04-eee3-44f2-b03b-5f814fda65c5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:20.333305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3ef28b04-eee3-44f2-b03b-5f814fda65c5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:20.336213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eecc0b45-e5cc-4476-bee5-f119a201a697 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:20.336707 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eecc0b45-e5cc-4476-bee5-f119a201a697 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:20.337164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eecc0b45-e5cc-4476-bee5-f119a201a697 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:20.338216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eecc0b45-e5cc-4476-bee5-f119a201a697 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:20.338346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eecc0b45-e5cc-4476-bee5-f119a201a697 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:20.338730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eecc0b45-e5cc-4476-bee5-f119a201a697 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:20.341810 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ef28b04-eee3-44f2-b03b-5f814fda65c5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:20.342325 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ef28b04-eee3-44f2-b03b-5f814fda65c5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:20.342749 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ef28b04-eee3-44f2-b03b-5f814fda65c5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:20.343479 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ef28b04-eee3-44f2-b03b-5f814fda65c5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:20.343988 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ef28b04-eee3-44f2-b03b-5f814fda65c5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:20.344484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ef28b04-eee3-44f2-b03b-5f814fda65c5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:20.543152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-eecc0b45-e5cc-4476-bee5-f119a201a697 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:20.546553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3ef28b04-eee3-44f2-b03b-5f814fda65c5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:20.551705 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-eecc0b45-e5cc-4476-bee5-f119a201a697 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.227698 Aug 28 01:18:20.552346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1016/1806] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:20 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:20.555070 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3ef28b04-eee3-44f2-b03b-5f814fda65c5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.227742 Aug 28 01:18:20.555762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 791/1807] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:20 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:20.557129 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1bf829ce-63f6-459f-87e5-fffd92e1e906 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:20.557921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1bf829ce-63f6-459f-87e5-fffd92e1e906 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:20.564261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1bf829ce-63f6-459f-87e5-fffd92e1e906 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:20.564766 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1bf829ce-63f6-459f-87e5-fffd92e1e906 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:20.565225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1bf829ce-63f6-459f-87e5-fffd92e1e906 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:20.565953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1bf829ce-63f6-459f-87e5-fffd92e1e906 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:20.566373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1bf829ce-63f6-459f-87e5-fffd92e1e906 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:20.566818 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1bf829ce-63f6-459f-87e5-fffd92e1e906 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:20.705493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1bf829ce-63f6-459f-87e5-fffd92e1e906 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:20.718299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1bf829ce-63f6-459f-87e5-fffd92e1e906 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:20.720611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1bf829ce-63f6-459f-87e5-fffd92e1e906 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:20.722551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1bf829ce-63f6-459f-87e5-fffd92e1e906 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.168876 Aug 28 01:18:20.723395 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1017/1808] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:20 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:21.574018 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-08f54e25-1ed3-42b0-a1f4-04ac214444f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:21.574588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-08f54e25-1ed3-42b0-a1f4-04ac214444f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:21.577020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1cdac82d-4bc4-4cd2-9555-d0a0e7085e53 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:21.577633 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1cdac82d-4bc4-4cd2-9555-d0a0e7085e53 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:21.581664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08f54e25-1ed3-42b0-a1f4-04ac214444f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:21.582203 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08f54e25-1ed3-42b0-a1f4-04ac214444f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:21.582625 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08f54e25-1ed3-42b0-a1f4-04ac214444f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:21.583348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08f54e25-1ed3-42b0-a1f4-04ac214444f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:21.583791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08f54e25-1ed3-42b0-a1f4-04ac214444f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:21.584213 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08f54e25-1ed3-42b0-a1f4-04ac214444f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:21.584359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1cdac82d-4bc4-4cd2-9555-d0a0e7085e53 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:21.584797 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1cdac82d-4bc4-4cd2-9555-d0a0e7085e53 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:21.585259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1cdac82d-4bc4-4cd2-9555-d0a0e7085e53 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:21.585944 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1cdac82d-4bc4-4cd2-9555-d0a0e7085e53 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:21.586387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1cdac82d-4bc4-4cd2-9555-d0a0e7085e53 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:21.586792 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1cdac82d-4bc4-4cd2-9555-d0a0e7085e53 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:21.803197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-08f54e25-1ed3-42b0-a1f4-04ac214444f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:21.812072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1cdac82d-4bc4-4cd2-9555-d0a0e7085e53 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:21.814091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-08f54e25-1ed3-42b0-a1f4-04ac214444f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.244052 Aug 28 01:18:21.814651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 792/1809] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:21 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:21.819723 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-766c81d9-648e-46f0-adc2-68a128133c88 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:21.820320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-766c81d9-648e-46f0-adc2-68a128133c88 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:21.821544 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1cdac82d-4bc4-4cd2-9555-d0a0e7085e53 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.248437 Aug 28 01:18:21.822115 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1018/1810] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:21 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:21.826599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-766c81d9-648e-46f0-adc2-68a128133c88 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:21.827276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-766c81d9-648e-46f0-adc2-68a128133c88 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:21.827848 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-766c81d9-648e-46f0-adc2-68a128133c88 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:21.828614 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-766c81d9-648e-46f0-adc2-68a128133c88 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:21.829194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-766c81d9-648e-46f0-adc2-68a128133c88 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:21.829617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-766c81d9-648e-46f0-adc2-68a128133c88 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:21.979527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-766c81d9-648e-46f0-adc2-68a128133c88 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:21.986768 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-766c81d9-648e-46f0-adc2-68a128133c88 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:21.988193 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-766c81d9-648e-46f0-adc2-68a128133c88 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:21.989422 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-766c81d9-648e-46f0-adc2-68a128133c88 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.173801 Aug 28 01:18:21.990015 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 793/1811] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:21 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:22.836066 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4ce8e9a1-8c51-481f-90ff-3d5865eac7bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:22.836872 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4ce8e9a1-8c51-481f-90ff-3d5865eac7bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:22.843755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ce8e9a1-8c51-481f-90ff-3d5865eac7bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:22.844268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ce8e9a1-8c51-481f-90ff-3d5865eac7bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:22.844651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-63e69be9-ea6a-4453-96c0-fec17fd9f631 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:22.845372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ce8e9a1-8c51-481f-90ff-3d5865eac7bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:22.845372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ce8e9a1-8c51-481f-90ff-3d5865eac7bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:22.845725 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-63e69be9-ea6a-4453-96c0-fec17fd9f631 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:22.845977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ce8e9a1-8c51-481f-90ff-3d5865eac7bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:22.846245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4ce8e9a1-8c51-481f-90ff-3d5865eac7bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:22.854180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-63e69be9-ea6a-4453-96c0-fec17fd9f631 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:22.854766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-63e69be9-ea6a-4453-96c0-fec17fd9f631 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:22.855287 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-63e69be9-ea6a-4453-96c0-fec17fd9f631 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:22.856070 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-63e69be9-ea6a-4453-96c0-fec17fd9f631 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:22.856606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-63e69be9-ea6a-4453-96c0-fec17fd9f631 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:22.857156 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-63e69be9-ea6a-4453-96c0-fec17fd9f631 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:23.039797 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4ce8e9a1-8c51-481f-90ff-3d5865eac7bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:23.043746 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-63e69be9-ea6a-4453-96c0-fec17fd9f631 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:23.048667 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4ce8e9a1-8c51-481f-90ff-3d5865eac7bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.216533 Aug 28 01:18:23.049405 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1019/1812] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:22 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:23.052076 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-63e69be9-ea6a-4453-96c0-fec17fd9f631 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.213134 Aug 28 01:18:23.052908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 794/1813] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:22 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:18:23.054351 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8b4355d6-7f09-4cd3-a41e-eb922d04518d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:23.055026 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8b4355d6-7f09-4cd3-a41e-eb922d04518d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:23.061357 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b4355d6-7f09-4cd3-a41e-eb922d04518d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:23.061926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b4355d6-7f09-4cd3-a41e-eb922d04518d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:23.062431 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b4355d6-7f09-4cd3-a41e-eb922d04518d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:23.063193 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b4355d6-7f09-4cd3-a41e-eb922d04518d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:23.063702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b4355d6-7f09-4cd3-a41e-eb922d04518d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:23.064207 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b4355d6-7f09-4cd3-a41e-eb922d04518d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:23.201245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8b4355d6-7f09-4cd3-a41e-eb922d04518d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:23.210267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8b4355d6-7f09-4cd3-a41e-eb922d04518d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:23.211688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8b4355d6-7f09-4cd3-a41e-eb922d04518d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:23.212843 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8b4355d6-7f09-4cd3-a41e-eb922d04518d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.162427 Aug 28 01:18:23.213424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1020/1814] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:23 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:24.075006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3a47aac0-a21c-430a-bd3f-e615698c95fd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:24.075798 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9c68e34b-52bc-49f7-b013-5f6a5e8037fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:24.076302 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3a47aac0-a21c-430a-bd3f-e615698c95fd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:24.076442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9c68e34b-52bc-49f7-b013-5f6a5e8037fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:24.083900 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a47aac0-a21c-430a-bd3f-e615698c95fd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:24.084072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c68e34b-52bc-49f7-b013-5f6a5e8037fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:24.084386 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a47aac0-a21c-430a-bd3f-e615698c95fd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:24.084559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c68e34b-52bc-49f7-b013-5f6a5e8037fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:24.084795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a47aac0-a21c-430a-bd3f-e615698c95fd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:24.084912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c68e34b-52bc-49f7-b013-5f6a5e8037fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:24.085543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a47aac0-a21c-430a-bd3f-e615698c95fd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:24.085646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c68e34b-52bc-49f7-b013-5f6a5e8037fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:24.085994 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a47aac0-a21c-430a-bd3f-e615698c95fd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:24.086107 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c68e34b-52bc-49f7-b013-5f6a5e8037fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:24.086413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a47aac0-a21c-430a-bd3f-e615698c95fd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:24.086524 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c68e34b-52bc-49f7-b013-5f6a5e8037fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:24.089085 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:18:24.288449 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9c68e34b-52bc-49f7-b013-5f6a5e8037fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:24.291851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3a47aac0-a21c-430a-bd3f-e615698c95fd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:24.296997 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9c68e34b-52bc-49f7-b013-5f6a5e8037fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.226598 Aug 28 01:18:24.297811 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1021/1815] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:24 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:24.301417 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3a47aac0-a21c-430a-bd3f-e615698c95fd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.231568 Aug 28 01:18:24.302244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 795/1816] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:24 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:24.302362 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9c222b01-f802-4621-9280-55094c6a9b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:24.302826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9c222b01-f802-4621-9280-55094c6a9b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:24.308692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c222b01-f802-4621-9280-55094c6a9b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:24.309212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c222b01-f802-4621-9280-55094c6a9b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:24.309626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c222b01-f802-4621-9280-55094c6a9b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:24.310330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c222b01-f802-4621-9280-55094c6a9b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:24.310780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c222b01-f802-4621-9280-55094c6a9b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:24.311196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c222b01-f802-4621-9280-55094c6a9b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:24.468485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9c222b01-f802-4621-9280-55094c6a9b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:24.471622 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:18:24.480047 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9c222b01-f802-4621-9280-55094c6a9b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:24.481514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9c222b01-f802-4621-9280-55094c6a9b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:24.482754 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9c222b01-f802-4621-9280-55094c6a9b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.184246 Aug 28 01:18:24.483313 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1022/1817] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:24 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:25.317726 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-81b38c80-7519-478d-9f83-0907b9311a76 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:25.318660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-81b38c80-7519-478d-9f83-0907b9311a76 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:25.322730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8d60ee53-a61c-4fc3-91c3-03ada60c18fd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:25.323384 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8d60ee53-a61c-4fc3-91c3-03ada60c18fd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:25.325403 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-81b38c80-7519-478d-9f83-0907b9311a76 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:25.325898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-81b38c80-7519-478d-9f83-0907b9311a76 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:25.326585 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-81b38c80-7519-478d-9f83-0907b9311a76 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:25.327032 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-81b38c80-7519-478d-9f83-0907b9311a76 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:25.327479 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-81b38c80-7519-478d-9f83-0907b9311a76 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:25.327906 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-81b38c80-7519-478d-9f83-0907b9311a76 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:25.330132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d60ee53-a61c-4fc3-91c3-03ada60c18fd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:25.330783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d60ee53-a61c-4fc3-91c3-03ada60c18fd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:25.331222 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d60ee53-a61c-4fc3-91c3-03ada60c18fd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:25.332087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d60ee53-a61c-4fc3-91c3-03ada60c18fd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:25.332357 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d60ee53-a61c-4fc3-91c3-03ada60c18fd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:25.332772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d60ee53-a61c-4fc3-91c3-03ada60c18fd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:25.519751 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-81b38c80-7519-478d-9f83-0907b9311a76 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:25.526779 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8d60ee53-a61c-4fc3-91c3-03ada60c18fd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:25.528356 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-81b38c80-7519-478d-9f83-0907b9311a76 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.214586 Aug 28 01:18:25.528936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 796/1818] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:25 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:25.533548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-26dbefc3-ba76-4ebb-aa8a-bc127345c730 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:25.534144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-26dbefc3-ba76-4ebb-aa8a-bc127345c730 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:25.535890 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8d60ee53-a61c-4fc3-91c3-03ada60c18fd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.216905 Aug 28 01:18:25.536746 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1023/1819] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:25 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:25.539958 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26dbefc3-ba76-4ebb-aa8a-bc127345c730 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:25.540429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26dbefc3-ba76-4ebb-aa8a-bc127345c730 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:25.540852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26dbefc3-ba76-4ebb-aa8a-bc127345c730 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:25.541562 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26dbefc3-ba76-4ebb-aa8a-bc127345c730 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:25.542001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26dbefc3-ba76-4ebb-aa8a-bc127345c730 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:25.542397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26dbefc3-ba76-4ebb-aa8a-bc127345c730 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:25.672006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-26dbefc3-ba76-4ebb-aa8a-bc127345c730 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:25.681453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-26dbefc3-ba76-4ebb-aa8a-bc127345c730 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:25.682853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-26dbefc3-ba76-4ebb-aa8a-bc127345c730 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:25.684030 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-26dbefc3-ba76-4ebb-aa8a-bc127345c730 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.154102 Aug 28 01:18:25.684650 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 797/1820] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:25 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:26.550204 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-47487d88-166f-4b20-8440-1a47fab9e3cc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:26.550848 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-47487d88-166f-4b20-8440-1a47fab9e3cc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:26.557332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-82385f50-8001-4e46-a990-affee616805a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:26.557900 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47487d88-166f-4b20-8440-1a47fab9e3cc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:26.558031 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-82385f50-8001-4e46-a990-affee616805a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:26.558409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47487d88-166f-4b20-8440-1a47fab9e3cc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:26.558832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47487d88-166f-4b20-8440-1a47fab9e3cc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:26.559512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47487d88-166f-4b20-8440-1a47fab9e3cc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:26.559957 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47487d88-166f-4b20-8440-1a47fab9e3cc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:26.560375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47487d88-166f-4b20-8440-1a47fab9e3cc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:26.565476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82385f50-8001-4e46-a990-affee616805a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:26.565999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82385f50-8001-4e46-a990-affee616805a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:26.566413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82385f50-8001-4e46-a990-affee616805a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:26.567127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82385f50-8001-4e46-a990-affee616805a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:26.567584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82385f50-8001-4e46-a990-affee616805a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:26.568362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-82385f50-8001-4e46-a990-affee616805a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:26.761888 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-47487d88-166f-4b20-8440-1a47fab9e3cc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:26.764448 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-82385f50-8001-4e46-a990-affee616805a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:26.770839 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-47487d88-166f-4b20-8440-1a47fab9e3cc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.224523 Aug 28 01:18:26.771551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1024/1821] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:26 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:26.773100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-82385f50-8001-4e46-a990-affee616805a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.219682 Aug 28 01:18:26.773918 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 798/1822] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:26 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:26.776699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c3b64779-89a6-41e2-9d57-c52163bd69f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:26.777510 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c3b64779-89a6-41e2-9d57-c52163bd69f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:26.783919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3b64779-89a6-41e2-9d57-c52163bd69f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:26.784730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3b64779-89a6-41e2-9d57-c52163bd69f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:26.785301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3b64779-89a6-41e2-9d57-c52163bd69f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:26.786090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3b64779-89a6-41e2-9d57-c52163bd69f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:26.786666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3b64779-89a6-41e2-9d57-c52163bd69f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:26.787159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3b64779-89a6-41e2-9d57-c52163bd69f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:26.940164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c3b64779-89a6-41e2-9d57-c52163bd69f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:26.950086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c3b64779-89a6-41e2-9d57-c52163bd69f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:26.952134 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c3b64779-89a6-41e2-9d57-c52163bd69f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:26.953356 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c3b64779-89a6-41e2-9d57-c52163bd69f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.180800 Aug 28 01:18:26.953921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1025/1823] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:26 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:27.795662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c0934547-f4bf-4e77-a291-f951d473ef7a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:27.796393 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d805d7af-809f-4770-b1be-47ef226908bd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:27.796859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c0934547-f4bf-4e77-a291-f951d473ef7a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:27.797015 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d805d7af-809f-4770-b1be-47ef226908bd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:27.805195 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d805d7af-809f-4770-b1be-47ef226908bd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:27.805586 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0934547-f4bf-4e77-a291-f951d473ef7a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:27.805757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d805d7af-809f-4770-b1be-47ef226908bd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:27.806069 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0934547-f4bf-4e77-a291-f951d473ef7a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:27.806229 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d805d7af-809f-4770-b1be-47ef226908bd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:27.806498 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0934547-f4bf-4e77-a291-f951d473ef7a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:27.806750 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d805d7af-809f-4770-b1be-47ef226908bd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:27.807186 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0934547-f4bf-4e77-a291-f951d473ef7a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:27.807294 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d805d7af-809f-4770-b1be-47ef226908bd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:27.807597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d805d7af-809f-4770-b1be-47ef226908bd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:27.807752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0934547-f4bf-4e77-a291-f951d473ef7a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:27.808165 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0934547-f4bf-4e77-a291-f951d473ef7a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:28.005120 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c0934547-f4bf-4e77-a291-f951d473ef7a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:28.008360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d805d7af-809f-4770-b1be-47ef226908bd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:28.013718 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c0934547-f4bf-4e77-a291-f951d473ef7a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.222950 Aug 28 01:18:28.014368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 799/1824] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:27 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:28.016701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d805d7af-809f-4770-b1be-47ef226908bd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.225462 Aug 28 01:18:28.017445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1026/1825] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:27 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:28.019349 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3a00fbcd-e9f9-4ee9-a252-59008750ddcd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:28.020023 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3a00fbcd-e9f9-4ee9-a252-59008750ddcd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:28.026048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a00fbcd-e9f9-4ee9-a252-59008750ddcd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:28.026609 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a00fbcd-e9f9-4ee9-a252-59008750ddcd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:28.027111 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a00fbcd-e9f9-4ee9-a252-59008750ddcd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:28.027877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a00fbcd-e9f9-4ee9-a252-59008750ddcd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:28.028415 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a00fbcd-e9f9-4ee9-a252-59008750ddcd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:28.028901 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a00fbcd-e9f9-4ee9-a252-59008750ddcd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:28.164484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3a00fbcd-e9f9-4ee9-a252-59008750ddcd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:28.173607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3a00fbcd-e9f9-4ee9-a252-59008750ddcd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:28.175088 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3a00fbcd-e9f9-4ee9-a252-59008750ddcd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:28.176253 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3a00fbcd-e9f9-4ee9-a252-59008750ddcd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.160851 Aug 28 01:18:28.176807 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 800/1826] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:28 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:29.036152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a83e84c4-c1da-42c8-9071-8622a2c8ff40 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:29.036799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a83e84c4-c1da-42c8-9071-8622a2c8ff40 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:29.037561 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-766e8145-90d6-4584-b23f-528e9bb4ef83 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:29.038176 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-766e8145-90d6-4584-b23f-528e9bb4ef83 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:29.043618 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a83e84c4-c1da-42c8-9071-8622a2c8ff40 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:29.044105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a83e84c4-c1da-42c8-9071-8622a2c8ff40 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:29.044530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a83e84c4-c1da-42c8-9071-8622a2c8ff40 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:29.045078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-766e8145-90d6-4584-b23f-528e9bb4ef83 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:29.045243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a83e84c4-c1da-42c8-9071-8622a2c8ff40 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:29.045562 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-766e8145-90d6-4584-b23f-528e9bb4ef83 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:29.045688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a83e84c4-c1da-42c8-9071-8622a2c8ff40 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:29.045985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-766e8145-90d6-4584-b23f-528e9bb4ef83 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:29.046125 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a83e84c4-c1da-42c8-9071-8622a2c8ff40 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:29.046684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-766e8145-90d6-4584-b23f-528e9bb4ef83 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:29.047123 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-766e8145-90d6-4584-b23f-528e9bb4ef83 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:29.047548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-766e8145-90d6-4584-b23f-528e9bb4ef83 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:29.241841 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a83e84c4-c1da-42c8-9071-8622a2c8ff40 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:29.245008 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-766e8145-90d6-4584-b23f-528e9bb4ef83 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:29.250488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a83e84c4-c1da-42c8-9071-8622a2c8ff40 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.218318 Aug 28 01:18:29.251081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1027/1827] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:29 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:29.253744 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-766e8145-90d6-4584-b23f-528e9bb4ef83 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.219937 Aug 28 01:18:29.255059 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 801/1828] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:29 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:29.255883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1565e33f-ee47-492d-828e-2f7e668342a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:29.256482 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1565e33f-ee47-492d-828e-2f7e668342a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:29.262483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1565e33f-ee47-492d-828e-2f7e668342a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:29.262969 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1565e33f-ee47-492d-828e-2f7e668342a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:29.263411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1565e33f-ee47-492d-828e-2f7e668342a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:29.264132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1565e33f-ee47-492d-828e-2f7e668342a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:29.264578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1565e33f-ee47-492d-828e-2f7e668342a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:29.265045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1565e33f-ee47-492d-828e-2f7e668342a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:29.399496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1565e33f-ee47-492d-828e-2f7e668342a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:29.408995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1565e33f-ee47-492d-828e-2f7e668342a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:29.410440 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1565e33f-ee47-492d-828e-2f7e668342a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:29.411602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1565e33f-ee47-492d-828e-2f7e668342a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.159385 Aug 28 01:18:29.412130 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1028/1829] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:29 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:30.272563 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fd6cbdcf-9bf2-44d6-87dd-02654aaa564a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:30.273220 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fd6cbdcf-9bf2-44d6-87dd-02654aaa564a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:30.275452 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9d39ce18-58e6-4454-86d1-605e5916e035 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:30.275893 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=72 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:18:30.276199 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9d39ce18-58e6-4454-86d1-605e5916e035 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:30.281833 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd6cbdcf-9bf2-44d6-87dd-02654aaa564a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:30.282324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd6cbdcf-9bf2-44d6-87dd-02654aaa564a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:30.282748 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd6cbdcf-9bf2-44d6-87dd-02654aaa564a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:30.283158 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d39ce18-58e6-4454-86d1-605e5916e035 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:30.283473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd6cbdcf-9bf2-44d6-87dd-02654aaa564a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:30.283616 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d39ce18-58e6-4454-86d1-605e5916e035 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:30.283921 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd6cbdcf-9bf2-44d6-87dd-02654aaa564a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:30.284036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d39ce18-58e6-4454-86d1-605e5916e035 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:30.284338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd6cbdcf-9bf2-44d6-87dd-02654aaa564a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:30.284737 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d39ce18-58e6-4454-86d1-605e5916e035 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:30.285213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d39ce18-58e6-4454-86d1-605e5916e035 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:30.285639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d39ce18-58e6-4454-86d1-605e5916e035 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:30.488024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9d39ce18-58e6-4454-86d1-605e5916e035 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:30.490789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fd6cbdcf-9bf2-44d6-87dd-02654aaa564a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:30.498210 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9d39ce18-58e6-4454-86d1-605e5916e035 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.226850 Aug 28 01:18:30.499053 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1029/1830] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:30 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:18:30.500793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fd6cbdcf-9bf2-44d6-87dd-02654aaa564a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.232629 Aug 28 01:18:30.505099 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 802/1831] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:30 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:30.505300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ea2feccb-d269-4a7e-a5b1-a0f1540ad068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:30.505300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ea2feccb-d269-4a7e-a5b1-a0f1540ad068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:30.506250 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=72 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:18:30.511624 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea2feccb-d269-4a7e-a5b1-a0f1540ad068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:30.512080 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea2feccb-d269-4a7e-a5b1-a0f1540ad068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:30.512536 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea2feccb-d269-4a7e-a5b1-a0f1540ad068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:30.513265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea2feccb-d269-4a7e-a5b1-a0f1540ad068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:30.513710 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea2feccb-d269-4a7e-a5b1-a0f1540ad068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:30.514106 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea2feccb-d269-4a7e-a5b1-a0f1540ad068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:30.653781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ea2feccb-d269-4a7e-a5b1-a0f1540ad068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:30.663818 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ea2feccb-d269-4a7e-a5b1-a0f1540ad068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:30.665278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ea2feccb-d269-4a7e-a5b1-a0f1540ad068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:30.666444 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ea2feccb-d269-4a7e-a5b1-a0f1540ad068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.166354 Aug 28 01:18:30.667002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1030/1832] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:30 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:31.520791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-258c3bfa-90d8-4542-81a3-71c99ec4610b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:31.521412 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-258c3bfa-90d8-4542-81a3-71c99ec4610b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:31.522730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-62270016-bdb5-498f-bed5-3c92f4b8714a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:31.523320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-62270016-bdb5-498f-bed5-3c92f4b8714a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:31.528542 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-258c3bfa-90d8-4542-81a3-71c99ec4610b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:31.529061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-258c3bfa-90d8-4542-81a3-71c99ec4610b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:31.529484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-258c3bfa-90d8-4542-81a3-71c99ec4610b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:31.530254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-258c3bfa-90d8-4542-81a3-71c99ec4610b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:31.530735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-258c3bfa-90d8-4542-81a3-71c99ec4610b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:31.531151 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-258c3bfa-90d8-4542-81a3-71c99ec4610b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:31.531440 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62270016-bdb5-498f-bed5-3c92f4b8714a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:31.531927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62270016-bdb5-498f-bed5-3c92f4b8714a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:31.532354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62270016-bdb5-498f-bed5-3c92f4b8714a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:31.533094 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62270016-bdb5-498f-bed5-3c92f4b8714a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:31.533543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62270016-bdb5-498f-bed5-3c92f4b8714a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:31.533990 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62270016-bdb5-498f-bed5-3c92f4b8714a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:31.725000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-258c3bfa-90d8-4542-81a3-71c99ec4610b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:31.736719 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-258c3bfa-90d8-4542-81a3-71c99ec4610b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.219837 Aug 28 01:18:31.737498 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 803/1833] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:31 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:31.742332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7ae06e0e-283e-48f1-9924-f88dcc4cfde6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:31.742941 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7ae06e0e-283e-48f1-9924-f88dcc4cfde6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:31.743731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-62270016-bdb5-498f-bed5-3c92f4b8714a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:31.749152 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ae06e0e-283e-48f1-9924-f88dcc4cfde6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:31.749890 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ae06e0e-283e-48f1-9924-f88dcc4cfde6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:31.750320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ae06e0e-283e-48f1-9924-f88dcc4cfde6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:31.751023 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ae06e0e-283e-48f1-9924-f88dcc4cfde6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:31.751469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ae06e0e-283e-48f1-9924-f88dcc4cfde6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:31.751885 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ae06e0e-283e-48f1-9924-f88dcc4cfde6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:31.760297 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-62270016-bdb5-498f-bed5-3c92f4b8714a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.241125 Aug 28 01:18:31.761481 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1031/1834] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:31 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:31.891604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7ae06e0e-283e-48f1-9924-f88dcc4cfde6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:31.901084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7ae06e0e-283e-48f1-9924-f88dcc4cfde6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:31.902540 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7ae06e0e-283e-48f1-9924-f88dcc4cfde6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:31.903723 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7ae06e0e-283e-48f1-9924-f88dcc4cfde6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.165263 Aug 28 01:18:31.904283 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 804/1835] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:31 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:32.760107 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5b5d0b01-7ac9-4da9-b3a8-b592b097d3ab tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:32.760712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5b5d0b01-7ac9-4da9-b3a8-b592b097d3ab tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:32.767504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b5d0b01-7ac9-4da9-b3a8-b592b097d3ab tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:32.767988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b5d0b01-7ac9-4da9-b3a8-b592b097d3ab tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:32.768414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b5d0b01-7ac9-4da9-b3a8-b592b097d3ab tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:32.769341 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b5d0b01-7ac9-4da9-b3a8-b592b097d3ab tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:32.769792 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b5d0b01-7ac9-4da9-b3a8-b592b097d3ab tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:32.770211 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b5d0b01-7ac9-4da9-b3a8-b592b097d3ab tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:32.782715 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-da74d12e-0e69-4b0c-a444-94a6342af2e0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:32.783326 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-da74d12e-0e69-4b0c-a444-94a6342af2e0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:32.789761 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da74d12e-0e69-4b0c-a444-94a6342af2e0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:32.790218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da74d12e-0e69-4b0c-a444-94a6342af2e0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:32.790642 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da74d12e-0e69-4b0c-a444-94a6342af2e0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:32.791337 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da74d12e-0e69-4b0c-a444-94a6342af2e0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:32.791808 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da74d12e-0e69-4b0c-a444-94a6342af2e0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:32.792222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da74d12e-0e69-4b0c-a444-94a6342af2e0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:32.964653 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5b5d0b01-7ac9-4da9-b3a8-b592b097d3ab tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:32.973499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5b5d0b01-7ac9-4da9-b3a8-b592b097d3ab tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.217333 Aug 28 01:18:32.974268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1032/1836] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:32 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:32.979204 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-937b9633-f37c-4437-bf0b-cad222993204 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:32.979808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-937b9633-f37c-4437-bf0b-cad222993204 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:32.986354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-937b9633-f37c-4437-bf0b-cad222993204 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:32.986843 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-937b9633-f37c-4437-bf0b-cad222993204 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:32.987275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-937b9633-f37c-4437-bf0b-cad222993204 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:32.987984 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-937b9633-f37c-4437-bf0b-cad222993204 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:32.988952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-937b9633-f37c-4437-bf0b-cad222993204 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:32.989720 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-937b9633-f37c-4437-bf0b-cad222993204 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:32.998843 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-da74d12e-0e69-4b0c-a444-94a6342af2e0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:33.007732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-da74d12e-0e69-4b0c-a444-94a6342af2e0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.228860 Aug 28 01:18:33.008433 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 805/1837] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:32 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:18:33.131829 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-937b9633-f37c-4437-bf0b-cad222993204 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:33.140749 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-937b9633-f37c-4437-bf0b-cad222993204 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:33.142233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-937b9633-f37c-4437-bf0b-cad222993204 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:33.143394 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-937b9633-f37c-4437-bf0b-cad222993204 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.168195 Aug 28 01:18:33.143938 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1033/1838] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:32 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:33.996006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6ad0dc1b-5839-423e-9bd8-704318026b65 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:33.996568 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6ad0dc1b-5839-423e-9bd8-704318026b65 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:34.009113 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6ad0dc1b-5839-423e-9bd8-704318026b65 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:34.009113 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6ad0dc1b-5839-423e-9bd8-704318026b65 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:34.009113 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6ad0dc1b-5839-423e-9bd8-704318026b65 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:34.009113 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6ad0dc1b-5839-423e-9bd8-704318026b65 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:34.010735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6ad0dc1b-5839-423e-9bd8-704318026b65 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:34.010735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6ad0dc1b-5839-423e-9bd8-704318026b65 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:34.029223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b4e6a287-5217-4506-acc9-90a1f3f56707 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:34.029816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b4e6a287-5217-4506-acc9-90a1f3f56707 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:34.036741 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b4e6a287-5217-4506-acc9-90a1f3f56707 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:34.037255 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b4e6a287-5217-4506-acc9-90a1f3f56707 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:34.037694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b4e6a287-5217-4506-acc9-90a1f3f56707 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:34.038378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b4e6a287-5217-4506-acc9-90a1f3f56707 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:34.038817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b4e6a287-5217-4506-acc9-90a1f3f56707 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:34.039227 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b4e6a287-5217-4506-acc9-90a1f3f56707 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:34.454936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6ad0dc1b-5839-423e-9bd8-704318026b65 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:34.464612 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6ad0dc1b-5839-423e-9bd8-704318026b65 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.472639 Aug 28 01:18:34.465238 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 806/1839] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:33 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:34.466840 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b4e6a287-5217-4506-acc9-90a1f3f56707 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:34.470002 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a7cd929f-31e4-4bde-8543-4eefac1a3783 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:34.470610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a7cd929f-31e4-4bde-8543-4eefac1a3783 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:34.477151 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7cd929f-31e4-4bde-8543-4eefac1a3783 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:34.477647 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7cd929f-31e4-4bde-8543-4eefac1a3783 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:34.478075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7cd929f-31e4-4bde-8543-4eefac1a3783 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:34.478794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7cd929f-31e4-4bde-8543-4eefac1a3783 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:34.479247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7cd929f-31e4-4bde-8543-4eefac1a3783 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:34.479673 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7cd929f-31e4-4bde-8543-4eefac1a3783 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:34.481519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b4e6a287-5217-4506-acc9-90a1f3f56707 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.456104 Aug 28 01:18:34.482117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1034/1840] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:34 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:34.619063 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a7cd929f-31e4-4bde-8543-4eefac1a3783 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:34.627853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a7cd929f-31e4-4bde-8543-4eefac1a3783 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:34.629312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a7cd929f-31e4-4bde-8543-4eefac1a3783 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:34.630602 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a7cd929f-31e4-4bde-8543-4eefac1a3783 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.164378 Aug 28 01:18:34.631163 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 807/1841] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:34 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:35.488387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1b74a638-a815-45a4-a777-13b9efecd8ff tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:35.488936 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1b74a638-a815-45a4-a777-13b9efecd8ff tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:35.497133 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b74a638-a815-45a4-a777-13b9efecd8ff tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:35.497670 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b74a638-a815-45a4-a777-13b9efecd8ff tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:35.498108 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b74a638-a815-45a4-a777-13b9efecd8ff tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:35.498792 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b74a638-a815-45a4-a777-13b9efecd8ff tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:35.499266 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b74a638-a815-45a4-a777-13b9efecd8ff tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:35.499683 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b74a638-a815-45a4-a777-13b9efecd8ff tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:35.503755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5e412780-3ffa-4b2a-94af-4b16f74188bf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:35.504366 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5e412780-3ffa-4b2a-94af-4b16f74188bf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:35.511406 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e412780-3ffa-4b2a-94af-4b16f74188bf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:35.511884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e412780-3ffa-4b2a-94af-4b16f74188bf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:35.512313 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e412780-3ffa-4b2a-94af-4b16f74188bf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:35.513015 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e412780-3ffa-4b2a-94af-4b16f74188bf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:35.513483 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e412780-3ffa-4b2a-94af-4b16f74188bf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:35.513902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e412780-3ffa-4b2a-94af-4b16f74188bf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:35.695122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1b74a638-a815-45a4-a777-13b9efecd8ff tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:35.701642 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5e412780-3ffa-4b2a-94af-4b16f74188bf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:35.707734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1b74a638-a815-45a4-a777-13b9efecd8ff tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.224253 Aug 28 01:18:35.708605 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1035/1842] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:35 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:35.710333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5e412780-3ffa-4b2a-94af-4b16f74188bf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.210456 Aug 28 01:18:35.710970 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 808/1843] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:35 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:35.716416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fe4e97a7-3173-4d18-ae1f-0eadfef1d1ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:35.717775 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fe4e97a7-3173-4d18-ae1f-0eadfef1d1ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:35.726841 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe4e97a7-3173-4d18-ae1f-0eadfef1d1ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:35.727674 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe4e97a7-3173-4d18-ae1f-0eadfef1d1ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:35.728340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe4e97a7-3173-4d18-ae1f-0eadfef1d1ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:35.729280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe4e97a7-3173-4d18-ae1f-0eadfef1d1ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:35.729911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe4e97a7-3173-4d18-ae1f-0eadfef1d1ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:35.730508 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fe4e97a7-3173-4d18-ae1f-0eadfef1d1ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:35.881260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fe4e97a7-3173-4d18-ae1f-0eadfef1d1ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:35.890126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fe4e97a7-3173-4d18-ae1f-0eadfef1d1ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:35.891538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fe4e97a7-3173-4d18-ae1f-0eadfef1d1ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:35.892684 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fe4e97a7-3173-4d18-ae1f-0eadfef1d1ac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.182619 Aug 28 01:18:35.893547 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1036/1844] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:35 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:36.730154 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a50d541b-19b9-454c-8449-f0ab9d4f89ee tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:36.730758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a50d541b-19b9-454c-8449-f0ab9d4f89ee tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:36.731546 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e8bc939c-e8cd-496c-b461-2d8a1d27ac56 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:36.732165 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e8bc939c-e8cd-496c-b461-2d8a1d27ac56 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:36.737568 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a50d541b-19b9-454c-8449-f0ab9d4f89ee tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:36.738057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a50d541b-19b9-454c-8449-f0ab9d4f89ee tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:36.738497 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a50d541b-19b9-454c-8449-f0ab9d4f89ee tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:36.739141 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e8bc939c-e8cd-496c-b461-2d8a1d27ac56 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:36.739303 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a50d541b-19b9-454c-8449-f0ab9d4f89ee tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:36.739633 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e8bc939c-e8cd-496c-b461-2d8a1d27ac56 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:36.739778 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a50d541b-19b9-454c-8449-f0ab9d4f89ee tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:36.740047 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a50d541b-19b9-454c-8449-f0ab9d4f89ee tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:36.740190 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e8bc939c-e8cd-496c-b461-2d8a1d27ac56 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:36.740758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e8bc939c-e8cd-496c-b461-2d8a1d27ac56 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:36.741230 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e8bc939c-e8cd-496c-b461-2d8a1d27ac56 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:36.741783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e8bc939c-e8cd-496c-b461-2d8a1d27ac56 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:36.946053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a50d541b-19b9-454c-8449-f0ab9d4f89ee tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:36.953926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e8bc939c-e8cd-496c-b461-2d8a1d27ac56 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:36.962000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a50d541b-19b9-454c-8449-f0ab9d4f89ee tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.235710 Aug 28 01:18:36.962733 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 809/1845] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:36 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:36.965757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e8bc939c-e8cd-496c-b461-2d8a1d27ac56 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.237670 Aug 28 01:18:36.976999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9e7917b6-7b77-46ed-95df-43333326b732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:36.976999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9e7917b6-7b77-46ed-95df-43333326b732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:36.976999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e7917b6-7b77-46ed-95df-43333326b732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:36.976999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e7917b6-7b77-46ed-95df-43333326b732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:36.977449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e7917b6-7b77-46ed-95df-43333326b732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:36.977449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e7917b6-7b77-46ed-95df-43333326b732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:36.977449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e7917b6-7b77-46ed-95df-43333326b732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:36.977449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e7917b6-7b77-46ed-95df-43333326b732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:36.978758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1037/1846] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:36 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:37.161296 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9e7917b6-7b77-46ed-95df-43333326b732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:37.171589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9e7917b6-7b77-46ed-95df-43333326b732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:37.173035 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9e7917b6-7b77-46ed-95df-43333326b732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:37.174247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9e7917b6-7b77-46ed-95df-43333326b732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.210521 Aug 28 01:18:37.174825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 810/1847] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:36 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:37.985604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ca65298c-4ed5-4651-bc44-a3a0c944ddbf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:37.986205 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ca65298c-4ed5-4651-bc44-a3a0c944ddbf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:37.990332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b5e1c78c-55b6-4045-81be-cb6d8ccc09e4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:37.991003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b5e1c78c-55b6-4045-81be-cb6d8ccc09e4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:37.993216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ca65298c-4ed5-4651-bc44-a3a0c944ddbf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:37.993704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ca65298c-4ed5-4651-bc44-a3a0c944ddbf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:37.994114 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ca65298c-4ed5-4651-bc44-a3a0c944ddbf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:37.994821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ca65298c-4ed5-4651-bc44-a3a0c944ddbf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:37.995267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ca65298c-4ed5-4651-bc44-a3a0c944ddbf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:37.995681 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ca65298c-4ed5-4651-bc44-a3a0c944ddbf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:37.997950 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5e1c78c-55b6-4045-81be-cb6d8ccc09e4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:38.000291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5e1c78c-55b6-4045-81be-cb6d8ccc09e4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:38.000291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5e1c78c-55b6-4045-81be-cb6d8ccc09e4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:38.000291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5e1c78c-55b6-4045-81be-cb6d8ccc09e4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:38.000291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5e1c78c-55b6-4045-81be-cb6d8ccc09e4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:38.000847 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5e1c78c-55b6-4045-81be-cb6d8ccc09e4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:38.200856 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ca65298c-4ed5-4651-bc44-a3a0c944ddbf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:38.209907 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b5e1c78c-55b6-4045-81be-cb6d8ccc09e4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:38.211911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ca65298c-4ed5-4651-bc44-a3a0c944ddbf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.230189 Aug 28 01:18:38.212354 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:18:38.212486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1038/1848] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:37 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:38.217153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7ac4f280-ea77-4c0b-8bc9-b18122aa4d68 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:38.217866 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7ac4f280-ea77-4c0b-8bc9-b18122aa4d68 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:38.223866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b5e1c78c-55b6-4045-81be-cb6d8ccc09e4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.237429 Aug 28 01:18:38.224814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ac4f280-ea77-4c0b-8bc9-b18122aa4d68 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:38.224982 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 811/1849] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:37 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:18:38.225254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ac4f280-ea77-4c0b-8bc9-b18122aa4d68 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:38.225921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ac4f280-ea77-4c0b-8bc9-b18122aa4d68 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:38.226772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ac4f280-ea77-4c0b-8bc9-b18122aa4d68 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:38.227275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ac4f280-ea77-4c0b-8bc9-b18122aa4d68 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:38.227721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7ac4f280-ea77-4c0b-8bc9-b18122aa4d68 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:38.247040 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:18:38.372502 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7ac4f280-ea77-4c0b-8bc9-b18122aa4d68 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:38.382004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7ac4f280-ea77-4c0b-8bc9-b18122aa4d68 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:38.383392 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7ac4f280-ea77-4c0b-8bc9-b18122aa4d68 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:38.384599 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7ac4f280-ea77-4c0b-8bc9-b18122aa4d68 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.171168 Aug 28 01:18:38.385205 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1039/1850] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:38 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:39.233835 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3b0398f6-3bf2-4537-aca1-e04795937a66 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:39.234389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3b0398f6-3bf2-4537-aca1-e04795937a66 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:39.241745 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3b0398f6-3bf2-4537-aca1-e04795937a66 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:39.242224 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3b0398f6-3bf2-4537-aca1-e04795937a66 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:39.242651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3b0398f6-3bf2-4537-aca1-e04795937a66 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:39.243330 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3b0398f6-3bf2-4537-aca1-e04795937a66 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:39.243774 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3b0398f6-3bf2-4537-aca1-e04795937a66 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:39.244188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3b0398f6-3bf2-4537-aca1-e04795937a66 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:39.247277 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5d365fe5-fee3-4d9f-9f21-3b0fe6994ecd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:39.247889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5d365fe5-fee3-4d9f-9f21-3b0fe6994ecd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:39.254599 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d365fe5-fee3-4d9f-9f21-3b0fe6994ecd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:39.255078 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d365fe5-fee3-4d9f-9f21-3b0fe6994ecd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:39.255504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d365fe5-fee3-4d9f-9f21-3b0fe6994ecd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:39.256187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d365fe5-fee3-4d9f-9f21-3b0fe6994ecd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:39.256635 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d365fe5-fee3-4d9f-9f21-3b0fe6994ecd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:39.257081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d365fe5-fee3-4d9f-9f21-3b0fe6994ecd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:39.445725 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3b0398f6-3bf2-4537-aca1-e04795937a66 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:39.450169 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5d365fe5-fee3-4d9f-9f21-3b0fe6994ecd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:39.454188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3b0398f6-3bf2-4537-aca1-e04795937a66 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.224568 Aug 28 01:18:39.454807 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 812/1851] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:39 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:39.459098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5d365fe5-fee3-4d9f-9f21-3b0fe6994ecd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.215596 Aug 28 01:18:39.459272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c0aed215-9eba-4cd1-8253-ac1753373f08 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:39.459862 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c0aed215-9eba-4cd1-8253-ac1753373f08 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:39.465818 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0aed215-9eba-4cd1-8253-ac1753373f08 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:39.466293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0aed215-9eba-4cd1-8253-ac1753373f08 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:39.466918 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0aed215-9eba-4cd1-8253-ac1753373f08 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:39.467644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0aed215-9eba-4cd1-8253-ac1753373f08 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:39.468001 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1040/1852] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:39 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:39.468135 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0aed215-9eba-4cd1-8253-ac1753373f08 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:39.468507 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0aed215-9eba-4cd1-8253-ac1753373f08 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:39.614730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c0aed215-9eba-4cd1-8253-ac1753373f08 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:39.628024 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c0aed215-9eba-4cd1-8253-ac1753373f08 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:39.629850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c0aed215-9eba-4cd1-8253-ac1753373f08 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:39.631489 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c0aed215-9eba-4cd1-8253-ac1753373f08 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.175560 Aug 28 01:18:39.632400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 813/1853] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:39 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:40.476334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-77fcae77-0433-4e0e-a96d-b79bd6ef896b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:40.476923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-77fcae77-0433-4e0e-a96d-b79bd6ef896b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:40.479681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-808b0129-470f-485d-8d87-583eb67e67cd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:40.480355 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-808b0129-470f-485d-8d87-583eb67e67cd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:40.483865 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-77fcae77-0433-4e0e-a96d-b79bd6ef896b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:40.484351 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-77fcae77-0433-4e0e-a96d-b79bd6ef896b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:40.484804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-77fcae77-0433-4e0e-a96d-b79bd6ef896b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:40.485482 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-77fcae77-0433-4e0e-a96d-b79bd6ef896b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:40.485943 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-77fcae77-0433-4e0e-a96d-b79bd6ef896b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:40.486365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-77fcae77-0433-4e0e-a96d-b79bd6ef896b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:40.486961 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-808b0129-470f-485d-8d87-583eb67e67cd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:40.487441 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-808b0129-470f-485d-8d87-583eb67e67cd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:40.487874 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-808b0129-470f-485d-8d87-583eb67e67cd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:40.488563 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-808b0129-470f-485d-8d87-583eb67e67cd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:40.489031 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-808b0129-470f-485d-8d87-583eb67e67cd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:40.489455 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-808b0129-470f-485d-8d87-583eb67e67cd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:40.692249 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-77fcae77-0433-4e0e-a96d-b79bd6ef896b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:40.695728 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-808b0129-470f-485d-8d87-583eb67e67cd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:40.701226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-77fcae77-0433-4e0e-a96d-b79bd6ef896b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.228734 Aug 28 01:18:40.701832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1041/1854] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:40 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:40.705114 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-808b0129-470f-485d-8d87-583eb67e67cd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.229258 Aug 28 01:18:40.705695 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 814/1855] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:40 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:40.706479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ea5c1911-4e00-46dd-9f18-eeff57a6e844 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:40.707340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ea5c1911-4e00-46dd-9f18-eeff57a6e844 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:40.713564 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea5c1911-4e00-46dd-9f18-eeff57a6e844 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:40.714027 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea5c1911-4e00-46dd-9f18-eeff57a6e844 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:40.714450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea5c1911-4e00-46dd-9f18-eeff57a6e844 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:40.715138 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea5c1911-4e00-46dd-9f18-eeff57a6e844 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:40.715586 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea5c1911-4e00-46dd-9f18-eeff57a6e844 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:40.715987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea5c1911-4e00-46dd-9f18-eeff57a6e844 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:40.848372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ea5c1911-4e00-46dd-9f18-eeff57a6e844 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:40.858153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ea5c1911-4e00-46dd-9f18-eeff57a6e844 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:40.859567 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ea5c1911-4e00-46dd-9f18-eeff57a6e844 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:40.860707 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ea5c1911-4e00-46dd-9f18-eeff57a6e844 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.157776 Aug 28 01:18:40.861276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1042/1856] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:40 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:41.723514 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6b9e5065-5d5d-4237-9cbd-50cf3d400b02 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:41.724135 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6b9e5065-5d5d-4237-9cbd-50cf3d400b02 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:41.727158 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5e7affd2-90cc-45ce-b968-8f55193e7f29 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:41.727793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5e7affd2-90cc-45ce-b968-8f55193e7f29 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:41.731256 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b9e5065-5d5d-4237-9cbd-50cf3d400b02 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:41.731765 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b9e5065-5d5d-4237-9cbd-50cf3d400b02 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:41.732198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b9e5065-5d5d-4237-9cbd-50cf3d400b02 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:41.732900 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b9e5065-5d5d-4237-9cbd-50cf3d400b02 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:41.733371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b9e5065-5d5d-4237-9cbd-50cf3d400b02 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:41.733787 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6b9e5065-5d5d-4237-9cbd-50cf3d400b02 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:41.735239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e7affd2-90cc-45ce-b968-8f55193e7f29 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:41.735729 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e7affd2-90cc-45ce-b968-8f55193e7f29 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:41.736124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e7affd2-90cc-45ce-b968-8f55193e7f29 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:41.736805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e7affd2-90cc-45ce-b968-8f55193e7f29 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:41.737271 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e7affd2-90cc-45ce-b968-8f55193e7f29 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:41.737687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e7affd2-90cc-45ce-b968-8f55193e7f29 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:41.933651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6b9e5065-5d5d-4237-9cbd-50cf3d400b02 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:41.938894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5e7affd2-90cc-45ce-b968-8f55193e7f29 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:41.942448 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6b9e5065-5d5d-4237-9cbd-50cf3d400b02 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.222983 Aug 28 01:18:41.943379 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 815/1857] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:41 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:41.947999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8f9e5e51-1c28-467c-a8d8-99b059e14068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:41.948213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5e7affd2-90cc-45ce-b968-8f55193e7f29 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.224983 Aug 28 01:18:41.948680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1043/1858] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:41 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:41.949346 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8f9e5e51-1c28-467c-a8d8-99b059e14068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:41.955724 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8f9e5e51-1c28-467c-a8d8-99b059e14068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:41.956204 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8f9e5e51-1c28-467c-a8d8-99b059e14068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:41.956632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8f9e5e51-1c28-467c-a8d8-99b059e14068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:41.957354 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8f9e5e51-1c28-467c-a8d8-99b059e14068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:41.957811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8f9e5e51-1c28-467c-a8d8-99b059e14068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:41.958207 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8f9e5e51-1c28-467c-a8d8-99b059e14068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:42.102455 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8f9e5e51-1c28-467c-a8d8-99b059e14068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:42.115963 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8f9e5e51-1c28-467c-a8d8-99b059e14068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:42.117748 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8f9e5e51-1c28-467c-a8d8-99b059e14068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:42.119012 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8f9e5e51-1c28-467c-a8d8-99b059e14068 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.174742 Aug 28 01:18:42.119569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 816/1859] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:41 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:42.964794 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ea98eeb9-8425-49f4-928b-84a480ca86b0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:42.965470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ea98eeb9-8425-49f4-928b-84a480ca86b0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:42.970624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c1f8afb4-43a2-46f6-b3da-582d7cd96b15 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:42.971244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c1f8afb4-43a2-46f6-b3da-582d7cd96b15 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:42.972952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea98eeb9-8425-49f4-928b-84a480ca86b0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:42.973469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea98eeb9-8425-49f4-928b-84a480ca86b0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:42.973901 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea98eeb9-8425-49f4-928b-84a480ca86b0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:42.974626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea98eeb9-8425-49f4-928b-84a480ca86b0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:42.975053 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea98eeb9-8425-49f4-928b-84a480ca86b0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:42.975497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea98eeb9-8425-49f4-928b-84a480ca86b0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:42.978247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1f8afb4-43a2-46f6-b3da-582d7cd96b15 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:42.978718 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1f8afb4-43a2-46f6-b3da-582d7cd96b15 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:42.979146 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1f8afb4-43a2-46f6-b3da-582d7cd96b15 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:42.979855 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1f8afb4-43a2-46f6-b3da-582d7cd96b15 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:42.980296 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1f8afb4-43a2-46f6-b3da-582d7cd96b15 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:42.980710 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1f8afb4-43a2-46f6-b3da-582d7cd96b15 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:43.170260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ea98eeb9-8425-49f4-928b-84a480ca86b0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:43.175563 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c1f8afb4-43a2-46f6-b3da-582d7cd96b15 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:43.182154 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ea98eeb9-8425-49f4-928b-84a480ca86b0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.221038 Aug 28 01:18:43.182935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1044/1860] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:42 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:43.186228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c1f8afb4-43a2-46f6-b3da-582d7cd96b15 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.219387 Aug 28 01:18:43.187000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 817/1861] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:42 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:43.188086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ac491aaa-839a-4cd2-acfe-def8cc571dca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:43.189035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ac491aaa-839a-4cd2-acfe-def8cc571dca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:43.198239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac491aaa-839a-4cd2-acfe-def8cc571dca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:43.198951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac491aaa-839a-4cd2-acfe-def8cc571dca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:43.199601 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac491aaa-839a-4cd2-acfe-def8cc571dca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:43.200711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac491aaa-839a-4cd2-acfe-def8cc571dca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:43.201438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac491aaa-839a-4cd2-acfe-def8cc571dca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:43.202053 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac491aaa-839a-4cd2-acfe-def8cc571dca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:43.348178 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ac491aaa-839a-4cd2-acfe-def8cc571dca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:43.357337 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ac491aaa-839a-4cd2-acfe-def8cc571dca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:43.358769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ac491aaa-839a-4cd2-acfe-def8cc571dca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:43.360010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ac491aaa-839a-4cd2-acfe-def8cc571dca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.176050 Aug 28 01:18:43.360561 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1045/1862] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:43 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:44.208464 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f6860cf7-fb83-4dc2-b557-9538d8fff8c7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:44.209007 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-feceb51e-83fb-4b20-8972-24602ea0e2ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:44.209375 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f6860cf7-fb83-4dc2-b557-9538d8fff8c7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:44.210116 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-feceb51e-83fb-4b20-8972-24602ea0e2ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:44.216778 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-feceb51e-83fb-4b20-8972-24602ea0e2ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:44.217303 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-feceb51e-83fb-4b20-8972-24602ea0e2ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:44.217732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-feceb51e-83fb-4b20-8972-24602ea0e2ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:44.218450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-feceb51e-83fb-4b20-8972-24602ea0e2ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:44.218732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f6860cf7-fb83-4dc2-b557-9538d8fff8c7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:44.218861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-feceb51e-83fb-4b20-8972-24602ea0e2ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:44.219620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f6860cf7-fb83-4dc2-b557-9538d8fff8c7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:44.219759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-feceb51e-83fb-4b20-8972-24602ea0e2ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:44.220324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f6860cf7-fb83-4dc2-b557-9538d8fff8c7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:44.221067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f6860cf7-fb83-4dc2-b557-9538d8fff8c7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:44.221611 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f6860cf7-fb83-4dc2-b557-9538d8fff8c7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:44.222192 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f6860cf7-fb83-4dc2-b557-9538d8fff8c7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:44.425331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-feceb51e-83fb-4b20-8972-24602ea0e2ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:44.428275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f6860cf7-fb83-4dc2-b557-9538d8fff8c7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:44.434708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-feceb51e-83fb-4b20-8972-24602ea0e2ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.230539 Aug 28 01:18:44.440679 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1046/1863] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:44 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:44.440860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f6860cf7-fb83-4dc2-b557-9538d8fff8c7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.235369 Aug 28 01:18:44.440860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 818/1864] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:44 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:44.441074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3905c1d2-7b05-4c9d-a6f5-93f61e743887 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:44.441074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3905c1d2-7b05-4c9d-a6f5-93f61e743887 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:44.446725 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3905c1d2-7b05-4c9d-a6f5-93f61e743887 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:44.447222 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3905c1d2-7b05-4c9d-a6f5-93f61e743887 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:44.447662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3905c1d2-7b05-4c9d-a6f5-93f61e743887 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:44.448589 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3905c1d2-7b05-4c9d-a6f5-93f61e743887 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:44.449063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3905c1d2-7b05-4c9d-a6f5-93f61e743887 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:44.449484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3905c1d2-7b05-4c9d-a6f5-93f61e743887 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:44.579723 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3905c1d2-7b05-4c9d-a6f5-93f61e743887 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:44.593095 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3905c1d2-7b05-4c9d-a6f5-93f61e743887 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:44.595183 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3905c1d2-7b05-4c9d-a6f5-93f61e743887 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:44.597031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3905c1d2-7b05-4c9d-a6f5-93f61e743887 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.160462 Aug 28 01:18:44.597845 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1047/1865] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:44 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:45.457086 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-04f97338-66f8-4e24-ac0a-0fe72ab5a1a6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:45.457669 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-04f97338-66f8-4e24-ac0a-0fe72ab5a1a6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:45.461609 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4bb6697c-7b7b-4f47-b91b-f06473fc8606 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:45.462428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4bb6697c-7b7b-4f47-b91b-f06473fc8606 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:45.465398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04f97338-66f8-4e24-ac0a-0fe72ab5a1a6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:45.465595 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04f97338-66f8-4e24-ac0a-0fe72ab5a1a6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:45.466023 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04f97338-66f8-4e24-ac0a-0fe72ab5a1a6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:45.466775 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04f97338-66f8-4e24-ac0a-0fe72ab5a1a6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:45.467293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04f97338-66f8-4e24-ac0a-0fe72ab5a1a6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:45.467564 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04f97338-66f8-4e24-ac0a-0fe72ab5a1a6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:45.468902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bb6697c-7b7b-4f47-b91b-f06473fc8606 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:45.469498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bb6697c-7b7b-4f47-b91b-f06473fc8606 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:45.469923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bb6697c-7b7b-4f47-b91b-f06473fc8606 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:45.470806 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bb6697c-7b7b-4f47-b91b-f06473fc8606 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:45.471254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bb6697c-7b7b-4f47-b91b-f06473fc8606 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:45.471657 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bb6697c-7b7b-4f47-b91b-f06473fc8606 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:45.675204 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-04f97338-66f8-4e24-ac0a-0fe72ab5a1a6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:45.678639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4bb6697c-7b7b-4f47-b91b-f06473fc8606 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:45.684691 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-04f97338-66f8-4e24-ac0a-0fe72ab5a1a6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.231401 Aug 28 01:18:45.685354 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 819/1866] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:45 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:45.687373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4bb6697c-7b7b-4f47-b91b-f06473fc8606 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.229545 Aug 28 01:18:45.687940 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1048/1867] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:45 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:45.690329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-caaac6b3-e745-4b23-abf9-9eb00b205203 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:45.690944 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-caaac6b3-e745-4b23-abf9-9eb00b205203 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:45.697702 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-caaac6b3-e745-4b23-abf9-9eb00b205203 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:45.698202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-caaac6b3-e745-4b23-abf9-9eb00b205203 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:45.698641 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-caaac6b3-e745-4b23-abf9-9eb00b205203 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:45.699318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-caaac6b3-e745-4b23-abf9-9eb00b205203 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:45.699766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-caaac6b3-e745-4b23-abf9-9eb00b205203 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:45.700185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-caaac6b3-e745-4b23-abf9-9eb00b205203 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:45.846687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-caaac6b3-e745-4b23-abf9-9eb00b205203 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:45.855705 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-caaac6b3-e745-4b23-abf9-9eb00b205203 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:45.857169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-caaac6b3-e745-4b23-abf9-9eb00b205203 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:45.858375 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-caaac6b3-e745-4b23-abf9-9eb00b205203 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.171973 Aug 28 01:18:45.859288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 820/1868] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:45 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:46.708317 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cc94e4d9-14f1-4ddf-9b51-5ae4fa8e5352 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:46.709268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cc94e4d9-14f1-4ddf-9b51-5ae4fa8e5352 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:46.709452 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-53774ebe-d88f-48b8-a065-ef021f720f68 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:46.710106 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-53774ebe-d88f-48b8-a065-ef021f720f68 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:46.716122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cc94e4d9-14f1-4ddf-9b51-5ae4fa8e5352 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:46.716526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53774ebe-d88f-48b8-a065-ef021f720f68 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:46.716723 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cc94e4d9-14f1-4ddf-9b51-5ae4fa8e5352 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:46.717077 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53774ebe-d88f-48b8-a065-ef021f720f68 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:46.717284 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cc94e4d9-14f1-4ddf-9b51-5ae4fa8e5352 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:46.717472 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53774ebe-d88f-48b8-a065-ef021f720f68 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:46.717717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cc94e4d9-14f1-4ddf-9b51-5ae4fa8e5352 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:46.718108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53774ebe-d88f-48b8-a065-ef021f720f68 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:46.718299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cc94e4d9-14f1-4ddf-9b51-5ae4fa8e5352 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:46.718550 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53774ebe-d88f-48b8-a065-ef021f720f68 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:46.718734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cc94e4d9-14f1-4ddf-9b51-5ae4fa8e5352 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:46.718967 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-53774ebe-d88f-48b8-a065-ef021f720f68 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:46.920438 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-53774ebe-d88f-48b8-a065-ef021f720f68 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:46.925272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-cc94e4d9-14f1-4ddf-9b51-5ae4fa8e5352 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:46.929511 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-53774ebe-d88f-48b8-a065-ef021f720f68 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.224041 Aug 28 01:18:46.930166 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 821/1869] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:46 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:46.934931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-54f02013-2c4a-4390-90d6-c61be40cfc06 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:46.935121 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cc94e4d9-14f1-4ddf-9b51-5ae4fa8e5352 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.230514 Aug 28 01:18:46.935613 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-54f02013-2c4a-4390-90d6-c61be40cfc06 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:46.935922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1049/1870] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:46 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:46.949269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54f02013-2c4a-4390-90d6-c61be40cfc06 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:46.949269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54f02013-2c4a-4390-90d6-c61be40cfc06 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:46.950335 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54f02013-2c4a-4390-90d6-c61be40cfc06 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:46.950950 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54f02013-2c4a-4390-90d6-c61be40cfc06 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:46.951429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54f02013-2c4a-4390-90d6-c61be40cfc06 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:46.951847 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54f02013-2c4a-4390-90d6-c61be40cfc06 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:47.105538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-54f02013-2c4a-4390-90d6-c61be40cfc06 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:47.115835 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-54f02013-2c4a-4390-90d6-c61be40cfc06 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:47.117569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-54f02013-2c4a-4390-90d6-c61be40cfc06 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:47.119060 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-54f02013-2c4a-4390-90d6-c61be40cfc06 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.187539 Aug 28 01:18:47.119754 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 822/1871] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:46 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:47.953415 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7801a6ef-4915-4212-b46e-966c0bfa2d91 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:47.954035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7801a6ef-4915-4212-b46e-966c0bfa2d91 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:47.957381 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ccfef63d-8657-4e80-bbd2-246bdd20a582 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:47.958252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ccfef63d-8657-4e80-bbd2-246bdd20a582 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:47.963352 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7801a6ef-4915-4212-b46e-966c0bfa2d91 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:47.964281 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7801a6ef-4915-4212-b46e-966c0bfa2d91 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:47.964718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7801a6ef-4915-4212-b46e-966c0bfa2d91 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:47.965400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ccfef63d-8657-4e80-bbd2-246bdd20a582 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:47.965584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7801a6ef-4915-4212-b46e-966c0bfa2d91 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:47.965889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7801a6ef-4915-4212-b46e-966c0bfa2d91 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:47.966052 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ccfef63d-8657-4e80-bbd2-246bdd20a582 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:47.966288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7801a6ef-4915-4212-b46e-966c0bfa2d91 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:47.966416 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ccfef63d-8657-4e80-bbd2-246bdd20a582 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:47.966997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ccfef63d-8657-4e80-bbd2-246bdd20a582 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:47.967435 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ccfef63d-8657-4e80-bbd2-246bdd20a582 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:47.967853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ccfef63d-8657-4e80-bbd2-246bdd20a582 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:48.171121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ccfef63d-8657-4e80-bbd2-246bdd20a582 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:48.174411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7801a6ef-4915-4212-b46e-966c0bfa2d91 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:48.180837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ccfef63d-8657-4e80-bbd2-246bdd20a582 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.228950 Aug 28 01:18:48.181477 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 823/1872] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:47 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:48.183684 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7801a6ef-4915-4212-b46e-966c0bfa2d91 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.235162 Aug 28 01:18:48.185311 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1050/1873] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:47 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:48.186189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f1e8135a-7a29-46b1-a1a2-4ca8b49c22e6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:48.186803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f1e8135a-7a29-46b1-a1a2-4ca8b49c22e6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:48.193339 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1e8135a-7a29-46b1-a1a2-4ca8b49c22e6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:48.193825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1e8135a-7a29-46b1-a1a2-4ca8b49c22e6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:48.194255 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1e8135a-7a29-46b1-a1a2-4ca8b49c22e6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:48.195033 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1e8135a-7a29-46b1-a1a2-4ca8b49c22e6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:48.195490 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1e8135a-7a29-46b1-a1a2-4ca8b49c22e6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:48.195906 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1e8135a-7a29-46b1-a1a2-4ca8b49c22e6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:48.342784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f1e8135a-7a29-46b1-a1a2-4ca8b49c22e6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:48.352277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f1e8135a-7a29-46b1-a1a2-4ca8b49c22e6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:48.353779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f1e8135a-7a29-46b1-a1a2-4ca8b49c22e6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:48.354981 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f1e8135a-7a29-46b1-a1a2-4ca8b49c22e6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.172607 Aug 28 01:18:48.355549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 824/1874] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:48 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:49.203496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7c557b75-dd76-4d14-b0b9-dfd5366d5d0f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:49.204141 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7c557b75-dd76-4d14-b0b9-dfd5366d5d0f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:49.205161 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-449d26b7-2dcc-4344-ae15-7531ff2ad097 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:49.205768 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-449d26b7-2dcc-4344-ae15-7531ff2ad097 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:49.210776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c557b75-dd76-4d14-b0b9-dfd5366d5d0f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:49.211346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c557b75-dd76-4d14-b0b9-dfd5366d5d0f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:49.211757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c557b75-dd76-4d14-b0b9-dfd5366d5d0f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:49.212464 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c557b75-dd76-4d14-b0b9-dfd5366d5d0f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:49.212930 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c557b75-dd76-4d14-b0b9-dfd5366d5d0f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:49.213113 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-449d26b7-2dcc-4344-ae15-7531ff2ad097 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:49.213424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c557b75-dd76-4d14-b0b9-dfd5366d5d0f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:49.213586 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-449d26b7-2dcc-4344-ae15-7531ff2ad097 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:49.214059 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-449d26b7-2dcc-4344-ae15-7531ff2ad097 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:49.214732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-449d26b7-2dcc-4344-ae15-7531ff2ad097 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:49.215156 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-449d26b7-2dcc-4344-ae15-7531ff2ad097 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:49.215577 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-449d26b7-2dcc-4344-ae15-7531ff2ad097 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:49.410348 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7c557b75-dd76-4d14-b0b9-dfd5366d5d0f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:49.412527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-449d26b7-2dcc-4344-ae15-7531ff2ad097 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:49.419187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7c557b75-dd76-4d14-b0b9-dfd5366d5d0f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.219573 Aug 28 01:18:49.419960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1051/1875] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:49 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:49.420992 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-449d26b7-2dcc-4344-ae15-7531ff2ad097 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.219630 Aug 28 01:18:49.421734 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 825/1876] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:49 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:49.424592 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6e44b0ef-ca8e-4492-b530-906a47d861a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:49.425453 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6e44b0ef-ca8e-4492-b530-906a47d861a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:49.431792 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e44b0ef-ca8e-4492-b530-906a47d861a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:49.432269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e44b0ef-ca8e-4492-b530-906a47d861a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:49.433432 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e44b0ef-ca8e-4492-b530-906a47d861a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:49.434145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e44b0ef-ca8e-4492-b530-906a47d861a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:49.434664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e44b0ef-ca8e-4492-b530-906a47d861a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:49.435088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6e44b0ef-ca8e-4492-b530-906a47d861a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:49.580040 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6e44b0ef-ca8e-4492-b530-906a47d861a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:49.591245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6e44b0ef-ca8e-4492-b530-906a47d861a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:49.592686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6e44b0ef-ca8e-4492-b530-906a47d861a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:49.593952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6e44b0ef-ca8e-4492-b530-906a47d861a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.173092 Aug 28 01:18:49.594513 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1052/1877] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:49 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:50.441443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-070846d4-476e-4a5c-a409-d80752e37834 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:50.442060 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c8fff631-4d84-4850-8652-f3d4e081c397 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:50.442449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-070846d4-476e-4a5c-a409-d80752e37834 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:50.442585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c8fff631-4d84-4850-8652-f3d4e081c397 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:50.449527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-070846d4-476e-4a5c-a409-d80752e37834 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:50.449904 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c8fff631-4d84-4850-8652-f3d4e081c397 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:50.450082 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-070846d4-476e-4a5c-a409-d80752e37834 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:50.450375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c8fff631-4d84-4850-8652-f3d4e081c397 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:50.450502 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-070846d4-476e-4a5c-a409-d80752e37834 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:50.450803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c8fff631-4d84-4850-8652-f3d4e081c397 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:50.451108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-070846d4-476e-4a5c-a409-d80752e37834 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:50.451517 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c8fff631-4d84-4850-8652-f3d4e081c397 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:50.451681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-070846d4-476e-4a5c-a409-d80752e37834 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:50.451937 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c8fff631-4d84-4850-8652-f3d4e081c397 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:50.452076 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-070846d4-476e-4a5c-a409-d80752e37834 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:50.452350 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c8fff631-4d84-4850-8652-f3d4e081c397 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:50.649964 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c8fff631-4d84-4850-8652-f3d4e081c397 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:50.653357 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-070846d4-476e-4a5c-a409-d80752e37834 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:50.658816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c8fff631-4d84-4850-8652-f3d4e081c397 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.220706 Aug 28 01:18:50.659400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1053/1878] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:50 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:50.662324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-070846d4-476e-4a5c-a409-d80752e37834 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.224805 Aug 28 01:18:50.662915 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 826/1879] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:50 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:50.667586 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5d23959c-dbf0-4a9f-bc63-27f9570a5d42 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:50.668374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5d23959c-dbf0-4a9f-bc63-27f9570a5d42 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:50.674695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d23959c-dbf0-4a9f-bc63-27f9570a5d42 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:50.675238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d23959c-dbf0-4a9f-bc63-27f9570a5d42 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:50.675742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d23959c-dbf0-4a9f-bc63-27f9570a5d42 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:50.676438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d23959c-dbf0-4a9f-bc63-27f9570a5d42 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:50.676906 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d23959c-dbf0-4a9f-bc63-27f9570a5d42 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:50.677360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d23959c-dbf0-4a9f-bc63-27f9570a5d42 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:50.812087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5d23959c-dbf0-4a9f-bc63-27f9570a5d42 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:50.821519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5d23959c-dbf0-4a9f-bc63-27f9570a5d42 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:50.822968 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5d23959c-dbf0-4a9f-bc63-27f9570a5d42 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:50.824096 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5d23959c-dbf0-4a9f-bc63-27f9570a5d42 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.160726 Aug 28 01:18:50.824706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1054/1880] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:50 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:51.680798 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cba7d07f-0d62-4b0a-9d30-be06a79aa3f9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:51.681422 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cba7d07f-0d62-4b0a-9d30-be06a79aa3f9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:51.683538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e1429d98-f9d5-445c-a222-c030caa1617a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:51.684149 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e1429d98-f9d5-445c-a222-c030caa1617a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:51.688240 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cba7d07f-0d62-4b0a-9d30-be06a79aa3f9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:51.688719 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cba7d07f-0d62-4b0a-9d30-be06a79aa3f9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:51.689195 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cba7d07f-0d62-4b0a-9d30-be06a79aa3f9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:51.689912 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cba7d07f-0d62-4b0a-9d30-be06a79aa3f9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:51.690357 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cba7d07f-0d62-4b0a-9d30-be06a79aa3f9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:51.690775 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cba7d07f-0d62-4b0a-9d30-be06a79aa3f9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:51.691051 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1429d98-f9d5-445c-a222-c030caa1617a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:51.691557 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1429d98-f9d5-445c-a222-c030caa1617a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:51.691965 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1429d98-f9d5-445c-a222-c030caa1617a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:51.693007 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1429d98-f9d5-445c-a222-c030caa1617a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:51.693515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1429d98-f9d5-445c-a222-c030caa1617a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:51.693930 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1429d98-f9d5-445c-a222-c030caa1617a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:51.883111 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-cba7d07f-0d62-4b0a-9d30-be06a79aa3f9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:51.890460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e1429d98-f9d5-445c-a222-c030caa1617a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:51.897540 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cba7d07f-0d62-4b0a-9d30-be06a79aa3f9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.220619 Aug 28 01:18:51.898186 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 827/1881] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:51 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:51.898717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e1429d98-f9d5-445c-a222-c030caa1617a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.219110 Aug 28 01:18:51.899384 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1055/1882] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:51 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:51.903249 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7de93dc1-0f49-4901-ab79-3e97ccb63224 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:51.904011 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7de93dc1-0f49-4901-ab79-3e97ccb63224 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:51.910428 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7de93dc1-0f49-4901-ab79-3e97ccb63224 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:51.910917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7de93dc1-0f49-4901-ab79-3e97ccb63224 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:51.911348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7de93dc1-0f49-4901-ab79-3e97ccb63224 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:51.912042 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7de93dc1-0f49-4901-ab79-3e97ccb63224 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:51.912485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7de93dc1-0f49-4901-ab79-3e97ccb63224 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:51.912903 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7de93dc1-0f49-4901-ab79-3e97ccb63224 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:51.940184 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:18:52.062236 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7de93dc1-0f49-4901-ab79-3e97ccb63224 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:52.076742 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7de93dc1-0f49-4901-ab79-3e97ccb63224 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:52.078575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7de93dc1-0f49-4901-ab79-3e97ccb63224 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:52.080016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7de93dc1-0f49-4901-ab79-3e97ccb63224 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.180452 Aug 28 01:18:52.080556 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 828/1883] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:51 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:52.919973 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-28138e02-2d9a-47a5-af60-b6785b280b11 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:52.920555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-28138e02-2d9a-47a5-af60-b6785b280b11 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:52.926003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-00fb3851-c3e3-485a-ac34-2e04806d598e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:52.926719 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-00fb3851-c3e3-485a-ac34-2e04806d598e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:52.933308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00fb3851-c3e3-485a-ac34-2e04806d598e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:52.933906 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00fb3851-c3e3-485a-ac34-2e04806d598e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:52.934219 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00fb3851-c3e3-485a-ac34-2e04806d598e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:52.934926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00fb3851-c3e3-485a-ac34-2e04806d598e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:52.935576 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00fb3851-c3e3-485a-ac34-2e04806d598e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:52.936001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00fb3851-c3e3-485a-ac34-2e04806d598e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:52.936515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-28138e02-2d9a-47a5-af60-b6785b280b11 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:52.937028 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-28138e02-2d9a-47a5-af60-b6785b280b11 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:52.937474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-28138e02-2d9a-47a5-af60-b6785b280b11 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:52.938182 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-28138e02-2d9a-47a5-af60-b6785b280b11 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:52.938818 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-28138e02-2d9a-47a5-af60-b6785b280b11 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:52.939247 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-28138e02-2d9a-47a5-af60-b6785b280b11 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:52.944215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:18:53.139145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-00fb3851-c3e3-485a-ac34-2e04806d598e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:53.143517 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-28138e02-2d9a-47a5-af60-b6785b280b11 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:53.147661 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-00fb3851-c3e3-485a-ac34-2e04806d598e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.225397 Aug 28 01:18:53.148430 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 829/1884] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:52 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:53.153406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-28138e02-2d9a-47a5-af60-b6785b280b11 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.237301 Aug 28 01:18:53.154204 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1056/1885] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:52 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:53.163180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f9da960a-7a1b-45d2-834e-e29bdc996819 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:53.163802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f9da960a-7a1b-45d2-834e-e29bdc996819 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:53.169980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9da960a-7a1b-45d2-834e-e29bdc996819 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:53.170468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9da960a-7a1b-45d2-834e-e29bdc996819 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:53.170896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9da960a-7a1b-45d2-834e-e29bdc996819 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:53.171597 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9da960a-7a1b-45d2-834e-e29bdc996819 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:53.172029 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9da960a-7a1b-45d2-834e-e29bdc996819 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:53.172446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9da960a-7a1b-45d2-834e-e29bdc996819 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:53.306253 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f9da960a-7a1b-45d2-834e-e29bdc996819 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:53.315111 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f9da960a-7a1b-45d2-834e-e29bdc996819 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:53.316537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f9da960a-7a1b-45d2-834e-e29bdc996819 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:53.318118 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f9da960a-7a1b-45d2-834e-e29bdc996819 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.168036 Aug 28 01:18:53.318664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 830/1886] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:53 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:54.171491 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a586503f-9913-4747-a4fa-9cf12930a555 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:54.172171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a586503f-9913-4747-a4fa-9cf12930a555 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:54.176199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-27f4be1c-0916-4fc7-beca-b2847ecd9f2e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:54.176815 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-27f4be1c-0916-4fc7-beca-b2847ecd9f2e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:54.180402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a586503f-9913-4747-a4fa-9cf12930a555 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:54.181059 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a586503f-9913-4747-a4fa-9cf12930a555 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:54.181676 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a586503f-9913-4747-a4fa-9cf12930a555 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:54.182374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a586503f-9913-4747-a4fa-9cf12930a555 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:54.182821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a586503f-9913-4747-a4fa-9cf12930a555 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:54.183235 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a586503f-9913-4747-a4fa-9cf12930a555 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:54.184144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-27f4be1c-0916-4fc7-beca-b2847ecd9f2e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:54.184626 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-27f4be1c-0916-4fc7-beca-b2847ecd9f2e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:54.185090 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-27f4be1c-0916-4fc7-beca-b2847ecd9f2e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:54.185789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-27f4be1c-0916-4fc7-beca-b2847ecd9f2e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:54.186219 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-27f4be1c-0916-4fc7-beca-b2847ecd9f2e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:54.186644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-27f4be1c-0916-4fc7-beca-b2847ecd9f2e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:54.377970 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a586503f-9913-4747-a4fa-9cf12930a555 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:54.380820 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-27f4be1c-0916-4fc7-beca-b2847ecd9f2e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:54.386409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a586503f-9913-4747-a4fa-9cf12930a555 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.220111 Aug 28 01:18:54.386960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1057/1887] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:54 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:54.389323 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-27f4be1c-0916-4fc7-beca-b2847ecd9f2e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.217539 Aug 28 01:18:54.389894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 831/1888] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:54 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:54.391407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ea8f4518-285d-461f-ab23-83a1e17e00d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:54.392137 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ea8f4518-285d-461f-ab23-83a1e17e00d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:54.398594 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea8f4518-285d-461f-ab23-83a1e17e00d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:54.399288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea8f4518-285d-461f-ab23-83a1e17e00d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:54.399510 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea8f4518-285d-461f-ab23-83a1e17e00d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:54.400532 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea8f4518-285d-461f-ab23-83a1e17e00d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:54.400993 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea8f4518-285d-461f-ab23-83a1e17e00d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:54.401424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea8f4518-285d-461f-ab23-83a1e17e00d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:54.545509 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ea8f4518-285d-461f-ab23-83a1e17e00d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:54.559618 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ea8f4518-285d-461f-ab23-83a1e17e00d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:54.562037 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ea8f4518-285d-461f-ab23-83a1e17e00d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:54.563905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ea8f4518-285d-461f-ab23-83a1e17e00d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.175715 Aug 28 01:18:54.564790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1058/1889] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:54 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:55.413030 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-57612ae3-e009-4e62-9782-bb097807c57d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:55.414138 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-57612ae3-e009-4e62-9782-bb097807c57d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:55.414330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-73bd26ae-6a97-44f7-9179-ba98da83a79d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:55.415278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-73bd26ae-6a97-44f7-9179-ba98da83a79d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:55.424195 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-57612ae3-e009-4e62-9782-bb097807c57d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:55.424934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-57612ae3-e009-4e62-9782-bb097807c57d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:55.425169 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-73bd26ae-6a97-44f7-9179-ba98da83a79d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:55.425657 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-57612ae3-e009-4e62-9782-bb097807c57d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:55.425858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-73bd26ae-6a97-44f7-9179-ba98da83a79d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:55.426346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-73bd26ae-6a97-44f7-9179-ba98da83a79d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:55.426738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-57612ae3-e009-4e62-9782-bb097807c57d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:55.427412 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-73bd26ae-6a97-44f7-9179-ba98da83a79d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:55.427567 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-57612ae3-e009-4e62-9782-bb097807c57d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:55.428035 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-57612ae3-e009-4e62-9782-bb097807c57d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:55.428199 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-73bd26ae-6a97-44f7-9179-ba98da83a79d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:55.428656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-73bd26ae-6a97-44f7-9179-ba98da83a79d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:55.630571 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-73bd26ae-6a97-44f7-9179-ba98da83a79d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:55.634730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-57612ae3-e009-4e62-9782-bb097807c57d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:55.644778 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-73bd26ae-6a97-44f7-9179-ba98da83a79d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.236525 Aug 28 01:18:55.645625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1059/1890] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:55 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:55.646628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-57612ae3-e009-4e62-9782-bb097807c57d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.240446 Aug 28 01:18:55.647880 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 832/1891] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:55 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:55.650863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3d12450f-88b8-491a-81ec-dbfe3ac5915e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:55.651815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3d12450f-88b8-491a-81ec-dbfe3ac5915e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:55.658016 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d12450f-88b8-491a-81ec-dbfe3ac5915e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:55.658493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d12450f-88b8-491a-81ec-dbfe3ac5915e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:55.658918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d12450f-88b8-491a-81ec-dbfe3ac5915e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:55.659603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d12450f-88b8-491a-81ec-dbfe3ac5915e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:55.660037 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d12450f-88b8-491a-81ec-dbfe3ac5915e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:55.660450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d12450f-88b8-491a-81ec-dbfe3ac5915e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:55.796055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3d12450f-88b8-491a-81ec-dbfe3ac5915e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:55.805238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3d12450f-88b8-491a-81ec-dbfe3ac5915e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:55.806811 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3d12450f-88b8-491a-81ec-dbfe3ac5915e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:55.807993 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3d12450f-88b8-491a-81ec-dbfe3ac5915e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.161240 Aug 28 01:18:55.808556 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1060/1892] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:55 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:56.667721 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-42bcf673-e09d-47e2-8a0a-cac8a7316ece tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:56.668345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d3bf6f48-1fe8-4fb1-a59c-e4a5ef68f33c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:56.668715 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-42bcf673-e09d-47e2-8a0a-cac8a7316ece tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:56.668831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d3bf6f48-1fe8-4fb1-a59c-e4a5ef68f33c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:56.675325 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42bcf673-e09d-47e2-8a0a-cac8a7316ece tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:56.675512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d3bf6f48-1fe8-4fb1-a59c-e4a5ef68f33c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:56.675810 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42bcf673-e09d-47e2-8a0a-cac8a7316ece tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:56.675959 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d3bf6f48-1fe8-4fb1-a59c-e4a5ef68f33c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:56.676287 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d3bf6f48-1fe8-4fb1-a59c-e4a5ef68f33c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:56.676434 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42bcf673-e09d-47e2-8a0a-cac8a7316ece tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:56.676988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d3bf6f48-1fe8-4fb1-a59c-e4a5ef68f33c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:56.677136 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42bcf673-e09d-47e2-8a0a-cac8a7316ece tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:56.677447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d3bf6f48-1fe8-4fb1-a59c-e4a5ef68f33c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:56.677623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42bcf673-e09d-47e2-8a0a-cac8a7316ece tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:56.677910 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d3bf6f48-1fe8-4fb1-a59c-e4a5ef68f33c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:56.678081 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42bcf673-e09d-47e2-8a0a-cac8a7316ece tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:56.878803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d3bf6f48-1fe8-4fb1-a59c-e4a5ef68f33c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:56.883096 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-42bcf673-e09d-47e2-8a0a-cac8a7316ece tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:56.891017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d3bf6f48-1fe8-4fb1-a59c-e4a5ef68f33c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.226635 Aug 28 01:18:56.891978 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1061/1893] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:56 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:56.894147 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-42bcf673-e09d-47e2-8a0a-cac8a7316ece tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.230606 Aug 28 01:18:56.894707 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 833/1894] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:56 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:56.896732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6c4b3087-ea2a-4afa-bbc1-62bb11da38a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:56.897454 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6c4b3087-ea2a-4afa-bbc1-62bb11da38a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:56.903877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c4b3087-ea2a-4afa-bbc1-62bb11da38a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:56.904621 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c4b3087-ea2a-4afa-bbc1-62bb11da38a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:56.905148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c4b3087-ea2a-4afa-bbc1-62bb11da38a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:56.905881 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c4b3087-ea2a-4afa-bbc1-62bb11da38a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:56.906507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c4b3087-ea2a-4afa-bbc1-62bb11da38a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:56.906991 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c4b3087-ea2a-4afa-bbc1-62bb11da38a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:57.038194 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6c4b3087-ea2a-4afa-bbc1-62bb11da38a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:57.050528 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6c4b3087-ea2a-4afa-bbc1-62bb11da38a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:57.052638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6c4b3087-ea2a-4afa-bbc1-62bb11da38a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:57.054092 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6c4b3087-ea2a-4afa-bbc1-62bb11da38a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.160980 Aug 28 01:18:57.054930 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1062/1895] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:56 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:57.915746 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6c80720b-558b-4c3b-99da-d71903220a2d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:57.916336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6c80720b-558b-4c3b-99da-d71903220a2d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:57.916816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bc8cd943-5ba6-4757-b117-f289fc93ee8d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:57.917752 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bc8cd943-5ba6-4757-b117-f289fc93ee8d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:57.925503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6c80720b-558b-4c3b-99da-d71903220a2d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:57.926208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6c80720b-558b-4c3b-99da-d71903220a2d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:57.926378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc8cd943-5ba6-4757-b117-f289fc93ee8d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:57.926726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc8cd943-5ba6-4757-b117-f289fc93ee8d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:57.926880 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6c80720b-558b-4c3b-99da-d71903220a2d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:57.927144 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc8cd943-5ba6-4757-b117-f289fc93ee8d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:57.927434 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6c80720b-558b-4c3b-99da-d71903220a2d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:57.927874 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6c80720b-558b-4c3b-99da-d71903220a2d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:57.928032 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc8cd943-5ba6-4757-b117-f289fc93ee8d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:57.928343 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6c80720b-558b-4c3b-99da-d71903220a2d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:57.928490 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc8cd943-5ba6-4757-b117-f289fc93ee8d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:57.928902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc8cd943-5ba6-4757-b117-f289fc93ee8d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:58.132284 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6c80720b-558b-4c3b-99da-d71903220a2d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:58.135916 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bc8cd943-5ba6-4757-b117-f289fc93ee8d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:58.141032 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6c80720b-558b-4c3b-99da-d71903220a2d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.230648 Aug 28 01:18:58.141728 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 834/1896] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:57 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:58.144393 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bc8cd943-5ba6-4757-b117-f289fc93ee8d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.232674 Aug 28 01:18:58.145132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1063/1897] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:57 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:58.146454 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5e732bb1-1de2-4938-92f4-80936ebebef1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:58.147059 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5e732bb1-1de2-4938-92f4-80936ebebef1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:58.153236 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e732bb1-1de2-4938-92f4-80936ebebef1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:58.153807 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e732bb1-1de2-4938-92f4-80936ebebef1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:58.154244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e732bb1-1de2-4938-92f4-80936ebebef1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:58.154946 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e732bb1-1de2-4938-92f4-80936ebebef1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:58.155400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e732bb1-1de2-4938-92f4-80936ebebef1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:58.155822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e732bb1-1de2-4938-92f4-80936ebebef1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:58.294247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5e732bb1-1de2-4938-92f4-80936ebebef1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:58.303044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5e732bb1-1de2-4938-92f4-80936ebebef1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:58.304517 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5e732bb1-1de2-4938-92f4-80936ebebef1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:58.305721 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5e732bb1-1de2-4938-92f4-80936ebebef1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.163087 Aug 28 01:18:58.306275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 835/1898] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:58 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:18:59.168264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-976958ac-f14c-485c-b439-241a55f7e9b2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:59.169019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-976958ac-f14c-485c-b439-241a55f7e9b2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:59.173614 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0bc8ce89-c44b-4f84-91fe-3fa37ae58859 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:59.174227 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0bc8ce89-c44b-4f84-91fe-3fa37ae58859 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:59.175727 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-976958ac-f14c-485c-b439-241a55f7e9b2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:59.176207 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-976958ac-f14c-485c-b439-241a55f7e9b2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:59.176658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-976958ac-f14c-485c-b439-241a55f7e9b2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:59.177387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-976958ac-f14c-485c-b439-241a55f7e9b2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:59.177827 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-976958ac-f14c-485c-b439-241a55f7e9b2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:59.178241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-976958ac-f14c-485c-b439-241a55f7e9b2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:59.181010 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0bc8ce89-c44b-4f84-91fe-3fa37ae58859 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:59.181722 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0bc8ce89-c44b-4f84-91fe-3fa37ae58859 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:59.182251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0bc8ce89-c44b-4f84-91fe-3fa37ae58859 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:59.183206 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0bc8ce89-c44b-4f84-91fe-3fa37ae58859 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:59.190235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0bc8ce89-c44b-4f84-91fe-3fa37ae58859 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:59.190678 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0bc8ce89-c44b-4f84-91fe-3fa37ae58859 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:59.382542 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-976958ac-f14c-485c-b439-241a55f7e9b2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:59.386628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0bc8ce89-c44b-4f84-91fe-3fa37ae58859 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:59.391100 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-976958ac-f14c-485c-b439-241a55f7e9b2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.226810 Aug 28 01:18:59.391686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1064/1899] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:59 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:18:59.395924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0bc8ce89-c44b-4f84-91fe-3fa37ae58859 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.226085 Aug 28 01:18:59.396226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-de821c5e-ceb4-48de-8676-762a1781eaa8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:18:59.396880 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 836/1900] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:18:59 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:18:59.397043 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-de821c5e-ceb4-48de-8676-762a1781eaa8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:18:59.402912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de821c5e-ceb4-48de-8676-762a1781eaa8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:59.403453 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de821c5e-ceb4-48de-8676-762a1781eaa8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:59.403962 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de821c5e-ceb4-48de-8676-762a1781eaa8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:59.404690 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de821c5e-ceb4-48de-8676-762a1781eaa8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:18:59.405184 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de821c5e-ceb4-48de-8676-762a1781eaa8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:18:59.405608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de821c5e-ceb4-48de-8676-762a1781eaa8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:18:59.543590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-de821c5e-ceb4-48de-8676-762a1781eaa8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:59.552345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-de821c5e-ceb4-48de-8676-762a1781eaa8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:59.553808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-de821c5e-ceb4-48de-8676-762a1781eaa8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:18:59.554950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-de821c5e-ceb4-48de-8676-762a1781eaa8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.162323 Aug 28 01:18:59.555500 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1065/1901] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:18:59 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:00.413391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-26ef93b5-b81b-4106-a9c4-b359b20ce5cc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:00.413985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-26ef93b5-b81b-4106-a9c4-b359b20ce5cc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:00.416502 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=70 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:19:00.417849 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ab45dbb8-a614-4e64-9c55-f02b6fa55b57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:00.418480 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ab45dbb8-a614-4e64-9c55-f02b6fa55b57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:00.422371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26ef93b5-b81b-4106-a9c4-b359b20ce5cc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:00.422859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26ef93b5-b81b-4106-a9c4-b359b20ce5cc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:00.423284 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26ef93b5-b81b-4106-a9c4-b359b20ce5cc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:00.424031 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26ef93b5-b81b-4106-a9c4-b359b20ce5cc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:00.424558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26ef93b5-b81b-4106-a9c4-b359b20ce5cc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:00.424902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26ef93b5-b81b-4106-a9c4-b359b20ce5cc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:00.425031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab45dbb8-a614-4e64-9c55-f02b6fa55b57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:00.425497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab45dbb8-a614-4e64-9c55-f02b6fa55b57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:00.425930 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab45dbb8-a614-4e64-9c55-f02b6fa55b57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:00.426645 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab45dbb8-a614-4e64-9c55-f02b6fa55b57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:00.427092 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab45dbb8-a614-4e64-9c55-f02b6fa55b57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:00.427515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab45dbb8-a614-4e64-9c55-f02b6fa55b57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:00.616665 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-26ef93b5-b81b-4106-a9c4-b359b20ce5cc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:00.619796 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ab45dbb8-a614-4e64-9c55-f02b6fa55b57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:00.625162 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-26ef93b5-b81b-4106-a9c4-b359b20ce5cc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.215686 Aug 28 01:19:00.625886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 837/1902] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:00 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:00.628230 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ab45dbb8-a614-4e64-9c55-f02b6fa55b57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.214144 Aug 28 01:19:00.628768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1066/1903] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:00 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:00.638253 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3d269a9a-8bc9-45e7-a5b6-21ab55af0d05 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:00.639010 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3d269a9a-8bc9-45e7-a5b6-21ab55af0d05 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:00.647295 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d269a9a-8bc9-45e7-a5b6-21ab55af0d05 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:00.647895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d269a9a-8bc9-45e7-a5b6-21ab55af0d05 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:00.648470 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d269a9a-8bc9-45e7-a5b6-21ab55af0d05 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:00.649495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d269a9a-8bc9-45e7-a5b6-21ab55af0d05 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:00.650140 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d269a9a-8bc9-45e7-a5b6-21ab55af0d05 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:00.650641 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d269a9a-8bc9-45e7-a5b6-21ab55af0d05 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:00.783397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3d269a9a-8bc9-45e7-a5b6-21ab55af0d05 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:00.797411 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3d269a9a-8bc9-45e7-a5b6-21ab55af0d05 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:00.799692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3d269a9a-8bc9-45e7-a5b6-21ab55af0d05 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:00.801537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3d269a9a-8bc9-45e7-a5b6-21ab55af0d05 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.167187 Aug 28 01:19:00.802377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 838/1904] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:00 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:01.647555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4d8b70ac-5d15-43a3-b9cc-413837bee77e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:01.648347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4d8b70ac-5d15-43a3-b9cc-413837bee77e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:01.649168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-db4f74ae-327c-4617-ae88-cac3f212a479 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:01.649771 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-db4f74ae-327c-4617-ae88-cac3f212a479 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:01.652060 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=74 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:19:01.655842 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4d8b70ac-5d15-43a3-b9cc-413837bee77e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:01.656275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4d8b70ac-5d15-43a3-b9cc-413837bee77e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:01.656439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-db4f74ae-327c-4617-ae88-cac3f212a479 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:01.656695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4d8b70ac-5d15-43a3-b9cc-413837bee77e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:01.656931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-db4f74ae-327c-4617-ae88-cac3f212a479 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:01.657375 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-db4f74ae-327c-4617-ae88-cac3f212a479 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:01.657503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4d8b70ac-5d15-43a3-b9cc-413837bee77e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:01.657869 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4d8b70ac-5d15-43a3-b9cc-413837bee77e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:01.658073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-db4f74ae-327c-4617-ae88-cac3f212a479 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:01.658499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4d8b70ac-5d15-43a3-b9cc-413837bee77e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:01.658609 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-db4f74ae-327c-4617-ae88-cac3f212a479 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:01.658911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-db4f74ae-327c-4617-ae88-cac3f212a479 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:02.095496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4d8b70ac-5d15-43a3-b9cc-413837bee77e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:02.100701 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-db4f74ae-327c-4617-ae88-cac3f212a479 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:02.107813 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4d8b70ac-5d15-43a3-b9cc-413837bee77e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.463821 Aug 28 01:19:02.108768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1067/1905] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:01 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:02.115603 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-db4f74ae-327c-4617-ae88-cac3f212a479 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.469920 Aug 28 01:19:02.120550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ae41b928-59f4-42ee-afd0-40e6fc572f76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:02.121495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ae41b928-59f4-42ee-afd0-40e6fc572f76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:02.126321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 839/1906] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:01 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:02.128862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ae41b928-59f4-42ee-afd0-40e6fc572f76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:02.129378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ae41b928-59f4-42ee-afd0-40e6fc572f76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:02.129815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ae41b928-59f4-42ee-afd0-40e6fc572f76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:02.130520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ae41b928-59f4-42ee-afd0-40e6fc572f76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:02.130944 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ae41b928-59f4-42ee-afd0-40e6fc572f76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:02.131372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ae41b928-59f4-42ee-afd0-40e6fc572f76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:02.266768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ae41b928-59f4-42ee-afd0-40e6fc572f76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:02.278055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ae41b928-59f4-42ee-afd0-40e6fc572f76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:02.279481 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ae41b928-59f4-42ee-afd0-40e6fc572f76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:02.280639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ae41b928-59f4-42ee-afd0-40e6fc572f76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.170009 Aug 28 01:19:02.281223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1068/1907] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:02 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:03.134438 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e481b0cb-c716-4bf2-8fc0-452a87c71439 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:03.135263 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e481b0cb-c716-4bf2-8fc0-452a87c71439 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:03.139701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-aa4317ed-bec1-493a-87f6-f6385dcbfade tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:03.140265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-aa4317ed-bec1-493a-87f6-f6385dcbfade tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:03.142464 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e481b0cb-c716-4bf2-8fc0-452a87c71439 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:03.142941 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e481b0cb-c716-4bf2-8fc0-452a87c71439 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:03.143356 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e481b0cb-c716-4bf2-8fc0-452a87c71439 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:03.144051 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e481b0cb-c716-4bf2-8fc0-452a87c71439 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:03.144486 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e481b0cb-c716-4bf2-8fc0-452a87c71439 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:03.144901 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e481b0cb-c716-4bf2-8fc0-452a87c71439 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:03.147408 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa4317ed-bec1-493a-87f6-f6385dcbfade tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:03.147562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa4317ed-bec1-493a-87f6-f6385dcbfade tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:03.147940 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa4317ed-bec1-493a-87f6-f6385dcbfade tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:03.148647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa4317ed-bec1-493a-87f6-f6385dcbfade tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:03.149122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa4317ed-bec1-493a-87f6-f6385dcbfade tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:03.149543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa4317ed-bec1-493a-87f6-f6385dcbfade tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:03.347594 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e481b0cb-c716-4bf2-8fc0-452a87c71439 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:03.352646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-aa4317ed-bec1-493a-87f6-f6385dcbfade tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:03.358737 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e481b0cb-c716-4bf2-8fc0-452a87c71439 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.228225 Aug 28 01:19:03.359330 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 840/1908] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:03 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:03.361338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-aa4317ed-bec1-493a-87f6-f6385dcbfade tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.226297 Aug 28 01:19:03.361981 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1069/1909] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:03 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:19:03.363967 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-118e0526-92d9-4eaa-9659-78e30c656c8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:03.364584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-118e0526-92d9-4eaa-9659-78e30c656c8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:03.370839 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-118e0526-92d9-4eaa-9659-78e30c656c8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:03.371294 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-118e0526-92d9-4eaa-9659-78e30c656c8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:03.371959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-118e0526-92d9-4eaa-9659-78e30c656c8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:03.372729 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-118e0526-92d9-4eaa-9659-78e30c656c8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:03.373103 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-118e0526-92d9-4eaa-9659-78e30c656c8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:03.373552 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-118e0526-92d9-4eaa-9659-78e30c656c8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:03.510684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-118e0526-92d9-4eaa-9659-78e30c656c8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:03.519770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-118e0526-92d9-4eaa-9659-78e30c656c8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:03.521241 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-118e0526-92d9-4eaa-9659-78e30c656c8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:03.522409 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-118e0526-92d9-4eaa-9659-78e30c656c8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.162136 Aug 28 01:19:03.522967 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 841/1910] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:03 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:04.383187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a83de82c-5653-4ef4-a0f9-c353404edecf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:04.383763 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a83de82c-5653-4ef4-a0f9-c353404edecf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:04.384799 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-08d58a66-0f3d-4653-8740-d241d6872bf2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:04.386117 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-08d58a66-0f3d-4653-8740-d241d6872bf2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:04.390866 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a83de82c-5653-4ef4-a0f9-c353404edecf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:04.391350 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a83de82c-5653-4ef4-a0f9-c353404edecf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:04.391775 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a83de82c-5653-4ef4-a0f9-c353404edecf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:04.392475 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a83de82c-5653-4ef4-a0f9-c353404edecf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:04.392913 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a83de82c-5653-4ef4-a0f9-c353404edecf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:04.393081 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08d58a66-0f3d-4653-8740-d241d6872bf2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:04.393369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a83de82c-5653-4ef4-a0f9-c353404edecf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:04.393520 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08d58a66-0f3d-4653-8740-d241d6872bf2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:04.393913 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08d58a66-0f3d-4653-8740-d241d6872bf2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:04.394615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08d58a66-0f3d-4653-8740-d241d6872bf2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:04.395055 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08d58a66-0f3d-4653-8740-d241d6872bf2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:04.395514 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08d58a66-0f3d-4653-8740-d241d6872bf2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:04.586567 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-08d58a66-0f3d-4653-8740-d241d6872bf2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:04.590112 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a83de82c-5653-4ef4-a0f9-c353404edecf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:04.595642 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-08d58a66-0f3d-4653-8740-d241d6872bf2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.216243 Aug 28 01:19:04.599091 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a83de82c-5653-4ef4-a0f9-c353404edecf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.221216 Aug 28 01:19:04.599729 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1070/1911] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:04 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:04.604388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-000391ea-4a54-46fd-ba80-2806277fa77d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:04.605295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-000391ea-4a54-46fd-ba80-2806277fa77d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:04.609527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 842/1912] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:04 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:19:04.612107 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-000391ea-4a54-46fd-ba80-2806277fa77d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:04.612675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-000391ea-4a54-46fd-ba80-2806277fa77d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:04.613236 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-000391ea-4a54-46fd-ba80-2806277fa77d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:04.614011 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-000391ea-4a54-46fd-ba80-2806277fa77d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:04.614529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-000391ea-4a54-46fd-ba80-2806277fa77d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:04.614958 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-000391ea-4a54-46fd-ba80-2806277fa77d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:04.751050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-000391ea-4a54-46fd-ba80-2806277fa77d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:04.763175 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-000391ea-4a54-46fd-ba80-2806277fa77d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:04.765343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-000391ea-4a54-46fd-ba80-2806277fa77d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:04.767059 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-000391ea-4a54-46fd-ba80-2806277fa77d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.165924 Aug 28 01:19:04.767672 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1071/1913] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:04 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:05.621556 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d772784a-24e8-417d-9122-d81239a2c3d2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:05.622467 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d772784a-24e8-417d-9122-d81239a2c3d2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:05.624616 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-081a679c-f213-4ca1-bd57-abec64739411 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:05.625224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-081a679c-f213-4ca1-bd57-abec64739411 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:05.629387 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d772784a-24e8-417d-9122-d81239a2c3d2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:05.629879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d772784a-24e8-417d-9122-d81239a2c3d2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:05.630318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d772784a-24e8-417d-9122-d81239a2c3d2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:05.631027 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d772784a-24e8-417d-9122-d81239a2c3d2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:05.631476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d772784a-24e8-417d-9122-d81239a2c3d2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:05.631735 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-081a679c-f213-4ca1-bd57-abec64739411 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:05.631891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d772784a-24e8-417d-9122-d81239a2c3d2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:05.632224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-081a679c-f213-4ca1-bd57-abec64739411 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:05.632651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-081a679c-f213-4ca1-bd57-abec64739411 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:05.633367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-081a679c-f213-4ca1-bd57-abec64739411 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:05.633928 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-081a679c-f213-4ca1-bd57-abec64739411 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:05.634225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-081a679c-f213-4ca1-bd57-abec64739411 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:05.834796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d772784a-24e8-417d-9122-d81239a2c3d2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:05.840618 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-081a679c-f213-4ca1-bd57-abec64739411 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:05.844176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:19:05.847995 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d772784a-24e8-417d-9122-d81239a2c3d2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.232031 Aug 28 01:19:05.849135 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 843/1914] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:05 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:05.855584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-081a679c-f213-4ca1-bd57-abec64739411 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.235655 Aug 28 01:19:05.856339 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f570b357-7b9b-41e9-8731-cbf21f34c8ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:05.856791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1072/1915] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:05 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:05.857869 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f570b357-7b9b-41e9-8731-cbf21f34c8ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:05.864387 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f570b357-7b9b-41e9-8731-cbf21f34c8ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:05.864876 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f570b357-7b9b-41e9-8731-cbf21f34c8ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:05.865338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f570b357-7b9b-41e9-8731-cbf21f34c8ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:05.866043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f570b357-7b9b-41e9-8731-cbf21f34c8ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:05.866489 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f570b357-7b9b-41e9-8731-cbf21f34c8ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:05.866907 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f570b357-7b9b-41e9-8731-cbf21f34c8ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:06.006340 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f570b357-7b9b-41e9-8731-cbf21f34c8ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:06.016428 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f570b357-7b9b-41e9-8731-cbf21f34c8ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:06.017924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f570b357-7b9b-41e9-8731-cbf21f34c8ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:06.019136 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f570b357-7b9b-41e9-8731-cbf21f34c8ad tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.168768 Aug 28 01:19:06.019701 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 844/1916] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:05 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:06.873554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e0bf9380-a0b3-4707-a73e-a3bc432e4907 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:06.874137 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e0bf9380-a0b3-4707-a73e-a3bc432e4907 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:06.876991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f5267070-ad3b-42dc-af5a-19d4e9e96bd8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:06.877614 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f5267070-ad3b-42dc-af5a-19d4e9e96bd8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:06.880794 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0bf9380-a0b3-4707-a73e-a3bc432e4907 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:06.881297 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0bf9380-a0b3-4707-a73e-a3bc432e4907 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:06.881732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0bf9380-a0b3-4707-a73e-a3bc432e4907 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:06.882365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0bf9380-a0b3-4707-a73e-a3bc432e4907 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:06.882844 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0bf9380-a0b3-4707-a73e-a3bc432e4907 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:06.883256 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0bf9380-a0b3-4707-a73e-a3bc432e4907 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:06.884046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f5267070-ad3b-42dc-af5a-19d4e9e96bd8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:06.884533 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f5267070-ad3b-42dc-af5a-19d4e9e96bd8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:06.884976 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f5267070-ad3b-42dc-af5a-19d4e9e96bd8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:06.885667 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f5267070-ad3b-42dc-af5a-19d4e9e96bd8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:06.886113 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f5267070-ad3b-42dc-af5a-19d4e9e96bd8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:06.886529 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f5267070-ad3b-42dc-af5a-19d4e9e96bd8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:06.890649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:19:07.087983 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e0bf9380-a0b3-4707-a73e-a3bc432e4907 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:07.095630 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f5267070-ad3b-42dc-af5a-19d4e9e96bd8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:07.096692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e0bf9380-a0b3-4707-a73e-a3bc432e4907 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.227384 Aug 28 01:19:07.097307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1073/1917] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:06 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:07.101975 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-47330e31-452a-48f6-b9c4-4c7d2ee82b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:07.102532 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-47330e31-452a-48f6-b9c4-4c7d2ee82b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:07.104495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f5267070-ad3b-42dc-af5a-19d4e9e96bd8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.231272 Aug 28 01:19:07.107436 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 845/1918] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:06 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:19:07.108823 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47330e31-452a-48f6-b9c4-4c7d2ee82b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:07.114117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47330e31-452a-48f6-b9c4-4c7d2ee82b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:07.114117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47330e31-452a-48f6-b9c4-4c7d2ee82b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:07.114117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47330e31-452a-48f6-b9c4-4c7d2ee82b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:07.114117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47330e31-452a-48f6-b9c4-4c7d2ee82b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:07.114545 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47330e31-452a-48f6-b9c4-4c7d2ee82b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:07.254564 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-47330e31-452a-48f6-b9c4-4c7d2ee82b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:07.268169 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-47330e31-452a-48f6-b9c4-4c7d2ee82b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:07.270244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-47330e31-452a-48f6-b9c4-4c7d2ee82b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:07.271903 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-47330e31-452a-48f6-b9c4-4c7d2ee82b0d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.173391 Aug 28 01:19:07.272604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1074/1919] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:07 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:08.119112 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f247347a-aa44-45de-9650-8a1b86582cd7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:08.119679 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f247347a-aa44-45de-9650-8a1b86582cd7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:08.126517 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f247347a-aa44-45de-9650-8a1b86582cd7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:08.127011 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f247347a-aa44-45de-9650-8a1b86582cd7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:08.127443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f247347a-aa44-45de-9650-8a1b86582cd7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:08.128148 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f247347a-aa44-45de-9650-8a1b86582cd7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:08.128587 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f247347a-aa44-45de-9650-8a1b86582cd7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:08.128748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d5939f93-3a0a-46b2-8d9c-a436e1d3950f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:08.129892 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f247347a-aa44-45de-9650-8a1b86582cd7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:08.130005 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d5939f93-3a0a-46b2-8d9c-a436e1d3950f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:08.138213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5939f93-3a0a-46b2-8d9c-a436e1d3950f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:08.138709 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5939f93-3a0a-46b2-8d9c-a436e1d3950f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:08.139133 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5939f93-3a0a-46b2-8d9c-a436e1d3950f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:08.139878 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5939f93-3a0a-46b2-8d9c-a436e1d3950f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:08.140268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5939f93-3a0a-46b2-8d9c-a436e1d3950f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:08.140686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5939f93-3a0a-46b2-8d9c-a436e1d3950f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:08.328761 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f247347a-aa44-45de-9650-8a1b86582cd7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:08.338113 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d5939f93-3a0a-46b2-8d9c-a436e1d3950f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:08.338305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f247347a-aa44-45de-9650-8a1b86582cd7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.223093 Aug 28 01:19:08.338884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 846/1920] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:08 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:08.343445 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c6822e1f-52ba-42d5-a5e6-54ccb9f9a319 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:08.344045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c6822e1f-52ba-42d5-a5e6-54ccb9f9a319 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:08.347626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d5939f93-3a0a-46b2-8d9c-a436e1d3950f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.224493 Aug 28 01:19:08.348192 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1075/1921] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:08 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:08.350186 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6822e1f-52ba-42d5-a5e6-54ccb9f9a319 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:08.350655 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6822e1f-52ba-42d5-a5e6-54ccb9f9a319 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:08.351078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6822e1f-52ba-42d5-a5e6-54ccb9f9a319 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:08.351761 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6822e1f-52ba-42d5-a5e6-54ccb9f9a319 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:08.352203 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6822e1f-52ba-42d5-a5e6-54ccb9f9a319 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:08.352615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6822e1f-52ba-42d5-a5e6-54ccb9f9a319 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:08.484346 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c6822e1f-52ba-42d5-a5e6-54ccb9f9a319 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:08.497548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c6822e1f-52ba-42d5-a5e6-54ccb9f9a319 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:08.498992 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c6822e1f-52ba-42d5-a5e6-54ccb9f9a319 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:08.500184 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c6822e1f-52ba-42d5-a5e6-54ccb9f9a319 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.160369 Aug 28 01:19:08.500739 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 847/1922] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:08 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:09.359783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f6164f01-84be-4197-a133-a94c632407ce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:09.360404 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f6164f01-84be-4197-a133-a94c632407ce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:09.367135 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6164f01-84be-4197-a133-a94c632407ce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:09.367618 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6164f01-84be-4197-a133-a94c632407ce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:09.368047 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6164f01-84be-4197-a133-a94c632407ce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:09.368769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6164f01-84be-4197-a133-a94c632407ce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:09.369086 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a98f8ed9-c5cb-4b69-8139-2e6383adbdb8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:09.369611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6164f01-84be-4197-a133-a94c632407ce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:09.369611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6164f01-84be-4197-a133-a94c632407ce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:09.369820 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a98f8ed9-c5cb-4b69-8139-2e6383adbdb8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:09.376985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a98f8ed9-c5cb-4b69-8139-2e6383adbdb8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:09.377493 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a98f8ed9-c5cb-4b69-8139-2e6383adbdb8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:09.377928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a98f8ed9-c5cb-4b69-8139-2e6383adbdb8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:09.378640 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a98f8ed9-c5cb-4b69-8139-2e6383adbdb8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:09.379095 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a98f8ed9-c5cb-4b69-8139-2e6383adbdb8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:09.379518 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a98f8ed9-c5cb-4b69-8139-2e6383adbdb8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:09.565092 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f6164f01-84be-4197-a133-a94c632407ce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:09.574022 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f6164f01-84be-4197-a133-a94c632407ce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.218104 Aug 28 01:19:09.577023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1076/1923] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:09 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:09.579140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-97b1eea4-3f01-4423-af92-4f6326776855 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:09.579741 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-97b1eea4-3f01-4423-af92-4f6326776855 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:09.585778 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97b1eea4-3f01-4423-af92-4f6326776855 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:09.586262 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97b1eea4-3f01-4423-af92-4f6326776855 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:09.586689 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97b1eea4-3f01-4423-af92-4f6326776855 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:09.587383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97b1eea4-3f01-4423-af92-4f6326776855 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:09.587824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97b1eea4-3f01-4423-af92-4f6326776855 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:09.588242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-97b1eea4-3f01-4423-af92-4f6326776855 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:09.590525 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a98f8ed9-c5cb-4b69-8139-2e6383adbdb8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:09.602231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a98f8ed9-c5cb-4b69-8139-2e6383adbdb8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.237004 Aug 28 01:19:09.602834 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 848/1924] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:09 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:09.731806 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-97b1eea4-3f01-4423-af92-4f6326776855 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:09.740859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-97b1eea4-3f01-4423-af92-4f6326776855 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:09.742314 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-97b1eea4-3f01-4423-af92-4f6326776855 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:09.743458 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-97b1eea4-3f01-4423-af92-4f6326776855 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.167954 Aug 28 01:19:09.744036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1077/1925] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:09 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:10.596126 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cf6330b7-49ea-49f0-9ef0-2e3337727979 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:10.596768 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cf6330b7-49ea-49f0-9ef0-2e3337727979 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:10.604041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cf6330b7-49ea-49f0-9ef0-2e3337727979 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:10.604576 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cf6330b7-49ea-49f0-9ef0-2e3337727979 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:10.605070 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cf6330b7-49ea-49f0-9ef0-2e3337727979 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:10.605762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cf6330b7-49ea-49f0-9ef0-2e3337727979 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:10.606199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cf6330b7-49ea-49f0-9ef0-2e3337727979 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:10.606635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cf6330b7-49ea-49f0-9ef0-2e3337727979 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:10.623995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b04f52e4-e3d5-4432-91cd-3067196495ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:10.624740 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b04f52e4-e3d5-4432-91cd-3067196495ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:10.631641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b04f52e4-e3d5-4432-91cd-3067196495ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:10.632136 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b04f52e4-e3d5-4432-91cd-3067196495ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:10.632566 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b04f52e4-e3d5-4432-91cd-3067196495ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:10.633305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b04f52e4-e3d5-4432-91cd-3067196495ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:10.633759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b04f52e4-e3d5-4432-91cd-3067196495ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:10.634179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b04f52e4-e3d5-4432-91cd-3067196495ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:10.804739 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-cf6330b7-49ea-49f0-9ef0-2e3337727979 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:10.811821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b04f52e4-e3d5-4432-91cd-3067196495ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:10.813691 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cf6330b7-49ea-49f0-9ef0-2e3337727979 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.221832 Aug 28 01:19:10.814322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 849/1926] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:10 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:10.819346 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a562bb7e-6591-4027-a9eb-5cb6506d3fa6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:10.819932 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a562bb7e-6591-4027-a9eb-5cb6506d3fa6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:10.820868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b04f52e4-e3d5-4432-91cd-3067196495ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.200928 Aug 28 01:19:10.824294 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1078/1927] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:10 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:10.825923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a562bb7e-6591-4027-a9eb-5cb6506d3fa6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:10.826410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a562bb7e-6591-4027-a9eb-5cb6506d3fa6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:10.826848 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a562bb7e-6591-4027-a9eb-5cb6506d3fa6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:10.827542 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a562bb7e-6591-4027-a9eb-5cb6506d3fa6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:10.827988 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a562bb7e-6591-4027-a9eb-5cb6506d3fa6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:10.828407 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a562bb7e-6591-4027-a9eb-5cb6506d3fa6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:10.963242 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a562bb7e-6591-4027-a9eb-5cb6506d3fa6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:10.973883 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a562bb7e-6591-4027-a9eb-5cb6506d3fa6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:10.975268 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a562bb7e-6591-4027-a9eb-5cb6506d3fa6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:10.976428 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a562bb7e-6591-4027-a9eb-5cb6506d3fa6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.161193 Aug 28 01:19:10.976988 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 850/1928] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:10 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:11.835387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-60ebb1cc-5d0a-4b03-b993-ed717a3ed6aa tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:11.836111 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-60ebb1cc-5d0a-4b03-b993-ed717a3ed6aa tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:11.842228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-81f46231-4fd9-4bfc-8921-099364da6236 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:11.842828 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-81f46231-4fd9-4bfc-8921-099364da6236 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:11.843429 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60ebb1cc-5d0a-4b03-b993-ed717a3ed6aa tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:11.843913 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60ebb1cc-5d0a-4b03-b993-ed717a3ed6aa tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:11.844351 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60ebb1cc-5d0a-4b03-b993-ed717a3ed6aa tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:11.845069 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60ebb1cc-5d0a-4b03-b993-ed717a3ed6aa tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:11.845493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60ebb1cc-5d0a-4b03-b993-ed717a3ed6aa tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:11.845954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60ebb1cc-5d0a-4b03-b993-ed717a3ed6aa tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:11.849692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-81f46231-4fd9-4bfc-8921-099364da6236 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:11.850176 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-81f46231-4fd9-4bfc-8921-099364da6236 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:11.850605 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-81f46231-4fd9-4bfc-8921-099364da6236 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:11.851303 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-81f46231-4fd9-4bfc-8921-099364da6236 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:11.851750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-81f46231-4fd9-4bfc-8921-099364da6236 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:11.852170 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-81f46231-4fd9-4bfc-8921-099364da6236 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:12.045926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-81f46231-4fd9-4bfc-8921-099364da6236 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:12.052369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-60ebb1cc-5d0a-4b03-b993-ed717a3ed6aa tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:12.060603 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-81f46231-4fd9-4bfc-8921-099364da6236 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.222151 Aug 28 01:19:12.061229 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-60ebb1cc-5d0a-4b03-b993-ed717a3ed6aa tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.229659 Aug 28 01:19:12.061695 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 851/1929] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:11 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:19:12.066217 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-71294728-8a78-46b2-975c-b3dac7ed46ba tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:12.066822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-71294728-8a78-46b2-975c-b3dac7ed46ba tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:12.073144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-71294728-8a78-46b2-975c-b3dac7ed46ba tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:12.073451 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1079/1930] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:11 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:12.073822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-71294728-8a78-46b2-975c-b3dac7ed46ba tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:12.074321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-71294728-8a78-46b2-975c-b3dac7ed46ba tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:12.075111 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-71294728-8a78-46b2-975c-b3dac7ed46ba tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:12.075628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-71294728-8a78-46b2-975c-b3dac7ed46ba tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:12.076155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-71294728-8a78-46b2-975c-b3dac7ed46ba tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:12.257678 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-71294728-8a78-46b2-975c-b3dac7ed46ba tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:12.267087 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-71294728-8a78-46b2-975c-b3dac7ed46ba tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:12.268631 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-71294728-8a78-46b2-975c-b3dac7ed46ba tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:12.269917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-71294728-8a78-46b2-975c-b3dac7ed46ba tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.207390 Aug 28 01:19:12.270631 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 852/1931] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:12 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:13.082958 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d4dda894-6c07-4cad-a1a3-f8b6b7e0e981 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:13.084141 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d4dda894-6c07-4cad-a1a3-f8b6b7e0e981 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:13.087901 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-90acbd7e-e1fc-4956-9d5b-87de5bf53b1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:13.088517 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-90acbd7e-e1fc-4956-9d5b-87de5bf53b1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:13.091179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4dda894-6c07-4cad-a1a3-f8b6b7e0e981 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:13.091658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4dda894-6c07-4cad-a1a3-f8b6b7e0e981 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:13.092081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4dda894-6c07-4cad-a1a3-f8b6b7e0e981 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:13.092870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4dda894-6c07-4cad-a1a3-f8b6b7e0e981 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:13.093356 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4dda894-6c07-4cad-a1a3-f8b6b7e0e981 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:13.093768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4dda894-6c07-4cad-a1a3-f8b6b7e0e981 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:13.094968 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90acbd7e-e1fc-4956-9d5b-87de5bf53b1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:13.095474 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90acbd7e-e1fc-4956-9d5b-87de5bf53b1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:13.095905 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90acbd7e-e1fc-4956-9d5b-87de5bf53b1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:13.096584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90acbd7e-e1fc-4956-9d5b-87de5bf53b1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:13.097058 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90acbd7e-e1fc-4956-9d5b-87de5bf53b1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:13.097477 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90acbd7e-e1fc-4956-9d5b-87de5bf53b1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:13.294945 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d4dda894-6c07-4cad-a1a3-f8b6b7e0e981 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:13.298331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-90acbd7e-e1fc-4956-9d5b-87de5bf53b1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:13.304054 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d4dda894-6c07-4cad-a1a3-f8b6b7e0e981 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.225085 Aug 28 01:19:13.305205 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1080/1932] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:13 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:13.307788 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-90acbd7e-e1fc-4956-9d5b-87de5bf53b1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.223490 Aug 28 01:19:13.310176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-aeffa948-02ee-45ca-9485-6ada32ae910a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:13.311079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-aeffa948-02ee-45ca-9485-6ada32ae910a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:13.318072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aeffa948-02ee-45ca-9485-6ada32ae910a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:13.318795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aeffa948-02ee-45ca-9485-6ada32ae910a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:13.319263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aeffa948-02ee-45ca-9485-6ada32ae910a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:13.319975 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aeffa948-02ee-45ca-9485-6ada32ae910a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:13.320096 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 853/1933] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:13 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:13.320410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aeffa948-02ee-45ca-9485-6ada32ae910a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:13.320827 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aeffa948-02ee-45ca-9485-6ada32ae910a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:13.464633 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-aeffa948-02ee-45ca-9485-6ada32ae910a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:13.477662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-aeffa948-02ee-45ca-9485-6ada32ae910a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:13.479882 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-aeffa948-02ee-45ca-9485-6ada32ae910a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:13.481681 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-aeffa948-02ee-45ca-9485-6ada32ae910a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.175525 Aug 28 01:19:13.482423 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1081/1934] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:13 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:14.292747 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9c6ab8df-6a44-4ec6-a940-59980171621e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:14.293334 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9c6ab8df-6a44-4ec6-a940-59980171621e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:14.301214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9c6ab8df-6a44-4ec6-a940-59980171621e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:14.301693 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9c6ab8df-6a44-4ec6-a940-59980171621e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:14.302110 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9c6ab8df-6a44-4ec6-a940-59980171621e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:14.302845 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9c6ab8df-6a44-4ec6-a940-59980171621e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:14.303287 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9c6ab8df-6a44-4ec6-a940-59980171621e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:14.303704 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9c6ab8df-6a44-4ec6-a940-59980171621e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:14.325962 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d20aa170-80ff-4920-bff6-d0ae2d1ee88f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:14.326788 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d20aa170-80ff-4920-bff6-d0ae2d1ee88f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:14.330165 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9c6ab8df-6a44-4ec6-a940-59980171621e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Going to try to terminate instance {{(pid=120524) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 28 01:19:14.333790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d20aa170-80ff-4920-bff6-d0ae2d1ee88f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:14.334278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d20aa170-80ff-4920-bff6-d0ae2d1ee88f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:14.334708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d20aa170-80ff-4920-bff6-d0ae2d1ee88f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:14.335392 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d20aa170-80ff-4920-bff6-d0ae2d1ee88f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:14.335839 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d20aa170-80ff-4920-bff6-d0ae2d1ee88f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:14.336273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d20aa170-80ff-4920-bff6-d0ae2d1ee88f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:14.415802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9c6ab8df-6a44-4ec6-a940-59980171621e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "DELETE /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 204 len: 0 microversion: 2.1 time: 0.127873 Aug 28 01:19:14.416562 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 854/1935] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 01:19:14 2022] DELETE /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 28 01:19:14.423782 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6ad8ecaa-62a6-477d-86e0-ecb8477ed97b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:14.423782 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6ad8ecaa-62a6-477d-86e0-ecb8477ed97b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:14.430643 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6ad8ecaa-62a6-477d-86e0-ecb8477ed97b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:14.431285 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6ad8ecaa-62a6-477d-86e0-ecb8477ed97b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:14.431727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6ad8ecaa-62a6-477d-86e0-ecb8477ed97b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:14.432445 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6ad8ecaa-62a6-477d-86e0-ecb8477ed97b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:14.432898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6ad8ecaa-62a6-477d-86e0-ecb8477ed97b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:14.433408 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6ad8ecaa-62a6-477d-86e0-ecb8477ed97b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:14.488861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d20aa170-80ff-4920-bff6-d0ae2d1ee88f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:14.498099 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d20aa170-80ff-4920-bff6-d0ae2d1ee88f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.176062 Aug 28 01:19:14.498896 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1082/1936] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:14 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:14.504909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-153331d5-4779-4036-8663-d93700e533b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:14.505778 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-153331d5-4779-4036-8663-d93700e533b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:14.512168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-153331d5-4779-4036-8663-d93700e533b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:14.512627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-153331d5-4779-4036-8663-d93700e533b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:14.513369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-153331d5-4779-4036-8663-d93700e533b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:14.514117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-153331d5-4779-4036-8663-d93700e533b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:14.514568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-153331d5-4779-4036-8663-d93700e533b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:14.515004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-153331d5-4779-4036-8663-d93700e533b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:14.610295 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6ad8ecaa-62a6-477d-86e0-ecb8477ed97b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:14.619506 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6ad8ecaa-62a6-477d-86e0-ecb8477ed97b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.201668 Aug 28 01:19:14.620096 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 855/1937] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:14 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:14.624996 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8185dd2d-ba39-4436-838b-9e8c7e688cc7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:14.625646 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8185dd2d-ba39-4436-838b-9e8c7e688cc7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:14.634627 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8185dd2d-ba39-4436-838b-9e8c7e688cc7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:14.635116 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8185dd2d-ba39-4436-838b-9e8c7e688cc7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:14.635544 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8185dd2d-ba39-4436-838b-9e8c7e688cc7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:14.636236 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8185dd2d-ba39-4436-838b-9e8c7e688cc7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:14.636688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8185dd2d-ba39-4436-838b-9e8c7e688cc7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:14.637134 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8185dd2d-ba39-4436-838b-9e8c7e688cc7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:14.673323 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-153331d5-4779-4036-8663-d93700e533b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:14.681763 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-153331d5-4779-4036-8663-d93700e533b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1720 microversion: 2.1 time: 0.181458 Aug 28 01:19:14.682358 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1083/1938] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:14 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1720 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:14.775397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8185dd2d-ba39-4436-838b-9e8c7e688cc7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:14.788381 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8185dd2d-ba39-4436-838b-9e8c7e688cc7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:14.790559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8185dd2d-ba39-4436-838b-9e8c7e688cc7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:14.792283 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8185dd2d-ba39-4436-838b-9e8c7e688cc7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.170951 Aug 28 01:19:14.793053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 856/1939] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:14 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:15.520796 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-511dbf24-cc6e-42c0-bc76-293b6a98adb1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:15.521470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-511dbf24-cc6e-42c0-bc76-293b6a98adb1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:15.528466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-511dbf24-cc6e-42c0-bc76-293b6a98adb1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:15.528948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-511dbf24-cc6e-42c0-bc76-293b6a98adb1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:15.529415 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-511dbf24-cc6e-42c0-bc76-293b6a98adb1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:15.530102 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-511dbf24-cc6e-42c0-bc76-293b6a98adb1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:15.530621 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-511dbf24-cc6e-42c0-bc76-293b6a98adb1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:15.531041 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-511dbf24-cc6e-42c0-bc76-293b6a98adb1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:15.640999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-47446ac9-6192-4f06-863c-7ed7fe4adba5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:15.641926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-47446ac9-6192-4f06-863c-7ed7fe4adba5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:15.648832 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-47446ac9-6192-4f06-863c-7ed7fe4adba5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:15.649480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-47446ac9-6192-4f06-863c-7ed7fe4adba5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:15.650007 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-47446ac9-6192-4f06-863c-7ed7fe4adba5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:15.650715 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-47446ac9-6192-4f06-863c-7ed7fe4adba5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:15.651161 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-47446ac9-6192-4f06-863c-7ed7fe4adba5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:15.651581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-47446ac9-6192-4f06-863c-7ed7fe4adba5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:15.675706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-511dbf24-cc6e-42c0-bc76-293b6a98adb1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:15.684324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-511dbf24-cc6e-42c0-bc76-293b6a98adb1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.167563 Aug 28 01:19:15.684892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1084/1940] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:15 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:15.703810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-95cb5e91-5c79-4e2d-9e9d-78c2cf4f9582 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:15.704422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-95cb5e91-5c79-4e2d-9e9d-78c2cf4f9582 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:15.710471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-95cb5e91-5c79-4e2d-9e9d-78c2cf4f9582 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:15.710954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-95cb5e91-5c79-4e2d-9e9d-78c2cf4f9582 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:15.711403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-95cb5e91-5c79-4e2d-9e9d-78c2cf4f9582 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:15.712075 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-95cb5e91-5c79-4e2d-9e9d-78c2cf4f9582 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:15.712527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-95cb5e91-5c79-4e2d-9e9d-78c2cf4f9582 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:15.712951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-95cb5e91-5c79-4e2d-9e9d-78c2cf4f9582 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:15.827745 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-47446ac9-6192-4f06-863c-7ed7fe4adba5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:15.837282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-47446ac9-6192-4f06-863c-7ed7fe4adba5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.200200 Aug 28 01:19:15.837847 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 857/1941] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:15 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:15.843151 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f29911f3-968e-4be2-8070-f007bed813f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:15.844342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f29911f3-968e-4be2-8070-f007bed813f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:15.851209 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f29911f3-968e-4be2-8070-f007bed813f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:15.852247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f29911f3-968e-4be2-8070-f007bed813f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:15.852747 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f29911f3-968e-4be2-8070-f007bed813f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:15.853547 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f29911f3-968e-4be2-8070-f007bed813f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:15.854064 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f29911f3-968e-4be2-8070-f007bed813f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:15.854549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f29911f3-968e-4be2-8070-f007bed813f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:15.955140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-95cb5e91-5c79-4e2d-9e9d-78c2cf4f9582 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:15.966188 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-95cb5e91-5c79-4e2d-9e9d-78c2cf4f9582 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1720 microversion: 2.1 time: 0.265957 Aug 28 01:19:15.966863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1085/1942] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:15 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1720 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:19:16.125819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f29911f3-968e-4be2-8070-f007bed813f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:16.135580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f29911f3-968e-4be2-8070-f007bed813f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:16.137075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f29911f3-968e-4be2-8070-f007bed813f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:16.138550 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f29911f3-968e-4be2-8070-f007bed813f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.299498 Aug 28 01:19:16.139112 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 858/1943] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:15 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:16.706773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-709a2eca-3eff-4bf0-9217-c340062b95b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:16.707519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-709a2eca-3eff-4bf0-9217-c340062b95b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:16.715004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-709a2eca-3eff-4bf0-9217-c340062b95b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:16.715460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-709a2eca-3eff-4bf0-9217-c340062b95b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:16.715920 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-709a2eca-3eff-4bf0-9217-c340062b95b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:16.716624 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-709a2eca-3eff-4bf0-9217-c340062b95b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:16.717140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-709a2eca-3eff-4bf0-9217-c340062b95b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:16.717618 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-709a2eca-3eff-4bf0-9217-c340062b95b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:16.859857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f7f12e97-6219-4d0a-8fcb-dcdfe1b708a0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:16.860587 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f7f12e97-6219-4d0a-8fcb-dcdfe1b708a0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:16.867329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f7f12e97-6219-4d0a-8fcb-dcdfe1b708a0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:16.867910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f7f12e97-6219-4d0a-8fcb-dcdfe1b708a0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:16.868457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f7f12e97-6219-4d0a-8fcb-dcdfe1b708a0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:16.870030 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f7f12e97-6219-4d0a-8fcb-dcdfe1b708a0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:16.870847 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f7f12e97-6219-4d0a-8fcb-dcdfe1b708a0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:16.871261 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f7f12e97-6219-4d0a-8fcb-dcdfe1b708a0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:16.988759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-709a2eca-3eff-4bf0-9217-c340062b95b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:16.999472 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-709a2eca-3eff-4bf0-9217-c340062b95b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.297203 Aug 28 01:19:17.029954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1086/1944] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:16 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:17.038751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ef2ea2b0-9e21-4430-969f-95bd3bf11ffa tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:17.041339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ef2ea2b0-9e21-4430-969f-95bd3bf11ffa tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:17.049835 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef2ea2b0-9e21-4430-969f-95bd3bf11ffa tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:17.050346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef2ea2b0-9e21-4430-969f-95bd3bf11ffa tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:17.050791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef2ea2b0-9e21-4430-969f-95bd3bf11ffa tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:17.051504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef2ea2b0-9e21-4430-969f-95bd3bf11ffa tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:17.052035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef2ea2b0-9e21-4430-969f-95bd3bf11ffa tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:17.052436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef2ea2b0-9e21-4430-969f-95bd3bf11ffa tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:17.082937 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f7f12e97-6219-4d0a-8fcb-dcdfe1b708a0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:17.093828 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f7f12e97-6219-4d0a-8fcb-dcdfe1b708a0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.237857 Aug 28 01:19:17.094899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 859/1945] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:16 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:17.139018 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ef2ea2b0-9e21-4430-969f-95bd3bf11ffa tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:17.149046 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ef2ea2b0-9e21-4430-969f-95bd3bf11ffa tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.117399 Aug 28 01:19:17.149896 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1087/1946] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:17 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:17.163755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ef3dc216-0e71-49c6-8285-8004e8756584 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:17.164733 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ef3dc216-0e71-49c6-8285-8004e8756584 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:17.171117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef3dc216-0e71-49c6-8285-8004e8756584 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:17.171595 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef3dc216-0e71-49c6-8285-8004e8756584 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:17.172019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef3dc216-0e71-49c6-8285-8004e8756584 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:17.172917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef3dc216-0e71-49c6-8285-8004e8756584 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:17.173382 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef3dc216-0e71-49c6-8285-8004e8756584 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:17.173834 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ef3dc216-0e71-49c6-8285-8004e8756584 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:17.246988 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-fcd18902-3e6a-41f8-ac62-4a078fcb28db req-0d01cb30-73f7-4cfd-9d8e-b7700490fc09 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e434e4f9-9d0e-46f2-bea3-395bae33fc92", "name": "network-vif-deleted", "tag": "cb80f35b-738d-4eef-8768-99836091e5a6"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:19:17.258527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-fcd18902-3e6a-41f8-ac62-4a078fcb28db req-0d01cb30-73f7-4cfd-9d8e-b7700490fc09 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:17.258996 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-fcd18902-3e6a-41f8-ac62-4a078fcb28db req-0d01cb30-73f7-4cfd-9d8e-b7700490fc09 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:17.259678 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-fcd18902-3e6a-41f8-ac62-4a078fcb28db req-0d01cb30-73f7-4cfd-9d8e-b7700490fc09 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:17.266821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:19:17.279246 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-fcd18902-3e6a-41f8-ac62-4a078fcb28db req-0d01cb30-73f7-4cfd-9d8e-b7700490fc09 service nova] Creating event network-vif-deleted:cb80f35b-738d-4eef-8768-99836091e5a6 for instance e434e4f9-9d0e-46f2-bea3-395bae33fc92 on hv-cinder-maste Aug 28 01:19:17.284779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-fcd18902-3e6a-41f8-ac62-4a078fcb28db req-0d01cb30-73f7-4cfd-9d8e-b7700490fc09 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.188492 Aug 28 01:19:17.285805 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 860/1947] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:19:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:19:17.322307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ef3dc216-0e71-49c6-8285-8004e8756584 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:17.331186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ef3dc216-0e71-49c6-8285-8004e8756584 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:17.332647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ef3dc216-0e71-49c6-8285-8004e8756584 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:17.333821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ef3dc216-0e71-49c6-8285-8004e8756584 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.176058 Aug 28 01:19:17.334414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1088/1948] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:17 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 01:19:18.031087 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1910d792-ab75-410a-9a9f-cdc70d59edbd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:18.031940 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1910d792-ab75-410a-9a9f-cdc70d59edbd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:18.040825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1910d792-ab75-410a-9a9f-cdc70d59edbd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:18.041377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1910d792-ab75-410a-9a9f-cdc70d59edbd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:18.041842 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1910d792-ab75-410a-9a9f-cdc70d59edbd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:18.042926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1910d792-ab75-410a-9a9f-cdc70d59edbd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:18.043495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1910d792-ab75-410a-9a9f-cdc70d59edbd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:18.043926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1910d792-ab75-410a-9a9f-cdc70d59edbd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:18.120606 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f1f1fba3-44dd-4932-875c-8ca6913bc60d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:18.122412 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f1f1fba3-44dd-4932-875c-8ca6913bc60d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:18.131512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f1f1fba3-44dd-4932-875c-8ca6913bc60d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:18.132379 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f1f1fba3-44dd-4932-875c-8ca6913bc60d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:18.133073 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f1f1fba3-44dd-4932-875c-8ca6913bc60d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:18.134206 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f1f1fba3-44dd-4932-875c-8ca6913bc60d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:18.134760 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f1f1fba3-44dd-4932-875c-8ca6913bc60d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:18.135190 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f1f1fba3-44dd-4932-875c-8ca6913bc60d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:18.152173 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:19:18.237498 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1910d792-ab75-410a-9a9f-cdc70d59edbd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:18.247198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1910d792-ab75-410a-9a9f-cdc70d59edbd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.221506 Aug 28 01:19:18.247842 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 861/1949] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:18 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:18.252685 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2a8ebcca-9409-4441-b6d9-b5eb6437c97c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:18.253318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2a8ebcca-9409-4441-b6d9-b5eb6437c97c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:18.261086 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2a8ebcca-9409-4441-b6d9-b5eb6437c97c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:18.261794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2a8ebcca-9409-4441-b6d9-b5eb6437c97c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:18.262234 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2a8ebcca-9409-4441-b6d9-b5eb6437c97c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:18.263179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2a8ebcca-9409-4441-b6d9-b5eb6437c97c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:18.263631 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2a8ebcca-9409-4441-b6d9-b5eb6437c97c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:18.264034 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2a8ebcca-9409-4441-b6d9-b5eb6437c97c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:18.338869 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f1f1fba3-44dd-4932-875c-8ca6913bc60d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:18.344297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2a8ebcca-9409-4441-b6d9-b5eb6437c97c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:18.348300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f1f1fba3-44dd-4932-875c-8ca6913bc60d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.232919 Aug 28 01:19:18.348938 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1089/1950] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:18 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:18.354023 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2a8ebcca-9409-4441-b6d9-b5eb6437c97c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.105114 Aug 28 01:19:18.354633 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 862/1951] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:18 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:18.364858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-03c5333c-3acd-46ac-87e2-14fa8aa261a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:18.364858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-03c5333c-3acd-46ac-87e2-14fa8aa261a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:18.364858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03c5333c-3acd-46ac-87e2-14fa8aa261a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:18.364858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03c5333c-3acd-46ac-87e2-14fa8aa261a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:18.365353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03c5333c-3acd-46ac-87e2-14fa8aa261a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:18.365353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03c5333c-3acd-46ac-87e2-14fa8aa261a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:18.365353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03c5333c-3acd-46ac-87e2-14fa8aa261a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:18.365635 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03c5333c-3acd-46ac-87e2-14fa8aa261a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:18.532262 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-03c5333c-3acd-46ac-87e2-14fa8aa261a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:18.541076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-03c5333c-3acd-46ac-87e2-14fa8aa261a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:18.542488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-03c5333c-3acd-46ac-87e2-14fa8aa261a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:18.543621 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-03c5333c-3acd-46ac-87e2-14fa8aa261a3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.191265 Aug 28 01:19:18.544167 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1090/1952] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:18 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 01:19:19.270688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-45bad98e-12d4-4a7a-ba05-8e4b2440c73d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:19.271581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-45bad98e-12d4-4a7a-ba05-8e4b2440c73d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:19.281565 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-45bad98e-12d4-4a7a-ba05-8e4b2440c73d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:19.282105 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-45bad98e-12d4-4a7a-ba05-8e4b2440c73d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:19.282701 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-45bad98e-12d4-4a7a-ba05-8e4b2440c73d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:19.283894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-45bad98e-12d4-4a7a-ba05-8e4b2440c73d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:19.284697 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-45bad98e-12d4-4a7a-ba05-8e4b2440c73d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:19.285355 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-45bad98e-12d4-4a7a-ba05-8e4b2440c73d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:19.371184 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-49ab1b24-7515-42ee-acea-e31203a684a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:19.371714 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-49ab1b24-7515-42ee-acea-e31203a684a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:19.378474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49ab1b24-7515-42ee-acea-e31203a684a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:19.378953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49ab1b24-7515-42ee-acea-e31203a684a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:19.379367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49ab1b24-7515-42ee-acea-e31203a684a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:19.380302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49ab1b24-7515-42ee-acea-e31203a684a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:19.380893 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49ab1b24-7515-42ee-acea-e31203a684a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:19.381376 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49ab1b24-7515-42ee-acea-e31203a684a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:19.442049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-45bad98e-12d4-4a7a-ba05-8e4b2440c73d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:19.451525 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-45bad98e-12d4-4a7a-ba05-8e4b2440c73d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.186108 Aug 28 01:19:19.452098 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 863/1953] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:19 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:19.456630 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0bf221d5-ca8b-4b4b-9b7f-6b0093e7c135 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:19.457251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0bf221d5-ca8b-4b4b-9b7f-6b0093e7c135 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:19.463083 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0bf221d5-ca8b-4b4b-9b7f-6b0093e7c135 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:19.463566 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0bf221d5-ca8b-4b4b-9b7f-6b0093e7c135 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:19.463989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0bf221d5-ca8b-4b4b-9b7f-6b0093e7c135 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:19.464688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0bf221d5-ca8b-4b4b-9b7f-6b0093e7c135 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:19.465168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0bf221d5-ca8b-4b4b-9b7f-6b0093e7c135 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:19.465674 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0bf221d5-ca8b-4b4b-9b7f-6b0093e7c135 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:19.528992 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0bf221d5-ca8b-4b4b-9b7f-6b0093e7c135 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:19.537946 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0bf221d5-ca8b-4b4b-9b7f-6b0093e7c135 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.084827 Aug 28 01:19:19.538515 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 864/1954] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:19 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:19.551523 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-49ab1b24-7515-42ee-acea-e31203a684a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:19.559866 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-49ab1b24-7515-42ee-acea-e31203a684a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.193527 Aug 28 01:19:19.560436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1091/1955] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:19 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:19.572737 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-674d6f65-7021-4de9-9400-ee8fa5dfb309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:19.573487 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-674d6f65-7021-4de9-9400-ee8fa5dfb309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:19.579637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-674d6f65-7021-4de9-9400-ee8fa5dfb309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:19.580111 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-674d6f65-7021-4de9-9400-ee8fa5dfb309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:19.580535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-674d6f65-7021-4de9-9400-ee8fa5dfb309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:19.581239 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-674d6f65-7021-4de9-9400-ee8fa5dfb309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:19.581715 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-674d6f65-7021-4de9-9400-ee8fa5dfb309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:19.582127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-674d6f65-7021-4de9-9400-ee8fa5dfb309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:19.722329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-674d6f65-7021-4de9-9400-ee8fa5dfb309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:19.731453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-674d6f65-7021-4de9-9400-ee8fa5dfb309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:19.732908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-674d6f65-7021-4de9-9400-ee8fa5dfb309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:19.734105 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-674d6f65-7021-4de9-9400-ee8fa5dfb309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.165741 Aug 28 01:19:19.734666 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 865/1956] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:19 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 01:19:20.474764 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f3c211bc-13b6-4dd7-b664-6e0ee8597c4c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:20.475409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f3c211bc-13b6-4dd7-b664-6e0ee8597c4c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:20.483526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3c211bc-13b6-4dd7-b664-6e0ee8597c4c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:20.484030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3c211bc-13b6-4dd7-b664-6e0ee8597c4c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:20.484466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3c211bc-13b6-4dd7-b664-6e0ee8597c4c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:20.485220 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3c211bc-13b6-4dd7-b664-6e0ee8597c4c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:20.485676 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3c211bc-13b6-4dd7-b664-6e0ee8597c4c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:20.486109 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f3c211bc-13b6-4dd7-b664-6e0ee8597c4c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:20.560432 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-aa94fbe5-8aa0-455b-a1f9-49aa1b47da58 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:20.561465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-aa94fbe5-8aa0-455b-a1f9-49aa1b47da58 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:20.574143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aa94fbe5-8aa0-455b-a1f9-49aa1b47da58 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:20.574605 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aa94fbe5-8aa0-455b-a1f9-49aa1b47da58 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:20.575090 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aa94fbe5-8aa0-455b-a1f9-49aa1b47da58 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:20.575768 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aa94fbe5-8aa0-455b-a1f9-49aa1b47da58 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:20.576198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aa94fbe5-8aa0-455b-a1f9-49aa1b47da58 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:20.576612 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aa94fbe5-8aa0-455b-a1f9-49aa1b47da58 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:20.635383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f3c211bc-13b6-4dd7-b664-6e0ee8597c4c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:20.643966 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f3c211bc-13b6-4dd7-b664-6e0ee8597c4c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.173742 Aug 28 01:19:20.644525 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1092/1957] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:20 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:20.651681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-aa94fbe5-8aa0-455b-a1f9-49aa1b47da58 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:20.653911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3f656485-c645-4506-af67-f2397d2fa2e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:20.654622 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3f656485-c645-4506-af67-f2397d2fa2e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:20.661563 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-aa94fbe5-8aa0-455b-a1f9-49aa1b47da58 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.105126 Aug 28 01:19:20.661884 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3f656485-c645-4506-af67-f2397d2fa2e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:20.662017 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 866/1958] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:20 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:20.662122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3f656485-c645-4506-af67-f2397d2fa2e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:20.662323 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3f656485-c645-4506-af67-f2397d2fa2e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:20.663027 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3f656485-c645-4506-af67-f2397d2fa2e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:20.663480 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3f656485-c645-4506-af67-f2397d2fa2e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:20.663903 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3f656485-c645-4506-af67-f2397d2fa2e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:20.757942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e1e033b3-0f00-4a3e-aa17-7da02b903b8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:20.758622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e1e033b3-0f00-4a3e-aa17-7da02b903b8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:20.765188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1e033b3-0f00-4a3e-aa17-7da02b903b8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:20.765772 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1e033b3-0f00-4a3e-aa17-7da02b903b8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:20.766269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1e033b3-0f00-4a3e-aa17-7da02b903b8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:20.767052 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1e033b3-0f00-4a3e-aa17-7da02b903b8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:20.767524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1e033b3-0f00-4a3e-aa17-7da02b903b8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:20.767938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1e033b3-0f00-4a3e-aa17-7da02b903b8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:20.807694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3f656485-c645-4506-af67-f2397d2fa2e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:20.816550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3f656485-c645-4506-af67-f2397d2fa2e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.166392 Aug 28 01:19:20.817184 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1093/1959] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:20 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:20.905221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e1e033b3-0f00-4a3e-aa17-7da02b903b8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:20.915135 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e1e033b3-0f00-4a3e-aa17-7da02b903b8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:20.916559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e1e033b3-0f00-4a3e-aa17-7da02b903b8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:20.917768 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e1e033b3-0f00-4a3e-aa17-7da02b903b8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.163983 Aug 28 01:19:20.925138 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 867/1960] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:20 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:21.668780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-926b6cc5-8ecc-4a6f-ae22-1b5330aedfb8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:21.669703 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-926b6cc5-8ecc-4a6f-ae22-1b5330aedfb8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:21.677234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-926b6cc5-8ecc-4a6f-ae22-1b5330aedfb8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:21.677718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-926b6cc5-8ecc-4a6f-ae22-1b5330aedfb8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:21.678151 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-926b6cc5-8ecc-4a6f-ae22-1b5330aedfb8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:21.678840 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-926b6cc5-8ecc-4a6f-ae22-1b5330aedfb8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:21.679287 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-926b6cc5-8ecc-4a6f-ae22-1b5330aedfb8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:21.679711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-926b6cc5-8ecc-4a6f-ae22-1b5330aedfb8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:21.681620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-28633b05-d400-4d2e-853a-515dea1505cd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:21.682663 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-28633b05-d400-4d2e-853a-515dea1505cd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:21.689760 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-28633b05-d400-4d2e-853a-515dea1505cd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:21.690220 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-28633b05-d400-4d2e-853a-515dea1505cd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:21.690643 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-28633b05-d400-4d2e-853a-515dea1505cd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:21.691341 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-28633b05-d400-4d2e-853a-515dea1505cd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:21.691888 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-28633b05-d400-4d2e-853a-515dea1505cd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:21.692308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-28633b05-d400-4d2e-853a-515dea1505cd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:21.774644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-28633b05-d400-4d2e-853a-515dea1505cd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:21.783271 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-28633b05-d400-4d2e-853a-515dea1505cd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.105541 Aug 28 01:19:21.783856 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 868/1961] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:21 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:21.838626 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b5153fb1-fc7a-4a4d-bd25-b3d62b60938f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:21.839237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b5153fb1-fc7a-4a4d-bd25-b3d62b60938f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:21.839560 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-926b6cc5-8ecc-4a6f-ae22-1b5330aedfb8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:21.845529 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5153fb1-fc7a-4a4d-bd25-b3d62b60938f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:21.846020 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5153fb1-fc7a-4a4d-bd25-b3d62b60938f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:21.846451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5153fb1-fc7a-4a4d-bd25-b3d62b60938f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:21.847149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5153fb1-fc7a-4a4d-bd25-b3d62b60938f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:21.847597 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5153fb1-fc7a-4a4d-bd25-b3d62b60938f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:21.848015 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5153fb1-fc7a-4a4d-bd25-b3d62b60938f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:21.848713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-926b6cc5-8ecc-4a6f-ae22-1b5330aedfb8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.186162 Aug 28 01:19:21.849326 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1094/1962] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:21 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:21.941271 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b08c420a-79e2-4f06-b3d5-8fd3e3c32bda tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:21.942154 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b08c420a-79e2-4f06-b3d5-8fd3e3c32bda tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:21.949235 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b08c420a-79e2-4f06-b3d5-8fd3e3c32bda tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:21.950074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b08c420a-79e2-4f06-b3d5-8fd3e3c32bda tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:21.950763 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b08c420a-79e2-4f06-b3d5-8fd3e3c32bda tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:21.951532 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b08c420a-79e2-4f06-b3d5-8fd3e3c32bda tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:21.952014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b08c420a-79e2-4f06-b3d5-8fd3e3c32bda tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:21.952500 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b08c420a-79e2-4f06-b3d5-8fd3e3c32bda tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:21.983614 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b5153fb1-fc7a-4a4d-bd25-b3d62b60938f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:21.993202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b5153fb1-fc7a-4a4d-bd25-b3d62b60938f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.158469 Aug 28 01:19:21.993994 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 869/1963] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:21 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:19:22.339742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b08c420a-79e2-4f06-b3d5-8fd3e3c32bda tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:22.353078 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b08c420a-79e2-4f06-b3d5-8fd3e3c32bda tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:22.355165 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b08c420a-79e2-4f06-b3d5-8fd3e3c32bda tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:22.357050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b08c420a-79e2-4f06-b3d5-8fd3e3c32bda tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.419224 Aug 28 01:19:22.358020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1095/1964] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:21 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:22.806596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e1df23b9-e7e9-406a-9a34-c6b0a18d979d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:22.807253 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e1df23b9-e7e9-406a-9a34-c6b0a18d979d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:22.813959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1df23b9-e7e9-406a-9a34-c6b0a18d979d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:22.814443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1df23b9-e7e9-406a-9a34-c6b0a18d979d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:22.814907 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1df23b9-e7e9-406a-9a34-c6b0a18d979d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:22.815597 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1df23b9-e7e9-406a-9a34-c6b0a18d979d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:22.816049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1df23b9-e7e9-406a-9a34-c6b0a18d979d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:22.816470 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1df23b9-e7e9-406a-9a34-c6b0a18d979d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:22.870611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2750ead1-34f0-4778-b36c-3bc380e31212 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:22.871263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2750ead1-34f0-4778-b36c-3bc380e31212 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:22.879125 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2750ead1-34f0-4778-b36c-3bc380e31212 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:22.879614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2750ead1-34f0-4778-b36c-3bc380e31212 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:22.880036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2750ead1-34f0-4778-b36c-3bc380e31212 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:22.880980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2750ead1-34f0-4778-b36c-3bc380e31212 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:22.881433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2750ead1-34f0-4778-b36c-3bc380e31212 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:22.881874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2750ead1-34f0-4778-b36c-3bc380e31212 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:22.885508 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e1df23b9-e7e9-406a-9a34-c6b0a18d979d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:22.893681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e1df23b9-e7e9-406a-9a34-c6b0a18d979d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.091256 Aug 28 01:19:22.894254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 870/1965] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:22 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:23.014661 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-feb719fa-9bca-48c9-8964-8be4d320f726 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:23.015270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-feb719fa-9bca-48c9-8964-8be4d320f726 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:23.019231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2750ead1-34f0-4778-b36c-3bc380e31212 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:23.021775 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-feb719fa-9bca-48c9-8964-8be4d320f726 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:23.022268 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-feb719fa-9bca-48c9-8964-8be4d320f726 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:23.022715 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-feb719fa-9bca-48c9-8964-8be4d320f726 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:23.023418 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-feb719fa-9bca-48c9-8964-8be4d320f726 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:23.023871 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-feb719fa-9bca-48c9-8964-8be4d320f726 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:23.024276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-feb719fa-9bca-48c9-8964-8be4d320f726 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:23.028001 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2750ead1-34f0-4778-b36c-3bc380e31212 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.161258 Aug 28 01:19:23.028812 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1096/1966] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:22 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:23.158593 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-feb719fa-9bca-48c9-8964-8be4d320f726 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:23.167191 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-feb719fa-9bca-48c9-8964-8be4d320f726 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.156587 Aug 28 01:19:23.167769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 871/1967] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:23 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:23.380932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-857d6991-dc70-41a2-b064-aeec6085d8f6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:23.381489 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-857d6991-dc70-41a2-b064-aeec6085d8f6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:23.387863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-857d6991-dc70-41a2-b064-aeec6085d8f6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:23.388343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-857d6991-dc70-41a2-b064-aeec6085d8f6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:23.388768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-857d6991-dc70-41a2-b064-aeec6085d8f6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:23.389702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-857d6991-dc70-41a2-b064-aeec6085d8f6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:23.390142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-857d6991-dc70-41a2-b064-aeec6085d8f6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:23.390615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-857d6991-dc70-41a2-b064-aeec6085d8f6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:23.543516 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-857d6991-dc70-41a2-b064-aeec6085d8f6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:23.557254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-857d6991-dc70-41a2-b064-aeec6085d8f6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:23.559438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-857d6991-dc70-41a2-b064-aeec6085d8f6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:23.561209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-857d6991-dc70-41a2-b064-aeec6085d8f6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.184315 Aug 28 01:19:23.561769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1097/1968] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:23 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:23.915342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6386dfbe-3f23-4ed6-8af3-9f812f1160a1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:23.916010 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6386dfbe-3f23-4ed6-8af3-9f812f1160a1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:23.923100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6386dfbe-3f23-4ed6-8af3-9f812f1160a1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:23.923590 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6386dfbe-3f23-4ed6-8af3-9f812f1160a1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:23.924000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6386dfbe-3f23-4ed6-8af3-9f812f1160a1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:23.924711 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6386dfbe-3f23-4ed6-8af3-9f812f1160a1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:23.925206 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6386dfbe-3f23-4ed6-8af3-9f812f1160a1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:23.925613 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6386dfbe-3f23-4ed6-8af3-9f812f1160a1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:23.994820 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6386dfbe-3f23-4ed6-8af3-9f812f1160a1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:24.003887 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6386dfbe-3f23-4ed6-8af3-9f812f1160a1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.092402 Aug 28 01:19:24.004476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 872/1969] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:23 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:24.049903 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-758c3bfc-93ba-4db8-bfd2-5e306348c788 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:24.050571 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-758c3bfc-93ba-4db8-bfd2-5e306348c788 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:24.057670 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-758c3bfc-93ba-4db8-bfd2-5e306348c788 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:24.058160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-758c3bfc-93ba-4db8-bfd2-5e306348c788 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:24.058585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-758c3bfc-93ba-4db8-bfd2-5e306348c788 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:24.059274 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-758c3bfc-93ba-4db8-bfd2-5e306348c788 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:24.059718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-758c3bfc-93ba-4db8-bfd2-5e306348c788 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:24.060133 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-758c3bfc-93ba-4db8-bfd2-5e306348c788 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:24.196228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0672dcd9-b351-41cb-86a2-8a1c39d6858c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:24.196810 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0672dcd9-b351-41cb-86a2-8a1c39d6858c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:24.203581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-758c3bfc-93ba-4db8-bfd2-5e306348c788 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:24.205733 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0672dcd9-b351-41cb-86a2-8a1c39d6858c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:24.206437 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0672dcd9-b351-41cb-86a2-8a1c39d6858c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:24.206878 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0672dcd9-b351-41cb-86a2-8a1c39d6858c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:24.207778 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0672dcd9-b351-41cb-86a2-8a1c39d6858c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:24.208272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0672dcd9-b351-41cb-86a2-8a1c39d6858c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:24.208737 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0672dcd9-b351-41cb-86a2-8a1c39d6858c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:24.212575 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-758c3bfc-93ba-4db8-bfd2-5e306348c788 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.166497 Aug 28 01:19:24.213193 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1098/1970] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:24 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:24.352473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0672dcd9-b351-41cb-86a2-8a1c39d6858c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:24.361539 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0672dcd9-b351-41cb-86a2-8a1c39d6858c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.169571 Aug 28 01:19:24.362162 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 873/1971] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:24 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:19:24.585361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5f6dbcf9-89e9-413b-91e7-6f07377d8eb5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:24.585838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5f6dbcf9-89e9-413b-91e7-6f07377d8eb5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:24.592461 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f6dbcf9-89e9-413b-91e7-6f07377d8eb5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:24.592939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f6dbcf9-89e9-413b-91e7-6f07377d8eb5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:24.593404 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f6dbcf9-89e9-413b-91e7-6f07377d8eb5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:24.594083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f6dbcf9-89e9-413b-91e7-6f07377d8eb5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:24.594525 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f6dbcf9-89e9-413b-91e7-6f07377d8eb5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:24.594938 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f6dbcf9-89e9-413b-91e7-6f07377d8eb5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:24.758327 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5f6dbcf9-89e9-413b-91e7-6f07377d8eb5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:24.767513 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5f6dbcf9-89e9-413b-91e7-6f07377d8eb5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:24.768984 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5f6dbcf9-89e9-413b-91e7-6f07377d8eb5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:24.770102 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5f6dbcf9-89e9-413b-91e7-6f07377d8eb5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.188817 Aug 28 01:19:24.773017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1099/1972] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:24 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:25.026421 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8fc513f3-1833-4c5e-b3d4-5b34f4dbefa3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:25.027105 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8fc513f3-1833-4c5e-b3d4-5b34f4dbefa3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:25.034316 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fc513f3-1833-4c5e-b3d4-5b34f4dbefa3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:25.034802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fc513f3-1833-4c5e-b3d4-5b34f4dbefa3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:25.035236 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fc513f3-1833-4c5e-b3d4-5b34f4dbefa3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:25.035937 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fc513f3-1833-4c5e-b3d4-5b34f4dbefa3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:25.036388 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fc513f3-1833-4c5e-b3d4-5b34f4dbefa3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:25.036806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fc513f3-1833-4c5e-b3d4-5b34f4dbefa3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:25.108829 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8fc513f3-1833-4c5e-b3d4-5b34f4dbefa3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:25.117811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8fc513f3-1833-4c5e-b3d4-5b34f4dbefa3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.095282 Aug 28 01:19:25.118626 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 874/1973] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:25 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:25.235061 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c2f3336f-164b-42b4-8c8c-c8e45a6cd14a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:25.235653 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c2f3336f-164b-42b4-8c8c-c8e45a6cd14a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:25.242452 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2f3336f-164b-42b4-8c8c-c8e45a6cd14a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:25.242944 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2f3336f-164b-42b4-8c8c-c8e45a6cd14a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:25.243373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2f3336f-164b-42b4-8c8c-c8e45a6cd14a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:25.244207 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2f3336f-164b-42b4-8c8c-c8e45a6cd14a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:25.244652 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2f3336f-164b-42b4-8c8c-c8e45a6cd14a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:25.245103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2f3336f-164b-42b4-8c8c-c8e45a6cd14a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:25.385135 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c2f3336f-164b-42b4-8c8c-c8e45a6cd14a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:25.386274 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f685d61c-763c-42cb-b78f-48ad1d2a5209 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:25.386889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f685d61c-763c-42cb-b78f-48ad1d2a5209 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:25.393199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f685d61c-763c-42cb-b78f-48ad1d2a5209 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:25.393679 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f685d61c-763c-42cb-b78f-48ad1d2a5209 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:25.393909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c2f3336f-164b-42b4-8c8c-c8e45a6cd14a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.163173 Aug 28 01:19:25.394107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f685d61c-763c-42cb-b78f-48ad1d2a5209 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:25.394501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1100/1974] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:25 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:25.394887 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f685d61c-763c-42cb-b78f-48ad1d2a5209 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:25.395415 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f685d61c-763c-42cb-b78f-48ad1d2a5209 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:25.395824 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f685d61c-763c-42cb-b78f-48ad1d2a5209 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:25.535547 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f685d61c-763c-42cb-b78f-48ad1d2a5209 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:25.547276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f685d61c-763c-42cb-b78f-48ad1d2a5209 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.165229 Aug 28 01:19:25.547834 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 875/1975] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:25 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:19:25.795307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-595c6692-3854-4989-b119-2085c53172b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:25.796182 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-595c6692-3854-4989-b119-2085c53172b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:25.804505 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-595c6692-3854-4989-b119-2085c53172b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:25.805155 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-595c6692-3854-4989-b119-2085c53172b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:25.805752 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-595c6692-3854-4989-b119-2085c53172b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:25.806645 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-595c6692-3854-4989-b119-2085c53172b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:25.807215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-595c6692-3854-4989-b119-2085c53172b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:25.807762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-595c6692-3854-4989-b119-2085c53172b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:25.951974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-595c6692-3854-4989-b119-2085c53172b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:25.962194 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-595c6692-3854-4989-b119-2085c53172b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:25.963631 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-595c6692-3854-4989-b119-2085c53172b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:25.964757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-595c6692-3854-4989-b119-2085c53172b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.174986 Aug 28 01:19:25.965389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1101/1976] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:25 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:26.141761 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-95b4ade8-fa29-4662-97a3-482879d043e5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:26.142416 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-95b4ade8-fa29-4662-97a3-482879d043e5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:26.149352 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95b4ade8-fa29-4662-97a3-482879d043e5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:26.149852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95b4ade8-fa29-4662-97a3-482879d043e5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:26.150266 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95b4ade8-fa29-4662-97a3-482879d043e5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:26.150972 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95b4ade8-fa29-4662-97a3-482879d043e5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:26.151419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95b4ade8-fa29-4662-97a3-482879d043e5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:26.151838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95b4ade8-fa29-4662-97a3-482879d043e5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:26.222861 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-95b4ade8-fa29-4662-97a3-482879d043e5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:26.231469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-95b4ade8-fa29-4662-97a3-482879d043e5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.095145 Aug 28 01:19:26.232276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 876/1977] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:26 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:26.416484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fc61eace-3717-4406-abd3-a90d86ddb592 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:26.417149 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fc61eace-3717-4406-abd3-a90d86ddb592 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:26.424042 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc61eace-3717-4406-abd3-a90d86ddb592 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:26.424520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc61eace-3717-4406-abd3-a90d86ddb592 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:26.424943 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc61eace-3717-4406-abd3-a90d86ddb592 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:26.425750 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc61eace-3717-4406-abd3-a90d86ddb592 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:26.426183 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc61eace-3717-4406-abd3-a90d86ddb592 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:26.426591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc61eace-3717-4406-abd3-a90d86ddb592 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:26.566019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fc61eace-3717-4406-abd3-a90d86ddb592 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:26.568374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fb44334b-d4e6-41c0-8acb-aad8af0ca889 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:26.569020 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fb44334b-d4e6-41c0-8acb-aad8af0ca889 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:26.575218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fb44334b-d4e6-41c0-8acb-aad8af0ca889 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:26.575705 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fb44334b-d4e6-41c0-8acb-aad8af0ca889 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:26.575822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fc61eace-3717-4406-abd3-a90d86ddb592 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.163271 Aug 28 01:19:26.576163 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fb44334b-d4e6-41c0-8acb-aad8af0ca889 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:26.576994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1102/1978] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:26 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:26.577179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fb44334b-d4e6-41c0-8acb-aad8af0ca889 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:26.577372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fb44334b-d4e6-41c0-8acb-aad8af0ca889 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:26.577819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fb44334b-d4e6-41c0-8acb-aad8af0ca889 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:26.595299 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:19:26.714237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fb44334b-d4e6-41c0-8acb-aad8af0ca889 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:26.723410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fb44334b-d4e6-41c0-8acb-aad8af0ca889 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.159181 Aug 28 01:19:26.723993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 877/1979] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:26 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:26.989050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4b38ec0a-8139-4f29-9eaf-2b763e2401ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:26.989551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4b38ec0a-8139-4f29-9eaf-2b763e2401ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:26.995839 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b38ec0a-8139-4f29-9eaf-2b763e2401ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:26.996345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b38ec0a-8139-4f29-9eaf-2b763e2401ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:26.996776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b38ec0a-8139-4f29-9eaf-2b763e2401ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:26.997497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b38ec0a-8139-4f29-9eaf-2b763e2401ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:26.997940 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b38ec0a-8139-4f29-9eaf-2b763e2401ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:26.998356 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b38ec0a-8139-4f29-9eaf-2b763e2401ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:27.145283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4b38ec0a-8139-4f29-9eaf-2b763e2401ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:27.154002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4b38ec0a-8139-4f29-9eaf-2b763e2401ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:27.155414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4b38ec0a-8139-4f29-9eaf-2b763e2401ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:27.156569 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4b38ec0a-8139-4f29-9eaf-2b763e2401ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.171782 Aug 28 01:19:27.157267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1103/1980] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:26 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:27.253559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-869b66dd-4113-4995-bce7-396f49951ef9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:27.254135 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-869b66dd-4113-4995-bce7-396f49951ef9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:27.260678 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-869b66dd-4113-4995-bce7-396f49951ef9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:27.261201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-869b66dd-4113-4995-bce7-396f49951ef9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:27.261651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-869b66dd-4113-4995-bce7-396f49951ef9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:27.262335 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-869b66dd-4113-4995-bce7-396f49951ef9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:27.262804 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-869b66dd-4113-4995-bce7-396f49951ef9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:27.263212 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-869b66dd-4113-4995-bce7-396f49951ef9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:27.328013 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-869b66dd-4113-4995-bce7-396f49951ef9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:27.336614 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-869b66dd-4113-4995-bce7-396f49951ef9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.087118 Aug 28 01:19:27.337244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 878/1981] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:27 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:27.600582 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3bf867b7-e9bd-4977-ae78-fbbe66ecb7e6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:27.601184 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3bf867b7-e9bd-4977-ae78-fbbe66ecb7e6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:27.610732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bf867b7-e9bd-4977-ae78-fbbe66ecb7e6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:27.611206 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bf867b7-e9bd-4977-ae78-fbbe66ecb7e6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:27.611634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bf867b7-e9bd-4977-ae78-fbbe66ecb7e6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:27.612313 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bf867b7-e9bd-4977-ae78-fbbe66ecb7e6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:27.612765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bf867b7-e9bd-4977-ae78-fbbe66ecb7e6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:27.613199 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bf867b7-e9bd-4977-ae78-fbbe66ecb7e6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:27.745476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-31fe57c0-2621-4828-9b15-61d1e3e816fc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:27.746094 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-31fe57c0-2621-4828-9b15-61d1e3e816fc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:27.752534 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31fe57c0-2621-4828-9b15-61d1e3e816fc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:27.753056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31fe57c0-2621-4828-9b15-61d1e3e816fc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:27.753481 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31fe57c0-2621-4828-9b15-61d1e3e816fc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:27.754189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31fe57c0-2621-4828-9b15-61d1e3e816fc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:27.754619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31fe57c0-2621-4828-9b15-61d1e3e816fc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:27.755039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31fe57c0-2621-4828-9b15-61d1e3e816fc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:27.951723 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-31fe57c0-2621-4828-9b15-61d1e3e816fc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:27.955007 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3bf867b7-e9bd-4977-ae78-fbbe66ecb7e6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:27.960392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-31fe57c0-2621-4828-9b15-61d1e3e816fc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.218850 Aug 28 01:19:27.961315 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 879/1982] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:27 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:27.963830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3bf867b7-e9bd-4977-ae78-fbbe66ecb7e6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.369261 Aug 28 01:19:27.964385 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1104/1983] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:27 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:28.182393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2581ff54-a957-4bc6-9e13-031d849c70d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:28.183029 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2581ff54-a957-4bc6-9e13-031d849c70d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:28.190661 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2581ff54-a957-4bc6-9e13-031d849c70d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:28.191145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2581ff54-a957-4bc6-9e13-031d849c70d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:28.191567 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2581ff54-a957-4bc6-9e13-031d849c70d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:28.192254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2581ff54-a957-4bc6-9e13-031d849c70d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:28.193111 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2581ff54-a957-4bc6-9e13-031d849c70d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:28.193525 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2581ff54-a957-4bc6-9e13-031d849c70d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:28.344662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2581ff54-a957-4bc6-9e13-031d849c70d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:28.354902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2581ff54-a957-4bc6-9e13-031d849c70d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:28.356416 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2581ff54-a957-4bc6-9e13-031d849c70d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:28.357765 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2581ff54-a957-4bc6-9e13-031d849c70d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.181350 Aug 28 01:19:28.358265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-85b078bb-91e0-473d-a470-98b44613f609 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:28.358713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 880/1984] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:28 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:28.358883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-85b078bb-91e0-473d-a470-98b44613f609 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:28.365451 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85b078bb-91e0-473d-a470-98b44613f609 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:28.365976 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85b078bb-91e0-473d-a470-98b44613f609 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:28.366408 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85b078bb-91e0-473d-a470-98b44613f609 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:28.367128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85b078bb-91e0-473d-a470-98b44613f609 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:28.367593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85b078bb-91e0-473d-a470-98b44613f609 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:28.368131 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85b078bb-91e0-473d-a470-98b44613f609 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:28.434784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-85b078bb-91e0-473d-a470-98b44613f609 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:28.443693 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-85b078bb-91e0-473d-a470-98b44613f609 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089229 Aug 28 01:19:28.444260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1105/1985] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:28 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:28.982996 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-51aa065c-2d2a-4f77-98c3-2d250f9b4689 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:28.983694 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-51aa065c-2d2a-4f77-98c3-2d250f9b4689 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:28.985253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e0d7c22a-f9ca-4278-a39c-734aac179354 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:28.986020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e0d7c22a-f9ca-4278-a39c-734aac179354 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:28.990930 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-51aa065c-2d2a-4f77-98c3-2d250f9b4689 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:28.991393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-51aa065c-2d2a-4f77-98c3-2d250f9b4689 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:28.991850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-51aa065c-2d2a-4f77-98c3-2d250f9b4689 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:28.992558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-51aa065c-2d2a-4f77-98c3-2d250f9b4689 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:28.992856 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0d7c22a-f9ca-4278-a39c-734aac179354 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:28.993055 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-51aa065c-2d2a-4f77-98c3-2d250f9b4689 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:28.993384 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0d7c22a-f9ca-4278-a39c-734aac179354 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:28.993506 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-51aa065c-2d2a-4f77-98c3-2d250f9b4689 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:28.993831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0d7c22a-f9ca-4278-a39c-734aac179354 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:28.994536 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0d7c22a-f9ca-4278-a39c-734aac179354 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:28.994988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0d7c22a-f9ca-4278-a39c-734aac179354 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:28.995413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0d7c22a-f9ca-4278-a39c-734aac179354 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:28.998234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:19:29.211321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-51aa065c-2d2a-4f77-98c3-2d250f9b4689 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:29.218739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e0d7c22a-f9ca-4278-a39c-734aac179354 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:29.220263 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-51aa065c-2d2a-4f77-98c3-2d250f9b4689 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.242017 Aug 28 01:19:29.220852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 881/1986] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:28 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:29.228142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e0d7c22a-f9ca-4278-a39c-734aac179354 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.246711 Aug 28 01:19:29.228714 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1106/1987] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:28 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:29.383781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7d605101-0fd1-4084-855f-9477af60f635 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:29.384866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7d605101-0fd1-4084-855f-9477af60f635 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:29.393067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7d605101-0fd1-4084-855f-9477af60f635 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:29.393554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7d605101-0fd1-4084-855f-9477af60f635 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:29.393980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7d605101-0fd1-4084-855f-9477af60f635 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:29.394682 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7d605101-0fd1-4084-855f-9477af60f635 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:29.395126 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7d605101-0fd1-4084-855f-9477af60f635 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:29.395543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7d605101-0fd1-4084-855f-9477af60f635 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:29.465362 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1b2e2abd-70fe-43b9-aa9d-60f7139ba0b4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:29.466233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1b2e2abd-70fe-43b9-aa9d-60f7139ba0b4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:29.473398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b2e2abd-70fe-43b9-aa9d-60f7139ba0b4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:29.473920 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b2e2abd-70fe-43b9-aa9d-60f7139ba0b4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:29.474340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b2e2abd-70fe-43b9-aa9d-60f7139ba0b4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:29.475066 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b2e2abd-70fe-43b9-aa9d-60f7139ba0b4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:29.475529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b2e2abd-70fe-43b9-aa9d-60f7139ba0b4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:29.475961 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1b2e2abd-70fe-43b9-aa9d-60f7139ba0b4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:29.547375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1b2e2abd-70fe-43b9-aa9d-60f7139ba0b4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:29.556359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1b2e2abd-70fe-43b9-aa9d-60f7139ba0b4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.094923 Aug 28 01:19:29.557287 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1107/1988] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:29 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:29.578378 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7d605101-0fd1-4084-855f-9477af60f635 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:29.587829 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7d605101-0fd1-4084-855f-9477af60f635 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:29.589446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7d605101-0fd1-4084-855f-9477af60f635 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:29.590657 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7d605101-0fd1-4084-855f-9477af60f635 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.212595 Aug 28 01:19:29.591341 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 882/1989] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:29 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 01:19:30.244626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e15b90cb-736d-49eb-91f4-074ee156479c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:30.245392 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e15b90cb-736d-49eb-91f4-074ee156479c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:30.251742 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1fbcfc80-98cb-4eaa-af01-8d8354a04663 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:30.252423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1fbcfc80-98cb-4eaa-af01-8d8354a04663 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:30.253862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e15b90cb-736d-49eb-91f4-074ee156479c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:30.254486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e15b90cb-736d-49eb-91f4-074ee156479c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:30.254921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e15b90cb-736d-49eb-91f4-074ee156479c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:30.255710 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e15b90cb-736d-49eb-91f4-074ee156479c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:30.256177 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e15b90cb-736d-49eb-91f4-074ee156479c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:30.256596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e15b90cb-736d-49eb-91f4-074ee156479c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:30.258680 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1fbcfc80-98cb-4eaa-af01-8d8354a04663 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:30.259138 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1fbcfc80-98cb-4eaa-af01-8d8354a04663 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:30.259580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1fbcfc80-98cb-4eaa-af01-8d8354a04663 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:30.260733 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1fbcfc80-98cb-4eaa-af01-8d8354a04663 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:30.261213 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1fbcfc80-98cb-4eaa-af01-8d8354a04663 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:30.261634 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1fbcfc80-98cb-4eaa-af01-8d8354a04663 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:30.451030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e15b90cb-736d-49eb-91f4-074ee156479c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:30.455118 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1fbcfc80-98cb-4eaa-af01-8d8354a04663 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:30.460066 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e15b90cb-736d-49eb-91f4-074ee156479c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.220968 Aug 28 01:19:30.460998 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1108/1990] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:30 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:30.463767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1fbcfc80-98cb-4eaa-af01-8d8354a04663 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.217167 Aug 28 01:19:30.464336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 883/1991] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:30 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:19:30.586217 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3a307818-5762-4a4b-8a5c-fb0baa711a65 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:30.586859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3a307818-5762-4a4b-8a5c-fb0baa711a65 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:30.593826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a307818-5762-4a4b-8a5c-fb0baa711a65 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:30.594344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a307818-5762-4a4b-8a5c-fb0baa711a65 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:30.594789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a307818-5762-4a4b-8a5c-fb0baa711a65 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:30.595508 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a307818-5762-4a4b-8a5c-fb0baa711a65 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:30.596028 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a307818-5762-4a4b-8a5c-fb0baa711a65 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:30.596479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a307818-5762-4a4b-8a5c-fb0baa711a65 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:30.615410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4addd84b-eb78-4b79-9994-abcc7c5946cc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:30.615935 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4addd84b-eb78-4b79-9994-abcc7c5946cc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:30.617825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:19:30.623539 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4addd84b-eb78-4b79-9994-abcc7c5946cc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:30.624020 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4addd84b-eb78-4b79-9994-abcc7c5946cc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:30.624442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4addd84b-eb78-4b79-9994-abcc7c5946cc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:30.625150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4addd84b-eb78-4b79-9994-abcc7c5946cc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:30.625585 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4addd84b-eb78-4b79-9994-abcc7c5946cc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:30.626005 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4addd84b-eb78-4b79-9994-abcc7c5946cc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:30.662599 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3a307818-5762-4a4b-8a5c-fb0baa711a65 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:30.671211 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3a307818-5762-4a4b-8a5c-fb0baa711a65 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089060 Aug 28 01:19:30.671870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1109/1992] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:30 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:30.766889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4addd84b-eb78-4b79-9994-abcc7c5946cc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:30.776182 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4addd84b-eb78-4b79-9994-abcc7c5946cc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:30.777651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4addd84b-eb78-4b79-9994-abcc7c5946cc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:30.778817 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4addd84b-eb78-4b79-9994-abcc7c5946cc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.167683 Aug 28 01:19:30.779409 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 884/1993] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:30 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:31.482946 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-54ecabc2-58b6-4149-ad18-1cce5bbcbd94 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:31.483582 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-54ecabc2-58b6-4149-ad18-1cce5bbcbd94 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:31.487219 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d0a75a7b-2bf2-400a-89ad-33543390fc15 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:31.487845 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d0a75a7b-2bf2-400a-89ad-33543390fc15 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:31.490615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54ecabc2-58b6-4149-ad18-1cce5bbcbd94 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:31.491140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54ecabc2-58b6-4149-ad18-1cce5bbcbd94 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:31.491559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54ecabc2-58b6-4149-ad18-1cce5bbcbd94 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:31.492312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54ecabc2-58b6-4149-ad18-1cce5bbcbd94 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:31.492753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54ecabc2-58b6-4149-ad18-1cce5bbcbd94 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:31.493226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54ecabc2-58b6-4149-ad18-1cce5bbcbd94 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:31.494616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0a75a7b-2bf2-400a-89ad-33543390fc15 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:31.495110 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0a75a7b-2bf2-400a-89ad-33543390fc15 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:31.495687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0a75a7b-2bf2-400a-89ad-33543390fc15 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:31.496429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0a75a7b-2bf2-400a-89ad-33543390fc15 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:31.496883 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0a75a7b-2bf2-400a-89ad-33543390fc15 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:31.497361 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0a75a7b-2bf2-400a-89ad-33543390fc15 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:31.696176 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d0a75a7b-2bf2-400a-89ad-33543390fc15 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:31.699517 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-54ecabc2-58b6-4149-ad18-1cce5bbcbd94 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:31.704736 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d0a75a7b-2bf2-400a-89ad-33543390fc15 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.221668 Aug 28 01:19:31.705609 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 885/1994] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:31 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:31.710053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-67255c46-8e62-49f3-9bff-3936f778d97b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:31.710649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-67255c46-8e62-49f3-9bff-3936f778d97b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:31.711724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-54ecabc2-58b6-4149-ad18-1cce5bbcbd94 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.232724 Aug 28 01:19:31.717695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1110/1995] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:31 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:31.717935 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67255c46-8e62-49f3-9bff-3936f778d97b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:31.717935 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67255c46-8e62-49f3-9bff-3936f778d97b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:31.717935 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67255c46-8e62-49f3-9bff-3936f778d97b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:31.718620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67255c46-8e62-49f3-9bff-3936f778d97b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:31.719080 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67255c46-8e62-49f3-9bff-3936f778d97b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:31.719497 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67255c46-8e62-49f3-9bff-3936f778d97b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:31.788545 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-67255c46-8e62-49f3-9bff-3936f778d97b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:31.797063 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-67255c46-8e62-49f3-9bff-3936f778d97b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.090522 Aug 28 01:19:31.797639 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 886/1996] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:31 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:31.801898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-56619894-f272-40cd-a53d-49f82fbfbcbf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:31.802507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-56619894-f272-40cd-a53d-49f82fbfbcbf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:31.804396 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=88 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:19:31.810292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56619894-f272-40cd-a53d-49f82fbfbcbf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:31.810779 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56619894-f272-40cd-a53d-49f82fbfbcbf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:31.811214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56619894-f272-40cd-a53d-49f82fbfbcbf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:31.811916 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56619894-f272-40cd-a53d-49f82fbfbcbf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:31.812372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56619894-f272-40cd-a53d-49f82fbfbcbf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:31.812788 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56619894-f272-40cd-a53d-49f82fbfbcbf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:31.951806 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-56619894-f272-40cd-a53d-49f82fbfbcbf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:31.961234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-56619894-f272-40cd-a53d-49f82fbfbcbf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:31.962665 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-56619894-f272-40cd-a53d-49f82fbfbcbf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:31.963829 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-56619894-f272-40cd-a53d-49f82fbfbcbf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.165952 Aug 28 01:19:31.964626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1111/1997] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:31 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 01:19:32.726806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-eb22fcd4-17be-43c2-9569-72168c56df52 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:32.728180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-eb22fcd4-17be-43c2-9569-72168c56df52 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:32.733525 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2a81f9cd-315f-4cf2-95f0-ff7c720bf9a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:32.734551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2a81f9cd-315f-4cf2-95f0-ff7c720bf9a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:32.734724 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb22fcd4-17be-43c2-9569-72168c56df52 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:32.735097 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb22fcd4-17be-43c2-9569-72168c56df52 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:32.736109 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb22fcd4-17be-43c2-9569-72168c56df52 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:32.736336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb22fcd4-17be-43c2-9569-72168c56df52 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:32.736839 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb22fcd4-17be-43c2-9569-72168c56df52 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:32.737281 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb22fcd4-17be-43c2-9569-72168c56df52 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:32.742393 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a81f9cd-315f-4cf2-95f0-ff7c720bf9a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:32.742878 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a81f9cd-315f-4cf2-95f0-ff7c720bf9a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:32.743312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a81f9cd-315f-4cf2-95f0-ff7c720bf9a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:32.743979 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a81f9cd-315f-4cf2-95f0-ff7c720bf9a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:32.744439 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a81f9cd-315f-4cf2-95f0-ff7c720bf9a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:32.744842 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a81f9cd-315f-4cf2-95f0-ff7c720bf9a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:32.940065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-eb22fcd4-17be-43c2-9569-72168c56df52 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:32.943483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2a81f9cd-315f-4cf2-95f0-ff7c720bf9a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:32.949498 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-eb22fcd4-17be-43c2-9569-72168c56df52 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.226614 Aug 28 01:19:32.950122 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 887/1998] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:32 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:32.955240 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ebd92546-cb24-4db6-8b9c-ebc04cde48c6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:32.955845 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ebd92546-cb24-4db6-8b9c-ebc04cde48c6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:32.962466 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ebd92546-cb24-4db6-8b9c-ebc04cde48c6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:32.962946 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ebd92546-cb24-4db6-8b9c-ebc04cde48c6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:32.963372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ebd92546-cb24-4db6-8b9c-ebc04cde48c6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:32.964067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ebd92546-cb24-4db6-8b9c-ebc04cde48c6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:32.964512 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ebd92546-cb24-4db6-8b9c-ebc04cde48c6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:32.964926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ebd92546-cb24-4db6-8b9c-ebc04cde48c6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:32.967822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2a81f9cd-315f-4cf2-95f0-ff7c720bf9a2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.238288 Aug 28 01:19:32.973682 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1112/1999] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:32 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:32.990089 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fabc5325-4ead-401d-b742-3325bf0c6593 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:32.990577 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fabc5325-4ead-401d-b742-3325bf0c6593 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:32.997118 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fabc5325-4ead-401d-b742-3325bf0c6593 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:32.997602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fabc5325-4ead-401d-b742-3325bf0c6593 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:32.998034 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fabc5325-4ead-401d-b742-3325bf0c6593 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:32.998786 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fabc5325-4ead-401d-b742-3325bf0c6593 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:32.999255 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fabc5325-4ead-401d-b742-3325bf0c6593 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:32.999672 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fabc5325-4ead-401d-b742-3325bf0c6593 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:33.051757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ebd92546-cb24-4db6-8b9c-ebc04cde48c6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:33.061772 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ebd92546-cb24-4db6-8b9c-ebc04cde48c6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.110453 Aug 28 01:19:33.062377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 888/2000] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:32 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:33.155838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fabc5325-4ead-401d-b742-3325bf0c6593 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:33.165563 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fabc5325-4ead-401d-b742-3325bf0c6593 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:33.167051 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fabc5325-4ead-401d-b742-3325bf0c6593 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:33.168278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fabc5325-4ead-401d-b742-3325bf0c6593 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.185066 Aug 28 01:19:33.168846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1113/2001] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:32 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:33.979408 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4bf5a267-4d95-4c88-957f-886462468f23 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:33.980049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4bf5a267-4d95-4c88-957f-886462468f23 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:33.987760 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4bf5a267-4d95-4c88-957f-886462468f23 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:33.988247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4bf5a267-4d95-4c88-957f-886462468f23 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:33.988676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4bf5a267-4d95-4c88-957f-886462468f23 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:33.989419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4bf5a267-4d95-4c88-957f-886462468f23 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:33.989866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4bf5a267-4d95-4c88-957f-886462468f23 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:33.990588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4bf5a267-4d95-4c88-957f-886462468f23 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:33.991853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-86c2152e-7bb4-4f59-a412-f45527df0556 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:33.992620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-86c2152e-7bb4-4f59-a412-f45527df0556 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:33.999573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86c2152e-7bb4-4f59-a412-f45527df0556 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:34.000060 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86c2152e-7bb4-4f59-a412-f45527df0556 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:34.000489 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86c2152e-7bb4-4f59-a412-f45527df0556 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:34.001240 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86c2152e-7bb4-4f59-a412-f45527df0556 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:34.001704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86c2152e-7bb4-4f59-a412-f45527df0556 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:34.002224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-86c2152e-7bb4-4f59-a412-f45527df0556 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:34.194061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4bf5a267-4d95-4c88-957f-886462468f23 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:34.203303 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4bf5a267-4d95-4c88-957f-886462468f23 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.227938 Aug 28 01:19:34.203913 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 889/2002] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:33 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:34.213518 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-86c2152e-7bb4-4f59-a412-f45527df0556 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:34.213821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f216efa1-a3cf-4921-a644-4fd3ba72bfe3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:34.213821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f216efa1-a3cf-4921-a644-4fd3ba72bfe3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:34.216531 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f216efa1-a3cf-4921-a644-4fd3ba72bfe3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:34.217028 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f216efa1-a3cf-4921-a644-4fd3ba72bfe3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:34.217467 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f216efa1-a3cf-4921-a644-4fd3ba72bfe3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:34.218185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f216efa1-a3cf-4921-a644-4fd3ba72bfe3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:34.218636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f216efa1-a3cf-4921-a644-4fd3ba72bfe3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:34.219056 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f216efa1-a3cf-4921-a644-4fd3ba72bfe3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:34.219765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-86c2152e-7bb4-4f59-a412-f45527df0556 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.231679 Aug 28 01:19:34.220377 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1114/2003] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:33 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:34.225664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3990856d-4f86-48aa-8a96-9543186ee1ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:34.226263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3990856d-4f86-48aa-8a96-9543186ee1ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:34.233029 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3990856d-4f86-48aa-8a96-9543186ee1ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:34.233443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3990856d-4f86-48aa-8a96-9543186ee1ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:34.233810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3990856d-4f86-48aa-8a96-9543186ee1ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:34.234626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3990856d-4f86-48aa-8a96-9543186ee1ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:34.235111 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3990856d-4f86-48aa-8a96-9543186ee1ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:34.236077 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3990856d-4f86-48aa-8a96-9543186ee1ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:34.287499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f216efa1-a3cf-4921-a644-4fd3ba72bfe3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:34.299695 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f216efa1-a3cf-4921-a644-4fd3ba72bfe3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.094655 Aug 28 01:19:34.300448 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 890/2004] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:34 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:34.395695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3990856d-4f86-48aa-8a96-9543186ee1ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:34.405264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3990856d-4f86-48aa-8a96-9543186ee1ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:34.406730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3990856d-4f86-48aa-8a96-9543186ee1ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:34.408020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3990856d-4f86-48aa-8a96-9543186ee1ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.186616 Aug 28 01:19:34.408588 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1115/2005] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:34 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:35.224327 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e6005605-493f-45c7-b2b8-7a4107031bfb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:35.225205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e6005605-493f-45c7-b2b8-7a4107031bfb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:35.231970 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6005605-493f-45c7-b2b8-7a4107031bfb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:35.232447 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6005605-493f-45c7-b2b8-7a4107031bfb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:35.232877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6005605-493f-45c7-b2b8-7a4107031bfb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:35.233758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6005605-493f-45c7-b2b8-7a4107031bfb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:35.234045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6005605-493f-45c7-b2b8-7a4107031bfb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:35.234491 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6005605-493f-45c7-b2b8-7a4107031bfb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:35.241903 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9bfcb97a-0d50-46f7-85e8-35d662e3dcf0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:35.242474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9bfcb97a-0d50-46f7-85e8-35d662e3dcf0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:35.249677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9bfcb97a-0d50-46f7-85e8-35d662e3dcf0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:35.250167 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9bfcb97a-0d50-46f7-85e8-35d662e3dcf0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:35.250605 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9bfcb97a-0d50-46f7-85e8-35d662e3dcf0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:35.251326 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9bfcb97a-0d50-46f7-85e8-35d662e3dcf0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:35.251786 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9bfcb97a-0d50-46f7-85e8-35d662e3dcf0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:35.252208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9bfcb97a-0d50-46f7-85e8-35d662e3dcf0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:35.438485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e6005605-493f-45c7-b2b8-7a4107031bfb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:35.443173 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9bfcb97a-0d50-46f7-85e8-35d662e3dcf0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:35.448043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e6005605-493f-45c7-b2b8-7a4107031bfb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.227600 Aug 28 01:19:35.448632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 891/2006] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:35 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:35.451687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9bfcb97a-0d50-46f7-85e8-35d662e3dcf0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.213707 Aug 28 01:19:35.460988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1116/2007] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:35 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:35.461235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-865f2639-498b-4b43-a0d6-9527443bab32 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:35.461235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-865f2639-498b-4b43-a0d6-9527443bab32 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:35.461422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4fa162c9-3049-4af5-8cfc-1983e72200f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:35.461422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4fa162c9-3049-4af5-8cfc-1983e72200f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:35.461600 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-865f2639-498b-4b43-a0d6-9527443bab32 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:35.461600 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-865f2639-498b-4b43-a0d6-9527443bab32 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:35.461600 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-865f2639-498b-4b43-a0d6-9527443bab32 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:35.462062 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-865f2639-498b-4b43-a0d6-9527443bab32 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:35.463344 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-865f2639-498b-4b43-a0d6-9527443bab32 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:35.463928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-865f2639-498b-4b43-a0d6-9527443bab32 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:35.464181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fa162c9-3049-4af5-8cfc-1983e72200f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:35.464730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fa162c9-3049-4af5-8cfc-1983e72200f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:35.465257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fa162c9-3049-4af5-8cfc-1983e72200f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:35.466084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fa162c9-3049-4af5-8cfc-1983e72200f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:35.466677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fa162c9-3049-4af5-8cfc-1983e72200f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:35.467059 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fa162c9-3049-4af5-8cfc-1983e72200f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:35.556946 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-865f2639-498b-4b43-a0d6-9527443bab32 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:35.565944 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-865f2639-498b-4b43-a0d6-9527443bab32 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.116235 Aug 28 01:19:35.566536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 892/2008] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:35 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:35.629559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4fa162c9-3049-4af5-8cfc-1983e72200f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:35.639533 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4fa162c9-3049-4af5-8cfc-1983e72200f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:35.641710 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4fa162c9-3049-4af5-8cfc-1983e72200f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:35.643317 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4fa162c9-3049-4af5-8cfc-1983e72200f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.189821 Aug 28 01:19:35.643980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1117/2009] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:35 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:36.473072 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fecb77e1-107a-432c-8cda-24bbfb6f25ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:36.473799 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fecb77e1-107a-432c-8cda-24bbfb6f25ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:36.474373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e45d85e8-68d2-4694-8a3c-6136a42cc666 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:36.474978 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e45d85e8-68d2-4694-8a3c-6136a42cc666 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:36.480134 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fecb77e1-107a-432c-8cda-24bbfb6f25ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:36.480623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fecb77e1-107a-432c-8cda-24bbfb6f25ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:36.481074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fecb77e1-107a-432c-8cda-24bbfb6f25ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:36.481783 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fecb77e1-107a-432c-8cda-24bbfb6f25ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:36.481973 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e45d85e8-68d2-4694-8a3c-6136a42cc666 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:36.482224 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fecb77e1-107a-432c-8cda-24bbfb6f25ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:36.482436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e45d85e8-68d2-4694-8a3c-6136a42cc666 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:36.482652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fecb77e1-107a-432c-8cda-24bbfb6f25ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:36.482834 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e45d85e8-68d2-4694-8a3c-6136a42cc666 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:36.483493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e45d85e8-68d2-4694-8a3c-6136a42cc666 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:36.483952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e45d85e8-68d2-4694-8a3c-6136a42cc666 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:36.484371 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e45d85e8-68d2-4694-8a3c-6136a42cc666 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:36.690213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e45d85e8-68d2-4694-8a3c-6136a42cc666 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:36.694977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fecb77e1-107a-432c-8cda-24bbfb6f25ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:36.699577 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e45d85e8-68d2-4694-8a3c-6136a42cc666 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.229111 Aug 28 01:19:36.700168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1118/2010] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:36 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:36.704846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4eb6a5b4-e85e-455a-88cd-6334e5118842 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:36.705551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4eb6a5b4-e85e-455a-88cd-6334e5118842 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:36.711932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4eb6a5b4-e85e-455a-88cd-6334e5118842 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:36.712420 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4eb6a5b4-e85e-455a-88cd-6334e5118842 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:36.712853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4eb6a5b4-e85e-455a-88cd-6334e5118842 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:36.713589 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4eb6a5b4-e85e-455a-88cd-6334e5118842 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:36.714041 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4eb6a5b4-e85e-455a-88cd-6334e5118842 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:36.714470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4eb6a5b4-e85e-455a-88cd-6334e5118842 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:36.715237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fecb77e1-107a-432c-8cda-24bbfb6f25ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.246120 Aug 28 01:19:36.716059 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 893/2011] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:36 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:36.720869 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8b0260f1-c78c-4fb9-b829-8f70b652a86e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:36.721499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8b0260f1-c78c-4fb9-b829-8f70b652a86e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:36.728859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b0260f1-c78c-4fb9-b829-8f70b652a86e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:36.729347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b0260f1-c78c-4fb9-b829-8f70b652a86e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:36.729722 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b0260f1-c78c-4fb9-b829-8f70b652a86e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:36.730460 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b0260f1-c78c-4fb9-b829-8f70b652a86e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:36.730913 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b0260f1-c78c-4fb9-b829-8f70b652a86e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:36.731336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b0260f1-c78c-4fb9-b829-8f70b652a86e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:36.734653 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:19:36.814516 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4eb6a5b4-e85e-455a-88cd-6334e5118842 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:36.831164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4eb6a5b4-e85e-455a-88cd-6334e5118842 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.129835 Aug 28 01:19:36.832062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1119/2012] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:36 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:36.889750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8b0260f1-c78c-4fb9-b829-8f70b652a86e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:36.898802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8b0260f1-c78c-4fb9-b829-8f70b652a86e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:36.900216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8b0260f1-c78c-4fb9-b829-8f70b652a86e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:36.901416 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8b0260f1-c78c-4fb9-b829-8f70b652a86e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.184480 Aug 28 01:19:36.901990 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 894/2013] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:36 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:37.721231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b09ce4b8-9556-4426-85ad-2a445697ff66 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:37.721877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b09ce4b8-9556-4426-85ad-2a445697ff66 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:37.729221 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b09ce4b8-9556-4426-85ad-2a445697ff66 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:37.729723 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b09ce4b8-9556-4426-85ad-2a445697ff66 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:37.730197 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b09ce4b8-9556-4426-85ad-2a445697ff66 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:37.730901 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b09ce4b8-9556-4426-85ad-2a445697ff66 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:37.731353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b09ce4b8-9556-4426-85ad-2a445697ff66 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:37.731773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b09ce4b8-9556-4426-85ad-2a445697ff66 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:37.747269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-36f1dd20-1468-43df-81f4-b94e66cd0fb6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:37.748846 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-36f1dd20-1468-43df-81f4-b94e66cd0fb6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:37.755784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36f1dd20-1468-43df-81f4-b94e66cd0fb6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:37.756343 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36f1dd20-1468-43df-81f4-b94e66cd0fb6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:37.756773 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36f1dd20-1468-43df-81f4-b94e66cd0fb6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:37.757493 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36f1dd20-1468-43df-81f4-b94e66cd0fb6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:37.757942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36f1dd20-1468-43df-81f4-b94e66cd0fb6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:37.758347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36f1dd20-1468-43df-81f4-b94e66cd0fb6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:37.941442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b09ce4b8-9556-4426-85ad-2a445697ff66 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:37.947198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-36f1dd20-1468-43df-81f4-b94e66cd0fb6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:37.950808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b09ce4b8-9556-4426-85ad-2a445697ff66 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.233542 Aug 28 01:19:37.951529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1120/2014] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:37 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:37.956912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-14762014-e98f-4511-bc58-f5646dc3fb40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:37.957552 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-14762014-e98f-4511-bc58-f5646dc3fb40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:37.958596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-36f1dd20-1468-43df-81f4-b94e66cd0fb6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.214847 Aug 28 01:19:37.960544 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 895/2015] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:37 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:37.964694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14762014-e98f-4511-bc58-f5646dc3fb40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:37.965065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9bf2fa4b-e3f4-4be3-aca3-6028f889dd09 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:37.965691 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9bf2fa4b-e3f4-4be3-aca3-6028f889dd09 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:37.965849 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14762014-e98f-4511-bc58-f5646dc3fb40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:37.966187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14762014-e98f-4511-bc58-f5646dc3fb40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:37.966965 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14762014-e98f-4511-bc58-f5646dc3fb40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:37.967694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14762014-e98f-4511-bc58-f5646dc3fb40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:37.967837 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14762014-e98f-4511-bc58-f5646dc3fb40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:37.973747 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bf2fa4b-e3f4-4be3-aca3-6028f889dd09 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:37.974165 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bf2fa4b-e3f4-4be3-aca3-6028f889dd09 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:37.974605 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bf2fa4b-e3f4-4be3-aca3-6028f889dd09 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:37.976319 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bf2fa4b-e3f4-4be3-aca3-6028f889dd09 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:37.976789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bf2fa4b-e3f4-4be3-aca3-6028f889dd09 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:37.977317 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bf2fa4b-e3f4-4be3-aca3-6028f889dd09 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:38.049635 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-14762014-e98f-4511-bc58-f5646dc3fb40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:38.058730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-14762014-e98f-4511-bc58-f5646dc3fb40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.106109 Aug 28 01:19:38.059558 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1121/2016] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:37 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:38.157669 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9bf2fa4b-e3f4-4be3-aca3-6028f889dd09 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:38.167277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9bf2fa4b-e3f4-4be3-aca3-6028f889dd09 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:38.168688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9bf2fa4b-e3f4-4be3-aca3-6028f889dd09 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:38.170006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9bf2fa4b-e3f4-4be3-aca3-6028f889dd09 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.209527 Aug 28 01:19:38.170576 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 896/2017] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:37 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:38.972667 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cf0414c3-6866-4ce8-8e55-7c4655bb2d94 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:38.973429 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cf0414c3-6866-4ce8-8e55-7c4655bb2d94 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:38.980288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf0414c3-6866-4ce8-8e55-7c4655bb2d94 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:38.980733 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf0414c3-6866-4ce8-8e55-7c4655bb2d94 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:38.981196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf0414c3-6866-4ce8-8e55-7c4655bb2d94 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:38.981898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf0414c3-6866-4ce8-8e55-7c4655bb2d94 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:38.982374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf0414c3-6866-4ce8-8e55-7c4655bb2d94 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:38.982789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf0414c3-6866-4ce8-8e55-7c4655bb2d94 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:38.984420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fc0353fe-4f70-40b9-88b6-c6811044c3af tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:38.985161 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fc0353fe-4f70-40b9-88b6-c6811044c3af tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:38.992443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc0353fe-4f70-40b9-88b6-c6811044c3af tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:38.992938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc0353fe-4f70-40b9-88b6-c6811044c3af tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:38.993402 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc0353fe-4f70-40b9-88b6-c6811044c3af tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:38.994141 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc0353fe-4f70-40b9-88b6-c6811044c3af tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:38.994587 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc0353fe-4f70-40b9-88b6-c6811044c3af tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:38.995007 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc0353fe-4f70-40b9-88b6-c6811044c3af tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:39.244615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-cf0414c3-6866-4ce8-8e55-7c4655bb2d94 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:39.246928 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:19:39.254793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cf0414c3-6866-4ce8-8e55-7c4655bb2d94 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.285985 Aug 28 01:19:39.255394 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1122/2018] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:38 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:39.260173 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-88178e01-6be1-45f6-b729-13217de401f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:39.260790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-88178e01-6be1-45f6-b729-13217de401f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:39.261739 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fc0353fe-4f70-40b9-88b6-c6811044c3af tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:39.267980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88178e01-6be1-45f6-b729-13217de401f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:39.268466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88178e01-6be1-45f6-b729-13217de401f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:39.268894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88178e01-6be1-45f6-b729-13217de401f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:39.269798 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88178e01-6be1-45f6-b729-13217de401f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:39.270253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88178e01-6be1-45f6-b729-13217de401f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:39.270672 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88178e01-6be1-45f6-b729-13217de401f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:39.272283 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fc0353fe-4f70-40b9-88b6-c6811044c3af tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.292386 Aug 28 01:19:39.272914 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 897/2019] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:38 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:19:39.277694 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9f228094-6928-45cf-8054-39ec8fc4ae3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:39.278894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9f228094-6928-45cf-8054-39ec8fc4ae3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:39.287073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f228094-6928-45cf-8054-39ec8fc4ae3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:39.287881 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f228094-6928-45cf-8054-39ec8fc4ae3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:39.288609 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f228094-6928-45cf-8054-39ec8fc4ae3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:39.289475 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f228094-6928-45cf-8054-39ec8fc4ae3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:39.289937 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f228094-6928-45cf-8054-39ec8fc4ae3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:39.290364 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f228094-6928-45cf-8054-39ec8fc4ae3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:39.347031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-88178e01-6be1-45f6-b729-13217de401f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:39.358934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-88178e01-6be1-45f6-b729-13217de401f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.102469 Aug 28 01:19:39.359534 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1123/2020] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:39 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:39.447539 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9f228094-6928-45cf-8054-39ec8fc4ae3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:39.457050 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9f228094-6928-45cf-8054-39ec8fc4ae3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:39.458678 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9f228094-6928-45cf-8054-39ec8fc4ae3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:39.459964 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9f228094-6928-45cf-8054-39ec8fc4ae3e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.186055 Aug 28 01:19:39.460569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 898/2021] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:39 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:40.281020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b277539e-bd85-4aa6-8c42-5ed0cb3c890c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:40.281771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b277539e-bd85-4aa6-8c42-5ed0cb3c890c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:40.289131 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b277539e-bd85-4aa6-8c42-5ed0cb3c890c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:40.289626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b277539e-bd85-4aa6-8c42-5ed0cb3c890c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:40.290057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b277539e-bd85-4aa6-8c42-5ed0cb3c890c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:40.290772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b277539e-bd85-4aa6-8c42-5ed0cb3c890c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:40.291216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b277539e-bd85-4aa6-8c42-5ed0cb3c890c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:40.291639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b277539e-bd85-4aa6-8c42-5ed0cb3c890c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:40.300737 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a99fea68-2f35-4b6d-b90b-bd96b5db6df0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:40.301742 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a99fea68-2f35-4b6d-b90b-bd96b5db6df0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:40.312745 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a99fea68-2f35-4b6d-b90b-bd96b5db6df0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:40.313572 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a99fea68-2f35-4b6d-b90b-bd96b5db6df0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:40.314324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a99fea68-2f35-4b6d-b90b-bd96b5db6df0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:40.315504 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a99fea68-2f35-4b6d-b90b-bd96b5db6df0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:40.316302 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a99fea68-2f35-4b6d-b90b-bd96b5db6df0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:40.316795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a99fea68-2f35-4b6d-b90b-bd96b5db6df0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:40.494739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b277539e-bd85-4aa6-8c42-5ed0cb3c890c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:40.504761 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b277539e-bd85-4aa6-8c42-5ed0cb3c890c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.227502 Aug 28 01:19:40.505784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1124/2022] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:40 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:40.514679 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0ef5795f-9d24-4124-b9c0-17c87ded25ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:40.515834 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0ef5795f-9d24-4124-b9c0-17c87ded25ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:40.526743 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a99fea68-2f35-4b6d-b90b-bd96b5db6df0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:40.527103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ef5795f-9d24-4124-b9c0-17c87ded25ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:40.527671 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ef5795f-9d24-4124-b9c0-17c87ded25ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:40.528425 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ef5795f-9d24-4124-b9c0-17c87ded25ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:40.529770 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ef5795f-9d24-4124-b9c0-17c87ded25ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:40.530616 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ef5795f-9d24-4124-b9c0-17c87ded25ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:40.531403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ef5795f-9d24-4124-b9c0-17c87ded25ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:40.541427 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a99fea68-2f35-4b6d-b90b-bd96b5db6df0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.246246 Aug 28 01:19:40.542193 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 899/2023] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:40 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:40.547626 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-14df2d5f-7de4-4c09-a7cd-0087ccff3849 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:40.549450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-14df2d5f-7de4-4c09-a7cd-0087ccff3849 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:40.557483 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14df2d5f-7de4-4c09-a7cd-0087ccff3849 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:40.558310 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14df2d5f-7de4-4c09-a7cd-0087ccff3849 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:40.559021 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14df2d5f-7de4-4c09-a7cd-0087ccff3849 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:40.560119 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14df2d5f-7de4-4c09-a7cd-0087ccff3849 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:40.560815 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14df2d5f-7de4-4c09-a7cd-0087ccff3849 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:40.561524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14df2d5f-7de4-4c09-a7cd-0087ccff3849 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:40.623695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0ef5795f-9d24-4124-b9c0-17c87ded25ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:40.633271 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0ef5795f-9d24-4124-b9c0-17c87ded25ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.125931 Aug 28 01:19:40.633885 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1125/2024] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:40 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:40.733703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-14df2d5f-7de4-4c09-a7cd-0087ccff3849 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:40.744829 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-14df2d5f-7de4-4c09-a7cd-0087ccff3849 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:40.746348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-14df2d5f-7de4-4c09-a7cd-0087ccff3849 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:40.747548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-14df2d5f-7de4-4c09-a7cd-0087ccff3849 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.204285 Aug 28 01:19:40.748122 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 900/2025] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:40 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:41.539951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d378fbbc-054c-42a9-a043-9a0066176431 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:41.540931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d378fbbc-054c-42a9-a043-9a0066176431 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:41.547828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d378fbbc-054c-42a9-a043-9a0066176431 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:41.548336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d378fbbc-054c-42a9-a043-9a0066176431 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:41.548787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d378fbbc-054c-42a9-a043-9a0066176431 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:41.549520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d378fbbc-054c-42a9-a043-9a0066176431 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:41.549965 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d378fbbc-054c-42a9-a043-9a0066176431 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:41.550383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d378fbbc-054c-42a9-a043-9a0066176431 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:41.573691 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-acd3a839-6ad2-4454-b329-06e2ed961158 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:41.574359 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-acd3a839-6ad2-4454-b329-06e2ed961158 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:41.581413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-acd3a839-6ad2-4454-b329-06e2ed961158 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:41.581896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-acd3a839-6ad2-4454-b329-06e2ed961158 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:41.582329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-acd3a839-6ad2-4454-b329-06e2ed961158 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:41.583034 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-acd3a839-6ad2-4454-b329-06e2ed961158 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:41.583645 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-acd3a839-6ad2-4454-b329-06e2ed961158 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:41.584067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-acd3a839-6ad2-4454-b329-06e2ed961158 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:41.748787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d378fbbc-054c-42a9-a043-9a0066176431 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:41.758162 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d378fbbc-054c-42a9-a043-9a0066176431 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.222049 Aug 28 01:19:41.758732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1126/2026] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:41 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:41.763479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d6f68b8c-eb36-4763-a392-55b2ec469a12 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:41.764475 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d6f68b8c-eb36-4763-a392-55b2ec469a12 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:41.773376 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6f68b8c-eb36-4763-a392-55b2ec469a12 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:41.773921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6f68b8c-eb36-4763-a392-55b2ec469a12 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:41.774279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6f68b8c-eb36-4763-a392-55b2ec469a12 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:41.774976 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6f68b8c-eb36-4763-a392-55b2ec469a12 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:41.775416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6f68b8c-eb36-4763-a392-55b2ec469a12 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:41.775855 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d6f68b8c-eb36-4763-a392-55b2ec469a12 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:41.776586 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-acd3a839-6ad2-4454-b329-06e2ed961158 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:41.785274 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-acd3a839-6ad2-4454-b329-06e2ed961158 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.215756 Aug 28 01:19:41.785865 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 901/2027] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:41 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:41.790579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-95350150-2c10-4b4c-954e-e3b195ff8e03 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:41.791101 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-95350150-2c10-4b4c-954e-e3b195ff8e03 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:41.798414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95350150-2c10-4b4c-954e-e3b195ff8e03 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:41.799349 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95350150-2c10-4b4c-954e-e3b195ff8e03 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:41.800110 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95350150-2c10-4b4c-954e-e3b195ff8e03 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:41.801380 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95350150-2c10-4b4c-954e-e3b195ff8e03 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:41.802072 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95350150-2c10-4b4c-954e-e3b195ff8e03 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:41.802823 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95350150-2c10-4b4c-954e-e3b195ff8e03 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:41.843555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d6f68b8c-eb36-4763-a392-55b2ec469a12 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:41.852565 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d6f68b8c-eb36-4763-a392-55b2ec469a12 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.092811 Aug 28 01:19:41.853237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1127/2028] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:41 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:41.971084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-95350150-2c10-4b4c-954e-e3b195ff8e03 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:41.981604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-95350150-2c10-4b4c-954e-e3b195ff8e03 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:41.983036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-95350150-2c10-4b4c-954e-e3b195ff8e03 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:41.984347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-95350150-2c10-4b4c-954e-e3b195ff8e03 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.197551 Aug 28 01:19:41.984976 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 902/2029] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:41 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:42.780712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0ae072e4-b7c5-480c-a0e9-68f8b8acf4eb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:42.781425 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0ae072e4-b7c5-480c-a0e9-68f8b8acf4eb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:42.788071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ae072e4-b7c5-480c-a0e9-68f8b8acf4eb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:42.788556 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ae072e4-b7c5-480c-a0e9-68f8b8acf4eb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:42.789005 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ae072e4-b7c5-480c-a0e9-68f8b8acf4eb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:42.789671 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ae072e4-b7c5-480c-a0e9-68f8b8acf4eb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:42.790111 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ae072e4-b7c5-480c-a0e9-68f8b8acf4eb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:42.790527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ae072e4-b7c5-480c-a0e9-68f8b8acf4eb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:42.807938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7885809d-433f-4f3d-93eb-562e5472f23b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:42.808566 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7885809d-433f-4f3d-93eb-562e5472f23b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:42.815770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7885809d-433f-4f3d-93eb-562e5472f23b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:42.816344 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7885809d-433f-4f3d-93eb-562e5472f23b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:42.817018 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7885809d-433f-4f3d-93eb-562e5472f23b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:42.818235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7885809d-433f-4f3d-93eb-562e5472f23b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:42.818758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7885809d-433f-4f3d-93eb-562e5472f23b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:42.819316 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7885809d-433f-4f3d-93eb-562e5472f23b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:43.007645 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0ae072e4-b7c5-480c-a0e9-68f8b8acf4eb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:43.016165 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0ae072e4-b7c5-480c-a0e9-68f8b8acf4eb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.239665 Aug 28 01:19:43.016750 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1128/2030] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:42 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:43.021537 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6646228b-f9d0-499e-a2f2-17348884a21c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:43.022139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6646228b-f9d0-499e-a2f2-17348884a21c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:43.028406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6646228b-f9d0-499e-a2f2-17348884a21c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:43.028850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6646228b-f9d0-499e-a2f2-17348884a21c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:43.029323 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6646228b-f9d0-499e-a2f2-17348884a21c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:43.030203 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6646228b-f9d0-499e-a2f2-17348884a21c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:43.030649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6646228b-f9d0-499e-a2f2-17348884a21c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:43.031120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6646228b-f9d0-499e-a2f2-17348884a21c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:43.031873 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7885809d-433f-4f3d-93eb-562e5472f23b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:43.049651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7885809d-433f-4f3d-93eb-562e5472f23b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.244805 Aug 28 01:19:43.051125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 903/2031] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:42 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:43.062668 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0b4e3597-a52a-490d-b76c-3e3d06b0a21d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:43.063824 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0b4e3597-a52a-490d-b76c-3e3d06b0a21d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:43.075150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0b4e3597-a52a-490d-b76c-3e3d06b0a21d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:43.076045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0b4e3597-a52a-490d-b76c-3e3d06b0a21d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:43.076845 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0b4e3597-a52a-490d-b76c-3e3d06b0a21d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:43.078172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0b4e3597-a52a-490d-b76c-3e3d06b0a21d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:43.078975 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0b4e3597-a52a-490d-b76c-3e3d06b0a21d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:43.079770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0b4e3597-a52a-490d-b76c-3e3d06b0a21d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:43.103045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6646228b-f9d0-499e-a2f2-17348884a21c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:43.112828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6646228b-f9d0-499e-a2f2-17348884a21c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.094987 Aug 28 01:19:43.113692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1129/2032] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:43 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:43.219719 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0b4e3597-a52a-490d-b76c-3e3d06b0a21d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:43.230398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0b4e3597-a52a-490d-b76c-3e3d06b0a21d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:43.231921 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0b4e3597-a52a-490d-b76c-3e3d06b0a21d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:43.233167 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0b4e3597-a52a-490d-b76c-3e3d06b0a21d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.180031 Aug 28 01:19:43.233727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 904/2033] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:43 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:44.039175 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c3528fc8-ba7c-4b7e-989d-8a76216d758e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:44.039814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c3528fc8-ba7c-4b7e-989d-8a76216d758e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:44.047403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3528fc8-ba7c-4b7e-989d-8a76216d758e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:44.047899 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3528fc8-ba7c-4b7e-989d-8a76216d758e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:44.048332 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3528fc8-ba7c-4b7e-989d-8a76216d758e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:44.049176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3528fc8-ba7c-4b7e-989d-8a76216d758e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:44.049627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3528fc8-ba7c-4b7e-989d-8a76216d758e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:44.050045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3528fc8-ba7c-4b7e-989d-8a76216d758e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:44.076588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-adfb36d4-8526-4d36-94f9-fba07a0e1177 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:44.077429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-adfb36d4-8526-4d36-94f9-fba07a0e1177 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:44.084954 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-adfb36d4-8526-4d36-94f9-fba07a0e1177 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:44.085498 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-adfb36d4-8526-4d36-94f9-fba07a0e1177 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:44.085924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-adfb36d4-8526-4d36-94f9-fba07a0e1177 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:44.086608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-adfb36d4-8526-4d36-94f9-fba07a0e1177 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:44.087053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-adfb36d4-8526-4d36-94f9-fba07a0e1177 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:44.087470 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-adfb36d4-8526-4d36-94f9-fba07a0e1177 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:44.250964 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c3528fc8-ba7c-4b7e-989d-8a76216d758e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:44.262258 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c3528fc8-ba7c-4b7e-989d-8a76216d758e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.227124 Aug 28 01:19:44.262881 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1130/2034] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:44 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:44.267905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-79dfb5ea-58ae-4e48-aa90-c7f4040eb5fb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:44.268505 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-79dfb5ea-58ae-4e48-aa90-c7f4040eb5fb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:44.276025 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79dfb5ea-58ae-4e48-aa90-c7f4040eb5fb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:44.276546 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79dfb5ea-58ae-4e48-aa90-c7f4040eb5fb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:44.277014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79dfb5ea-58ae-4e48-aa90-c7f4040eb5fb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:44.277533 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-adfb36d4-8526-4d36-94f9-fba07a0e1177 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:44.277748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79dfb5ea-58ae-4e48-aa90-c7f4040eb5fb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:44.278208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79dfb5ea-58ae-4e48-aa90-c7f4040eb5fb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:44.278736 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79dfb5ea-58ae-4e48-aa90-c7f4040eb5fb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:44.286281 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-adfb36d4-8526-4d36-94f9-fba07a0e1177 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.216211 Aug 28 01:19:44.286872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 905/2035] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:44 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:19:44.291838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-43cf516b-e6ee-49b8-9a36-fd767558b0d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:44.292356 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-43cf516b-e6ee-49b8-9a36-fd767558b0d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:44.298995 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43cf516b-e6ee-49b8-9a36-fd767558b0d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:44.299495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43cf516b-e6ee-49b8-9a36-fd767558b0d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:44.299925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43cf516b-e6ee-49b8-9a36-fd767558b0d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:44.300636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43cf516b-e6ee-49b8-9a36-fd767558b0d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:44.301133 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43cf516b-e6ee-49b8-9a36-fd767558b0d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:44.301570 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-43cf516b-e6ee-49b8-9a36-fd767558b0d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:44.357507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-79dfb5ea-58ae-4e48-aa90-c7f4040eb5fb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:44.366231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-79dfb5ea-58ae-4e48-aa90-c7f4040eb5fb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.102273 Aug 28 01:19:44.366827 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1131/2036] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:44 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:44.459300 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-43cf516b-e6ee-49b8-9a36-fd767558b0d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:44.468216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-43cf516b-e6ee-49b8-9a36-fd767558b0d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:44.469683 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-43cf516b-e6ee-49b8-9a36-fd767558b0d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:44.470958 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-43cf516b-e6ee-49b8-9a36-fd767558b0d8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.183148 Aug 28 01:19:44.471543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 906/2037] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:44 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:45.283995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9e42f5b9-d069-43bc-b0e9-82de7fbf840a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:45.284622 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9e42f5b9-d069-43bc-b0e9-82de7fbf840a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:45.291473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9e42f5b9-d069-43bc-b0e9-82de7fbf840a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:45.291963 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9e42f5b9-d069-43bc-b0e9-82de7fbf840a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:45.292400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9e42f5b9-d069-43bc-b0e9-82de7fbf840a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:45.293173 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9e42f5b9-d069-43bc-b0e9-82de7fbf840a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:45.293625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9e42f5b9-d069-43bc-b0e9-82de7fbf840a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:45.294055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9e42f5b9-d069-43bc-b0e9-82de7fbf840a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:45.317798 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8d4644c1-2c98-4b5c-96ce-1220c9c5f32b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:45.319141 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8d4644c1-2c98-4b5c-96ce-1220c9c5f32b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:45.325939 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8d4644c1-2c98-4b5c-96ce-1220c9c5f32b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:45.326426 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8d4644c1-2c98-4b5c-96ce-1220c9c5f32b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:45.326863 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8d4644c1-2c98-4b5c-96ce-1220c9c5f32b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:45.327564 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8d4644c1-2c98-4b5c-96ce-1220c9c5f32b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:45.328015 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8d4644c1-2c98-4b5c-96ce-1220c9c5f32b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:45.328471 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8d4644c1-2c98-4b5c-96ce-1220c9c5f32b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:45.497708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9e42f5b9-d069-43bc-b0e9-82de7fbf840a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:45.510282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9e42f5b9-d069-43bc-b0e9-82de7fbf840a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.229332 Aug 28 01:19:45.511151 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1132/2038] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:45 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:45.512608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8d4644c1-2c98-4b5c-96ce-1220c9c5f32b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:45.520635 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-51a8ae91-7084-480e-a6be-fb9a43ccc576 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:45.521893 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-51a8ae91-7084-480e-a6be-fb9a43ccc576 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:45.525336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8d4644c1-2c98-4b5c-96ce-1220c9c5f32b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.211980 Aug 28 01:19:45.528016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 907/2039] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:45 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:45.533508 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-51a8ae91-7084-480e-a6be-fb9a43ccc576 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:45.534079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-51a8ae91-7084-480e-a6be-fb9a43ccc576 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:45.534524 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-51a8ae91-7084-480e-a6be-fb9a43ccc576 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:45.535237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-51a8ae91-7084-480e-a6be-fb9a43ccc576 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:45.535688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-51a8ae91-7084-480e-a6be-fb9a43ccc576 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:45.536112 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-51a8ae91-7084-480e-a6be-fb9a43ccc576 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:45.536866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-00c6a429-7d7b-421a-b693-6cf0130b54f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:45.538125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-00c6a429-7d7b-421a-b693-6cf0130b54f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:45.548769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00c6a429-7d7b-421a-b693-6cf0130b54f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:45.549311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00c6a429-7d7b-421a-b693-6cf0130b54f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:45.549798 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00c6a429-7d7b-421a-b693-6cf0130b54f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:45.550513 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00c6a429-7d7b-421a-b693-6cf0130b54f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:45.550961 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00c6a429-7d7b-421a-b693-6cf0130b54f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:45.551383 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-00c6a429-7d7b-421a-b693-6cf0130b54f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:45.641351 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-51a8ae91-7084-480e-a6be-fb9a43ccc576 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:45.650378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-51a8ae91-7084-480e-a6be-fb9a43ccc576 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.137589 Aug 28 01:19:45.650968 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1133/2040] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:45 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:45.713445 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-00c6a429-7d7b-421a-b693-6cf0130b54f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:45.722766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-00c6a429-7d7b-421a-b693-6cf0130b54f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:45.724193 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-00c6a429-7d7b-421a-b693-6cf0130b54f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:45.725535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-00c6a429-7d7b-421a-b693-6cf0130b54f1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.198130 Aug 28 01:19:45.726079 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 908/2041] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:45 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:46.535018 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4873fb68-0f22-4678-9bbf-6e8831c2b5b4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:46.535788 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4873fb68-0f22-4678-9bbf-6e8831c2b5b4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:46.543446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4873fb68-0f22-4678-9bbf-6e8831c2b5b4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:46.544047 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4873fb68-0f22-4678-9bbf-6e8831c2b5b4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:46.544580 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4873fb68-0f22-4678-9bbf-6e8831c2b5b4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:46.545527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4873fb68-0f22-4678-9bbf-6e8831c2b5b4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:46.546062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4873fb68-0f22-4678-9bbf-6e8831c2b5b4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:46.546492 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4873fb68-0f22-4678-9bbf-6e8831c2b5b4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:46.560125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9d256583-588a-4e90-9146-9af02a7f88d3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:46.560757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9d256583-588a-4e90-9146-9af02a7f88d3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:46.568039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d256583-588a-4e90-9146-9af02a7f88d3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:46.568540 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d256583-588a-4e90-9146-9af02a7f88d3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:46.568986 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d256583-588a-4e90-9146-9af02a7f88d3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:46.569686 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d256583-588a-4e90-9146-9af02a7f88d3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:46.570131 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d256583-588a-4e90-9146-9af02a7f88d3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:46.570553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d256583-588a-4e90-9146-9af02a7f88d3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:46.734628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4873fb68-0f22-4678-9bbf-6e8831c2b5b4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:46.743380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4873fb68-0f22-4678-9bbf-6e8831c2b5b4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.212591 Aug 28 01:19:46.744357 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1134/2042] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:46 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:46.749049 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b64e445d-9016-4e80-891f-fc60630e140b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:46.749731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b64e445d-9016-4e80-891f-fc60630e140b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:46.756049 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b64e445d-9016-4e80-891f-fc60630e140b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:46.756558 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b64e445d-9016-4e80-891f-fc60630e140b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:46.757079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b64e445d-9016-4e80-891f-fc60630e140b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:46.757847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b64e445d-9016-4e80-891f-fc60630e140b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:46.758275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b64e445d-9016-4e80-891f-fc60630e140b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:46.758749 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b64e445d-9016-4e80-891f-fc60630e140b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:46.759035 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9d256583-588a-4e90-9146-9af02a7f88d3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:46.768099 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9d256583-588a-4e90-9146-9af02a7f88d3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.212030 Aug 28 01:19:46.768760 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 909/2043] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:46 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:46.773550 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6a620d53-d392-4eda-983e-e253e434f6c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:46.774250 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6a620d53-d392-4eda-983e-e253e434f6c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:46.780209 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6a620d53-d392-4eda-983e-e253e434f6c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:46.780766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6a620d53-d392-4eda-983e-e253e434f6c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:46.781204 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6a620d53-d392-4eda-983e-e253e434f6c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:46.782204 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6a620d53-d392-4eda-983e-e253e434f6c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:46.782650 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6a620d53-d392-4eda-983e-e253e434f6c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:46.783127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6a620d53-d392-4eda-983e-e253e434f6c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:47.073628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b64e445d-9016-4e80-891f-fc60630e140b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:47.082481 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b64e445d-9016-4e80-891f-fc60630e140b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.336988 Aug 28 01:19:47.083054 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1135/2044] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:46 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:47.159162 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6a620d53-d392-4eda-983e-e253e434f6c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:47.161821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:19:47.170120 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6a620d53-d392-4eda-983e-e253e434f6c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:47.171551 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6a620d53-d392-4eda-983e-e253e434f6c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:47.172729 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6a620d53-d392-4eda-983e-e253e434f6c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.402935 Aug 28 01:19:47.173312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 910/2045] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:46 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:47.765826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-70af130b-f7e0-4c47-a3ed-5648d298be3f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:47.766494 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-70af130b-f7e0-4c47-a3ed-5648d298be3f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:47.772929 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70af130b-f7e0-4c47-a3ed-5648d298be3f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:47.773454 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70af130b-f7e0-4c47-a3ed-5648d298be3f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:47.773886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70af130b-f7e0-4c47-a3ed-5648d298be3f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:47.774577 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70af130b-f7e0-4c47-a3ed-5648d298be3f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:47.775024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70af130b-f7e0-4c47-a3ed-5648d298be3f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:47.775448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70af130b-f7e0-4c47-a3ed-5648d298be3f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:47.790294 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5b691e7e-accc-4b19-b9e2-3b2eb528525a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:47.790871 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5b691e7e-accc-4b19-b9e2-3b2eb528525a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:47.797471 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b691e7e-accc-4b19-b9e2-3b2eb528525a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:47.798213 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b691e7e-accc-4b19-b9e2-3b2eb528525a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:47.798465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b691e7e-accc-4b19-b9e2-3b2eb528525a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:47.799173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b691e7e-accc-4b19-b9e2-3b2eb528525a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:47.799604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b691e7e-accc-4b19-b9e2-3b2eb528525a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:47.800011 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b691e7e-accc-4b19-b9e2-3b2eb528525a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:47.960200 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-70af130b-f7e0-4c47-a3ed-5648d298be3f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:47.969158 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-70af130b-f7e0-4c47-a3ed-5648d298be3f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.207253 Aug 28 01:19:47.969693 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1136/2046] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:47 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:47.974207 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5b691e7e-accc-4b19-b9e2-3b2eb528525a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:47.983062 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5b691e7e-accc-4b19-b9e2-3b2eb528525a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.196629 Aug 28 01:19:47.983637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 911/2047] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:47 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:48.104841 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f17c9c72-630c-4fcf-bf8b-db9414cef1d6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:48.105504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f17c9c72-630c-4fcf-bf8b-db9414cef1d6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:48.112033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f17c9c72-630c-4fcf-bf8b-db9414cef1d6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:48.112548 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f17c9c72-630c-4fcf-bf8b-db9414cef1d6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:48.112991 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f17c9c72-630c-4fcf-bf8b-db9414cef1d6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:48.113700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f17c9c72-630c-4fcf-bf8b-db9414cef1d6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:48.114150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f17c9c72-630c-4fcf-bf8b-db9414cef1d6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:48.114640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f17c9c72-630c-4fcf-bf8b-db9414cef1d6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:48.177685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f17c9c72-630c-4fcf-bf8b-db9414cef1d6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:48.186229 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f17c9c72-630c-4fcf-bf8b-db9414cef1d6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.085227 Aug 28 01:19:48.186800 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1137/2048] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:48 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:48.196720 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f0120bba-1527-42d4-810a-5b01a81eecc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:48.197391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f0120bba-1527-42d4-810a-5b01a81eecc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:48.204128 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0120bba-1527-42d4-810a-5b01a81eecc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:48.204638 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0120bba-1527-42d4-810a-5b01a81eecc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:48.205114 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0120bba-1527-42d4-810a-5b01a81eecc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:48.205983 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0120bba-1527-42d4-810a-5b01a81eecc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:48.206304 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0120bba-1527-42d4-810a-5b01a81eecc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:48.206723 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0120bba-1527-42d4-810a-5b01a81eecc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:48.350872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f0120bba-1527-42d4-810a-5b01a81eecc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:48.359667 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f0120bba-1527-42d4-810a-5b01a81eecc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:48.361103 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f0120bba-1527-42d4-810a-5b01a81eecc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:48.362258 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f0120bba-1527-42d4-810a-5b01a81eecc0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.170285 Aug 28 01:19:48.362817 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 912/2049] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:48 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:48.990953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ebaa4ab0-ef6f-4e53-88f8-551975c95091 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:48.991533 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ebaa4ab0-ef6f-4e53-88f8-551975c95091 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:48.998605 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebaa4ab0-ef6f-4e53-88f8-551975c95091 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:48.999114 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebaa4ab0-ef6f-4e53-88f8-551975c95091 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:48.999552 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebaa4ab0-ef6f-4e53-88f8-551975c95091 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:49.000277 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebaa4ab0-ef6f-4e53-88f8-551975c95091 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:49.000709 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebaa4ab0-ef6f-4e53-88f8-551975c95091 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:49.001174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebaa4ab0-ef6f-4e53-88f8-551975c95091 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:49.003904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c2900817-6dee-443d-bac0-0a26ee66f275 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:49.004497 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c2900817-6dee-443d-bac0-0a26ee66f275 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:49.011004 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c2900817-6dee-443d-bac0-0a26ee66f275 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:49.011483 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c2900817-6dee-443d-bac0-0a26ee66f275 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:49.011910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c2900817-6dee-443d-bac0-0a26ee66f275 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:49.012599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c2900817-6dee-443d-bac0-0a26ee66f275 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:49.013067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c2900817-6dee-443d-bac0-0a26ee66f275 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:49.013483 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c2900817-6dee-443d-bac0-0a26ee66f275 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:49.198318 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ebaa4ab0-ef6f-4e53-88f8-551975c95091 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:49.205702 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c2900817-6dee-443d-bac0-0a26ee66f275 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:49.207406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ebaa4ab0-ef6f-4e53-88f8-551975c95091 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.220416 Aug 28 01:19:49.207966 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1138/2050] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:48 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:49.212586 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a125c304-4392-4357-ad30-6f40ed63e42b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:49.213354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a125c304-4392-4357-ad30-6f40ed63e42b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:49.214187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c2900817-6dee-443d-bac0-0a26ee66f275 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.214213 Aug 28 01:19:49.214887 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 913/2051] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:48 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:49.220357 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a125c304-4392-4357-ad30-6f40ed63e42b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:49.220854 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a125c304-4392-4357-ad30-6f40ed63e42b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:49.221319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a125c304-4392-4357-ad30-6f40ed63e42b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:49.222029 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a125c304-4392-4357-ad30-6f40ed63e42b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:49.222473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a125c304-4392-4357-ad30-6f40ed63e42b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:49.222899 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a125c304-4392-4357-ad30-6f40ed63e42b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:49.238575 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:19:49.290411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a125c304-4392-4357-ad30-6f40ed63e42b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:49.299046 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a125c304-4392-4357-ad30-6f40ed63e42b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.090092 Aug 28 01:19:49.299613 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1139/2052] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:49 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:49.388647 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b0eb4cbb-4cfd-467f-a4bc-334f66119ab8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:49.389465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b0eb4cbb-4cfd-467f-a4bc-334f66119ab8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:49.396807 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0eb4cbb-4cfd-467f-a4bc-334f66119ab8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:49.397346 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0eb4cbb-4cfd-467f-a4bc-334f66119ab8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:49.397780 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0eb4cbb-4cfd-467f-a4bc-334f66119ab8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:49.398468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0eb4cbb-4cfd-467f-a4bc-334f66119ab8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:49.398895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0eb4cbb-4cfd-467f-a4bc-334f66119ab8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:49.399334 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0eb4cbb-4cfd-467f-a4bc-334f66119ab8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:49.542587 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b0eb4cbb-4cfd-467f-a4bc-334f66119ab8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:49.554999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b0eb4cbb-4cfd-467f-a4bc-334f66119ab8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:49.557128 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b0eb4cbb-4cfd-467f-a4bc-334f66119ab8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:49.558585 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b0eb4cbb-4cfd-467f-a4bc-334f66119ab8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.175955 Aug 28 01:19:49.559310 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 914/2053] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:49 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:50.230548 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3f2cc452-5c14-451e-9c21-81e669535bbe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:50.231157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3f2cc452-5c14-451e-9c21-81e669535bbe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:50.235250 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-022f9fbd-ed19-4bff-aac7-a243907eb3b7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:50.235860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-022f9fbd-ed19-4bff-aac7-a243907eb3b7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:50.238149 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3f2cc452-5c14-451e-9c21-81e669535bbe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:50.238631 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3f2cc452-5c14-451e-9c21-81e669535bbe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:50.239084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3f2cc452-5c14-451e-9c21-81e669535bbe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:50.239771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3f2cc452-5c14-451e-9c21-81e669535bbe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:50.240216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3f2cc452-5c14-451e-9c21-81e669535bbe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:50.240636 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3f2cc452-5c14-451e-9c21-81e669535bbe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:50.243394 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-022f9fbd-ed19-4bff-aac7-a243907eb3b7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:50.244098 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-022f9fbd-ed19-4bff-aac7-a243907eb3b7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:50.244662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-022f9fbd-ed19-4bff-aac7-a243907eb3b7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:50.245872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-022f9fbd-ed19-4bff-aac7-a243907eb3b7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:50.246121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-022f9fbd-ed19-4bff-aac7-a243907eb3b7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:50.246554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-022f9fbd-ed19-4bff-aac7-a243907eb3b7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:50.445503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3f2cc452-5c14-451e-9c21-81e669535bbe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:50.448318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-022f9fbd-ed19-4bff-aac7-a243907eb3b7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:50.454187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3f2cc452-5c14-451e-9c21-81e669535bbe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.227828 Aug 28 01:19:50.458021 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1140/2054] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:50 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:50.458218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-022f9fbd-ed19-4bff-aac7-a243907eb3b7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.225491 Aug 28 01:19:50.458218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 915/2055] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:50 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:50.460726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-79435bce-4e5d-4451-9f1c-be0a6d2e42f5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:50.461343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-79435bce-4e5d-4451-9f1c-be0a6d2e42f5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:50.467442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79435bce-4e5d-4451-9f1c-be0a6d2e42f5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:50.468056 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79435bce-4e5d-4451-9f1c-be0a6d2e42f5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:50.468493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79435bce-4e5d-4451-9f1c-be0a6d2e42f5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:50.469363 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79435bce-4e5d-4451-9f1c-be0a6d2e42f5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:50.469804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79435bce-4e5d-4451-9f1c-be0a6d2e42f5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:50.470333 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79435bce-4e5d-4451-9f1c-be0a6d2e42f5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:50.544873 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-79435bce-4e5d-4451-9f1c-be0a6d2e42f5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:50.553868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-79435bce-4e5d-4451-9f1c-be0a6d2e42f5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.098055 Aug 28 01:19:50.554450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1141/2056] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:50 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:50.584094 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4e913e69-8f33-49ab-ab52-09caa53ac0bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:50.584741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4e913e69-8f33-49ab-ab52-09caa53ac0bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:50.591159 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e913e69-8f33-49ab-ab52-09caa53ac0bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:50.591642 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e913e69-8f33-49ab-ab52-09caa53ac0bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:50.592074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e913e69-8f33-49ab-ab52-09caa53ac0bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:50.592769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e913e69-8f33-49ab-ab52-09caa53ac0bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:50.593247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e913e69-8f33-49ab-ab52-09caa53ac0bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:50.593663 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4e913e69-8f33-49ab-ab52-09caa53ac0bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:50.724608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4e913e69-8f33-49ab-ab52-09caa53ac0bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:50.733886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4e913e69-8f33-49ab-ab52-09caa53ac0bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:50.735314 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4e913e69-8f33-49ab-ab52-09caa53ac0bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:50.736582 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4e913e69-8f33-49ab-ab52-09caa53ac0bc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.156427 Aug 28 01:19:50.737147 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 916/2057] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:50 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:51.477379 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d806dded-53cc-41d4-883c-c8826e936fc1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:51.478176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d806dded-53cc-41d4-883c-c8826e936fc1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:51.479711 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1d411a2c-ec87-48c8-8d98-963e6d573d32 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:51.480318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1d411a2c-ec87-48c8-8d98-963e6d573d32 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:51.485085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d806dded-53cc-41d4-883c-c8826e936fc1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:51.485595 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d806dded-53cc-41d4-883c-c8826e936fc1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:51.486008 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d806dded-53cc-41d4-883c-c8826e936fc1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:51.486777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d806dded-53cc-41d4-883c-c8826e936fc1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:51.487233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d806dded-53cc-41d4-883c-c8826e936fc1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:51.487660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1d411a2c-ec87-48c8-8d98-963e6d573d32 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:51.487836 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d806dded-53cc-41d4-883c-c8826e936fc1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:51.488145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1d411a2c-ec87-48c8-8d98-963e6d573d32 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:51.488554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1d411a2c-ec87-48c8-8d98-963e6d573d32 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:51.489347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1d411a2c-ec87-48c8-8d98-963e6d573d32 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:51.489794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1d411a2c-ec87-48c8-8d98-963e6d573d32 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:51.490204 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1d411a2c-ec87-48c8-8d98-963e6d573d32 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:51.692823 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d806dded-53cc-41d4-883c-c8826e936fc1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:51.697545 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1d411a2c-ec87-48c8-8d98-963e6d573d32 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:51.702349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d806dded-53cc-41d4-883c-c8826e936fc1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.228954 Aug 28 01:19:51.702936 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1142/2058] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:51 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:51.706092 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1d411a2c-ec87-48c8-8d98-963e6d573d32 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.230250 Aug 28 01:19:51.706675 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 917/2059] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:51 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:51.707630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-03653475-5272-42b5-a91c-deecdab09918 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:51.708296 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-03653475-5272-42b5-a91c-deecdab09918 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:51.714658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03653475-5272-42b5-a91c-deecdab09918 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:51.715147 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03653475-5272-42b5-a91c-deecdab09918 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:51.715578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03653475-5272-42b5-a91c-deecdab09918 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:51.716271 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03653475-5272-42b5-a91c-deecdab09918 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:51.716703 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03653475-5272-42b5-a91c-deecdab09918 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:51.717163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-03653475-5272-42b5-a91c-deecdab09918 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:51.762045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1bbab482-2e18-4b7b-9135-9e92ac9b958b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:51.762668 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1bbab482-2e18-4b7b-9135-9e92ac9b958b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:51.768786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bbab482-2e18-4b7b-9135-9e92ac9b958b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:51.769394 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bbab482-2e18-4b7b-9135-9e92ac9b958b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:51.769896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bbab482-2e18-4b7b-9135-9e92ac9b958b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:51.770659 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bbab482-2e18-4b7b-9135-9e92ac9b958b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:51.771176 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bbab482-2e18-4b7b-9135-9e92ac9b958b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:51.771664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bbab482-2e18-4b7b-9135-9e92ac9b958b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:51.782112 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-03653475-5272-42b5-a91c-deecdab09918 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:51.791392 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-03653475-5272-42b5-a91c-deecdab09918 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.087405 Aug 28 01:19:51.791958 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1143/2060] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:51 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:51.912116 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1bbab482-2e18-4b7b-9135-9e92ac9b958b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:51.925018 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1bbab482-2e18-4b7b-9135-9e92ac9b958b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:51.926987 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1bbab482-2e18-4b7b-9135-9e92ac9b958b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:51.928669 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1bbab482-2e18-4b7b-9135-9e92ac9b958b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.171743 Aug 28 01:19:51.929454 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 918/2061] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:51 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 01:19:52.724188 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9a5eb104-bd14-45c4-9b63-d977967376b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:52.725015 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9a5eb104-bd14-45c4-9b63-d977967376b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:52.728055 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3139ad3e-3e83-4efd-8bbd-cbcd459648e9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:52.729061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3139ad3e-3e83-4efd-8bbd-cbcd459648e9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:52.732732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a5eb104-bd14-45c4-9b63-d977967376b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:52.733271 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a5eb104-bd14-45c4-9b63-d977967376b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:52.733680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a5eb104-bd14-45c4-9b63-d977967376b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:52.734386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a5eb104-bd14-45c4-9b63-d977967376b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:52.734866 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a5eb104-bd14-45c4-9b63-d977967376b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:52.735276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a5eb104-bd14-45c4-9b63-d977967376b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:52.736672 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3139ad3e-3e83-4efd-8bbd-cbcd459648e9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:52.737414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3139ad3e-3e83-4efd-8bbd-cbcd459648e9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:52.738135 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3139ad3e-3e83-4efd-8bbd-cbcd459648e9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:52.739082 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3139ad3e-3e83-4efd-8bbd-cbcd459648e9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:52.739706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3139ad3e-3e83-4efd-8bbd-cbcd459648e9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:52.740291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3139ad3e-3e83-4efd-8bbd-cbcd459648e9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:52.933753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9a5eb104-bd14-45c4-9b63-d977967376b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:52.943667 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9a5eb104-bd14-45c4-9b63-d977967376b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.223632 Aug 28 01:19:52.944856 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1144/2062] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:52 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:52.948622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3139ad3e-3e83-4efd-8bbd-cbcd459648e9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:52.953351 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f866e3e2-5101-4f2c-853f-ce2387b3417c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:52.954356 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f866e3e2-5101-4f2c-853f-ce2387b3417c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:52.961665 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f866e3e2-5101-4f2c-853f-ce2387b3417c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:52.961967 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3139ad3e-3e83-4efd-8bbd-cbcd459648e9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.237583 Aug 28 01:19:52.962142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f866e3e2-5101-4f2c-853f-ce2387b3417c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:52.962486 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 919/2063] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:52 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:52.962662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f866e3e2-5101-4f2c-853f-ce2387b3417c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:52.963339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f866e3e2-5101-4f2c-853f-ce2387b3417c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:52.963872 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f866e3e2-5101-4f2c-853f-ce2387b3417c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:52.964296 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f866e3e2-5101-4f2c-853f-ce2387b3417c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:52.967408 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4ee582d7-d275-4a3f-a907-4a73b985e5af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:52.968027 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4ee582d7-d275-4a3f-a907-4a73b985e5af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:52.975045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ee582d7-d275-4a3f-a907-4a73b985e5af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:52.975429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ee582d7-d275-4a3f-a907-4a73b985e5af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:52.975864 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ee582d7-d275-4a3f-a907-4a73b985e5af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:52.976566 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ee582d7-d275-4a3f-a907-4a73b985e5af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:52.977237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ee582d7-d275-4a3f-a907-4a73b985e5af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:52.977840 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ee582d7-d275-4a3f-a907-4a73b985e5af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:53.041290 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f866e3e2-5101-4f2c-853f-ce2387b3417c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:53.050877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f866e3e2-5101-4f2c-853f-ce2387b3417c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.104472 Aug 28 01:19:53.052583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1145/2064] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:52 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:53.144463 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4ee582d7-d275-4a3f-a907-4a73b985e5af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:53.154451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4ee582d7-d275-4a3f-a907-4a73b985e5af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:53.155868 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4ee582d7-d275-4a3f-a907-4a73b985e5af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:53.157096 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4ee582d7-d275-4a3f-a907-4a73b985e5af tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.193450 Aug 28 01:19:53.157662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 920/2065] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:52 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:53.965911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4c99c0f8-27b7-43ec-ad36-9667d5e9cc10 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:53.966628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4c99c0f8-27b7-43ec-ad36-9667d5e9cc10 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:53.973776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c99c0f8-27b7-43ec-ad36-9667d5e9cc10 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:53.974179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c99c0f8-27b7-43ec-ad36-9667d5e9cc10 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:53.974643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c99c0f8-27b7-43ec-ad36-9667d5e9cc10 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:53.975339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c99c0f8-27b7-43ec-ad36-9667d5e9cc10 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:53.975801 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c99c0f8-27b7-43ec-ad36-9667d5e9cc10 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:53.976226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c99c0f8-27b7-43ec-ad36-9667d5e9cc10 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:53.990595 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-948441dc-ff88-4b98-b1fd-10a44378cf07 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:53.991243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-948441dc-ff88-4b98-b1fd-10a44378cf07 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:53.998290 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-948441dc-ff88-4b98-b1fd-10a44378cf07 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:53.998762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-948441dc-ff88-4b98-b1fd-10a44378cf07 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:53.999202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-948441dc-ff88-4b98-b1fd-10a44378cf07 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:53.999895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-948441dc-ff88-4b98-b1fd-10a44378cf07 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:54.000338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-948441dc-ff88-4b98-b1fd-10a44378cf07 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:54.000747 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-948441dc-ff88-4b98-b1fd-10a44378cf07 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:54.193388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4c99c0f8-27b7-43ec-ad36-9667d5e9cc10 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:54.202113 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4c99c0f8-27b7-43ec-ad36-9667d5e9cc10 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.240041 Aug 28 01:19:54.202685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1146/2066] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:53 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:54.207623 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7b8c6861-4e99-4def-a804-197fa106361f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:54.208230 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7b8c6861-4e99-4def-a804-197fa106361f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:54.214466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b8c6861-4e99-4def-a804-197fa106361f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:54.214956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b8c6861-4e99-4def-a804-197fa106361f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:54.215387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b8c6861-4e99-4def-a804-197fa106361f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:54.216239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b8c6861-4e99-4def-a804-197fa106361f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:54.216636 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b8c6861-4e99-4def-a804-197fa106361f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:54.217087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b8c6861-4e99-4def-a804-197fa106361f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:54.223394 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-948441dc-ff88-4b98-b1fd-10a44378cf07 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:54.232049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-948441dc-ff88-4b98-b1fd-10a44378cf07 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.245329 Aug 28 01:19:54.232642 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 921/2067] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:53 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:19:54.237370 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b2b07f45-7a1f-430e-a53e-faaff2e9c8ea tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:54.237977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b2b07f45-7a1f-430e-a53e-faaff2e9c8ea tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:54.244384 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b2b07f45-7a1f-430e-a53e-faaff2e9c8ea tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:54.245937 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b2b07f45-7a1f-430e-a53e-faaff2e9c8ea tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:54.246623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b2b07f45-7a1f-430e-a53e-faaff2e9c8ea tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:54.247342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b2b07f45-7a1f-430e-a53e-faaff2e9c8ea tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:54.247788 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b2b07f45-7a1f-430e-a53e-faaff2e9c8ea tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:54.248204 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b2b07f45-7a1f-430e-a53e-faaff2e9c8ea tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:54.282960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7b8c6861-4e99-4def-a804-197fa106361f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:54.292455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7b8c6861-4e99-4def-a804-197fa106361f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.088482 Aug 28 01:19:54.293088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1147/2068] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:54 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:54.390418 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b2b07f45-7a1f-430e-a53e-faaff2e9c8ea tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:54.400481 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b2b07f45-7a1f-430e-a53e-faaff2e9c8ea tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:54.401962 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b2b07f45-7a1f-430e-a53e-faaff2e9c8ea tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:54.403177 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b2b07f45-7a1f-430e-a53e-faaff2e9c8ea tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.169583 Aug 28 01:19:54.403742 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 922/2069] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:54 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:55.226856 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d8631985-42c3-4352-8514-723bf536d50a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:55.227473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d8631985-42c3-4352-8514-723bf536d50a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:55.236652 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d8631985-42c3-4352-8514-723bf536d50a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:55.237354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d8631985-42c3-4352-8514-723bf536d50a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:55.237903 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d8631985-42c3-4352-8514-723bf536d50a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:55.238864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d8631985-42c3-4352-8514-723bf536d50a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:55.239365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d8631985-42c3-4352-8514-723bf536d50a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:55.239782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d8631985-42c3-4352-8514-723bf536d50a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:55.254325 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1b091dc2-6fc0-4e72-8465-dbced9066943 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:55.255024 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1b091dc2-6fc0-4e72-8465-dbced9066943 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:55.261692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b091dc2-6fc0-4e72-8465-dbced9066943 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:55.262235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b091dc2-6fc0-4e72-8465-dbced9066943 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:55.262667 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b091dc2-6fc0-4e72-8465-dbced9066943 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:55.263347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b091dc2-6fc0-4e72-8465-dbced9066943 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:55.263786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b091dc2-6fc0-4e72-8465-dbced9066943 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:55.264207 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1b091dc2-6fc0-4e72-8465-dbced9066943 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:55.435234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d8631985-42c3-4352-8514-723bf536d50a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:55.443136 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1b091dc2-6fc0-4e72-8465-dbced9066943 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:55.444983 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d8631985-42c3-4352-8514-723bf536d50a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.224195 Aug 28 01:19:55.445610 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1148/2070] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:55 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:55.450164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2213c3f3-751f-4439-afdd-f964f97917c9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:55.450763 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2213c3f3-751f-4439-afdd-f964f97917c9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:55.452358 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1b091dc2-6fc0-4e72-8465-dbced9066943 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.201886 Aug 28 01:19:55.458167 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 923/2071] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:55 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:55.458338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2213c3f3-751f-4439-afdd-f964f97917c9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:55.458338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2213c3f3-751f-4439-afdd-f964f97917c9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:55.458338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2213c3f3-751f-4439-afdd-f964f97917c9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:55.458338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2213c3f3-751f-4439-afdd-f964f97917c9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:55.458798 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2213c3f3-751f-4439-afdd-f964f97917c9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:55.459213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2213c3f3-751f-4439-afdd-f964f97917c9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:55.462605 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a1142d22-f782-4ed1-a935-a687752b4608 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:55.463283 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a1142d22-f782-4ed1-a935-a687752b4608 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:55.469345 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a1142d22-f782-4ed1-a935-a687752b4608 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:55.469829 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a1142d22-f782-4ed1-a935-a687752b4608 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:55.470254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a1142d22-f782-4ed1-a935-a687752b4608 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:55.470943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a1142d22-f782-4ed1-a935-a687752b4608 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:55.471382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a1142d22-f782-4ed1-a935-a687752b4608 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:55.471811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a1142d22-f782-4ed1-a935-a687752b4608 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:55.526935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2213c3f3-751f-4439-afdd-f964f97917c9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:55.535536 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2213c3f3-751f-4439-afdd-f964f97917c9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.088905 Aug 28 01:19:55.536198 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1149/2072] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:55 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:55.633825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a1142d22-f782-4ed1-a935-a687752b4608 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:55.647460 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a1142d22-f782-4ed1-a935-a687752b4608 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:55.649301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a1142d22-f782-4ed1-a935-a687752b4608 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:55.651199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a1142d22-f782-4ed1-a935-a687752b4608 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.196563 Aug 28 01:19:55.652062 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 924/2073] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:55 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:56.466952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cff4d206-a539-4bef-acdb-3372bd7fb89d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:56.467585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cff4d206-a539-4bef-acdb-3372bd7fb89d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:56.474009 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f38a2fcb-af19-4c4a-831d-d832d5f08c7d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:56.474646 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f38a2fcb-af19-4c4a-831d-d832d5f08c7d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:56.474931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cff4d206-a539-4bef-acdb-3372bd7fb89d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:56.475666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cff4d206-a539-4bef-acdb-3372bd7fb89d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:56.476099 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cff4d206-a539-4bef-acdb-3372bd7fb89d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:56.476799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cff4d206-a539-4bef-acdb-3372bd7fb89d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:56.477270 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cff4d206-a539-4bef-acdb-3372bd7fb89d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:56.477691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cff4d206-a539-4bef-acdb-3372bd7fb89d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:56.481655 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f38a2fcb-af19-4c4a-831d-d832d5f08c7d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:56.482137 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f38a2fcb-af19-4c4a-831d-d832d5f08c7d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:56.482569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f38a2fcb-af19-4c4a-831d-d832d5f08c7d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:56.483265 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f38a2fcb-af19-4c4a-831d-d832d5f08c7d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:56.483714 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f38a2fcb-af19-4c4a-831d-d832d5f08c7d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:56.484136 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f38a2fcb-af19-4c4a-831d-d832d5f08c7d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:56.676035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-cff4d206-a539-4bef-acdb-3372bd7fb89d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:56.680089 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f38a2fcb-af19-4c4a-831d-d832d5f08c7d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:56.684549 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cff4d206-a539-4bef-acdb-3372bd7fb89d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.221452 Aug 28 01:19:56.685141 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1150/2074] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:56 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:56.688503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f38a2fcb-af19-4c4a-831d-d832d5f08c7d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.218326 Aug 28 01:19:56.689241 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 925/2075] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:56 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:56.693991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4513a921-aeea-4d20-86df-d5e9a87392b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:56.694583 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4513a921-aeea-4d20-86df-d5e9a87392b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:56.700372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4513a921-aeea-4d20-86df-d5e9a87392b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:56.700851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4513a921-aeea-4d20-86df-d5e9a87392b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:56.701240 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d913d901-0c09-4d2b-8938-445df1924233 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:56.701379 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4513a921-aeea-4d20-86df-d5e9a87392b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:56.701976 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d913d901-0c09-4d2b-8938-445df1924233 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:56.702108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4513a921-aeea-4d20-86df-d5e9a87392b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:56.702413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4513a921-aeea-4d20-86df-d5e9a87392b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:56.702827 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4513a921-aeea-4d20-86df-d5e9a87392b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:56.708211 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d913d901-0c09-4d2b-8938-445df1924233 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:56.708693 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d913d901-0c09-4d2b-8938-445df1924233 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:56.709184 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d913d901-0c09-4d2b-8938-445df1924233 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:56.709874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d913d901-0c09-4d2b-8938-445df1924233 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:56.710322 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d913d901-0c09-4d2b-8938-445df1924233 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:56.710745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d913d901-0c09-4d2b-8938-445df1924233 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:56.790654 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d913d901-0c09-4d2b-8938-445df1924233 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:56.799080 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d913d901-0c09-4d2b-8938-445df1924233 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.112909 Aug 28 01:19:56.799656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1151/2076] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:56 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:56.859282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4513a921-aeea-4d20-86df-d5e9a87392b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:56.868127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4513a921-aeea-4d20-86df-d5e9a87392b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:56.869570 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4513a921-aeea-4d20-86df-d5e9a87392b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:56.870749 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4513a921-aeea-4d20-86df-d5e9a87392b9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.180631 Aug 28 01:19:56.871306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 926/2077] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:56 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:57.706969 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d47ef60a-40e6-477d-8319-76b24a50de1b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:57.707573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d47ef60a-40e6-477d-8319-76b24a50de1b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:57.709773 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4baf1a37-9302-4fa6-9b52-888eff9458f0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:57.710380 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4baf1a37-9302-4fa6-9b52-888eff9458f0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:57.714282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d47ef60a-40e6-477d-8319-76b24a50de1b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:57.714757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d47ef60a-40e6-477d-8319-76b24a50de1b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:57.715254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d47ef60a-40e6-477d-8319-76b24a50de1b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:57.715990 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d47ef60a-40e6-477d-8319-76b24a50de1b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:57.716443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d47ef60a-40e6-477d-8319-76b24a50de1b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:57.716846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d47ef60a-40e6-477d-8319-76b24a50de1b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:57.717304 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4baf1a37-9302-4fa6-9b52-888eff9458f0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:57.717784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4baf1a37-9302-4fa6-9b52-888eff9458f0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:57.718212 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4baf1a37-9302-4fa6-9b52-888eff9458f0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:57.718904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4baf1a37-9302-4fa6-9b52-888eff9458f0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:57.719347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4baf1a37-9302-4fa6-9b52-888eff9458f0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:57.719763 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4baf1a37-9302-4fa6-9b52-888eff9458f0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:57.735811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:19:57.927313 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d47ef60a-40e6-477d-8319-76b24a50de1b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:57.932370 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4baf1a37-9302-4fa6-9b52-888eff9458f0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:57.940877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d47ef60a-40e6-477d-8319-76b24a50de1b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.237514 Aug 28 01:19:57.941886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1152/2078] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:57 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:57.945008 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4baf1a37-9302-4fa6-9b52-888eff9458f0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.238995 Aug 28 01:19:57.945788 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 927/2079] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:57 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:57.946982 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-286711d4-78c9-41b8-a22e-c2bb281e6090 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:57.947582 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-286711d4-78c9-41b8-a22e-c2bb281e6090 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:57.950018 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-936dc59e-7b5a-45ef-9416-439d93bf131a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:57.950614 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-936dc59e-7b5a-45ef-9416-439d93bf131a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:57.953662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-286711d4-78c9-41b8-a22e-c2bb281e6090 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:57.954150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-286711d4-78c9-41b8-a22e-c2bb281e6090 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:57.954567 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-286711d4-78c9-41b8-a22e-c2bb281e6090 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:57.955259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-286711d4-78c9-41b8-a22e-c2bb281e6090 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:57.955724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-286711d4-78c9-41b8-a22e-c2bb281e6090 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:57.956150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-286711d4-78c9-41b8-a22e-c2bb281e6090 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:57.956407 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-936dc59e-7b5a-45ef-9416-439d93bf131a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:57.956879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-936dc59e-7b5a-45ef-9416-439d93bf131a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:57.957414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-936dc59e-7b5a-45ef-9416-439d93bf131a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:57.958020 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-936dc59e-7b5a-45ef-9416-439d93bf131a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:57.958444 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-936dc59e-7b5a-45ef-9416-439d93bf131a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:57.958872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-936dc59e-7b5a-45ef-9416-439d93bf131a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:58.041139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-286711d4-78c9-41b8-a22e-c2bb281e6090 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:58.052292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-286711d4-78c9-41b8-a22e-c2bb281e6090 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.108130 Aug 28 01:19:58.052797 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1153/2080] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:57 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:58.122798 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-936dc59e-7b5a-45ef-9416-439d93bf131a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:58.132863 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-936dc59e-7b5a-45ef-9416-439d93bf131a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:58.135055 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-936dc59e-7b5a-45ef-9416-439d93bf131a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:58.136715 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-936dc59e-7b5a-45ef-9416-439d93bf131a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.189997 Aug 28 01:19:58.137530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 928/2081] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:57 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:19:58.967190 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1192dbf5-2a2d-46b5-a063-ba21f0efd8ef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:58.967782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1192dbf5-2a2d-46b5-a063-ba21f0efd8ef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:58.975455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1192dbf5-2a2d-46b5-a063-ba21f0efd8ef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:58.976045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1192dbf5-2a2d-46b5-a063-ba21f0efd8ef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:58.976476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1192dbf5-2a2d-46b5-a063-ba21f0efd8ef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:58.977157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e6154da7-e04c-4735-b539-67e85b9e3622 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:58.977666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1192dbf5-2a2d-46b5-a063-ba21f0efd8ef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:58.977666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1192dbf5-2a2d-46b5-a063-ba21f0efd8ef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:58.977863 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e6154da7-e04c-4735-b539-67e85b9e3622 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:58.978063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1192dbf5-2a2d-46b5-a063-ba21f0efd8ef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:58.984566 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6154da7-e04c-4735-b539-67e85b9e3622 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:58.985074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6154da7-e04c-4735-b539-67e85b9e3622 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:58.985502 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6154da7-e04c-4735-b539-67e85b9e3622 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:58.986199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6154da7-e04c-4735-b539-67e85b9e3622 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:58.986644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6154da7-e04c-4735-b539-67e85b9e3622 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:58.987061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6154da7-e04c-4735-b539-67e85b9e3622 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:59.175116 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1192dbf5-2a2d-46b5-a063-ba21f0efd8ef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:59.178355 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e6154da7-e04c-4735-b539-67e85b9e3622 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:59.183696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1192dbf5-2a2d-46b5-a063-ba21f0efd8ef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.220543 Aug 28 01:19:59.184650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1154/2082] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:58 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:59.186943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e6154da7-e04c-4735-b539-67e85b9e3622 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.213845 Aug 28 01:19:59.201014 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 929/2083] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:58 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:59.201246 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9a1634e0-cc82-4122-9ffa-152be1bf1839 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:59.201246 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9a1634e0-cc82-4122-9ffa-152be1bf1839 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:59.201497 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7aaf5225-e210-4c7e-8ae2-cf8cfa8d3cc1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:19:59.201497 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7aaf5225-e210-4c7e-8ae2-cf8cfa8d3cc1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:19:59.201701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a1634e0-cc82-4122-9ffa-152be1bf1839 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:59.201701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a1634e0-cc82-4122-9ffa-152be1bf1839 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:59.201701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a1634e0-cc82-4122-9ffa-152be1bf1839 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:59.201701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a1634e0-cc82-4122-9ffa-152be1bf1839 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:59.202089 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a1634e0-cc82-4122-9ffa-152be1bf1839 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:59.202089 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a1634e0-cc82-4122-9ffa-152be1bf1839 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:59.202272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7aaf5225-e210-4c7e-8ae2-cf8cfa8d3cc1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:59.202272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7aaf5225-e210-4c7e-8ae2-cf8cfa8d3cc1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:59.202272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7aaf5225-e210-4c7e-8ae2-cf8cfa8d3cc1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:59.202272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7aaf5225-e210-4c7e-8ae2-cf8cfa8d3cc1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:19:59.202637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7aaf5225-e210-4c7e-8ae2-cf8cfa8d3cc1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:19:59.202637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7aaf5225-e210-4c7e-8ae2-cf8cfa8d3cc1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:19:59.287452 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9a1634e0-cc82-4122-9ffa-152be1bf1839 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:59.295955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9a1634e0-cc82-4122-9ffa-152be1bf1839 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.110117 Aug 28 01:19:59.296547 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1155/2084] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:19:59 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:19:59.356237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7aaf5225-e210-4c7e-8ae2-cf8cfa8d3cc1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:59.365333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7aaf5225-e210-4c7e-8ae2-cf8cfa8d3cc1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:59.366805 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7aaf5225-e210-4c7e-8ae2-cf8cfa8d3cc1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:19:59.367961 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7aaf5225-e210-4c7e-8ae2-cf8cfa8d3cc1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.179592 Aug 28 01:19:59.368520 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 930/2085] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:19:59 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:00.207075 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a17a26b1-2d16-432a-b852-8a464eec87bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:00.207696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a17a26b1-2d16-432a-b852-8a464eec87bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:00.208334 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bc8ce0d6-284b-41ef-b829-a16b03ff3751 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:00.209382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bc8ce0d6-284b-41ef-b829-a16b03ff3751 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:00.215794 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a17a26b1-2d16-432a-b852-8a464eec87bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:00.216283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a17a26b1-2d16-432a-b852-8a464eec87bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:00.216469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc8ce0d6-284b-41ef-b829-a16b03ff3751 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:00.216741 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a17a26b1-2d16-432a-b852-8a464eec87bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:00.216975 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc8ce0d6-284b-41ef-b829-a16b03ff3751 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:00.217446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc8ce0d6-284b-41ef-b829-a16b03ff3751 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:00.217580 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a17a26b1-2d16-432a-b852-8a464eec87bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:00.217909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a17a26b1-2d16-432a-b852-8a464eec87bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:00.218120 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc8ce0d6-284b-41ef-b829-a16b03ff3751 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:00.218338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a17a26b1-2d16-432a-b852-8a464eec87bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:00.218573 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc8ce0d6-284b-41ef-b829-a16b03ff3751 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:00.218993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc8ce0d6-284b-41ef-b829-a16b03ff3751 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:00.220992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:20:00.413471 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bc8ce0d6-284b-41ef-b829-a16b03ff3751 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:00.417250 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a17a26b1-2d16-432a-b852-8a464eec87bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:00.426001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bc8ce0d6-284b-41ef-b829-a16b03ff3751 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.221180 Aug 28 01:20:00.426902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 931/2086] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:00 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:20:00.429886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a17a26b1-2d16-432a-b852-8a464eec87bf tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.226383 Aug 28 01:20:00.430893 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1156/2087] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:00 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:00.433380 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9aba1127-395c-4a1a-af01-77433b4107df tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:00.434272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9aba1127-395c-4a1a-af01-77433b4107df tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:00.437142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-45a881c8-b85c-44c4-a0c0-661b2f4c5f47 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:00.437942 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-45a881c8-b85c-44c4-a0c0-661b2f4c5f47 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:00.443007 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9aba1127-395c-4a1a-af01-77433b4107df tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:00.443902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9aba1127-395c-4a1a-af01-77433b4107df tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:00.444717 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9aba1127-395c-4a1a-af01-77433b4107df tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:00.445565 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9aba1127-395c-4a1a-af01-77433b4107df tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:00.446045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45a881c8-b85c-44c4-a0c0-661b2f4c5f47 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:00.446176 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9aba1127-395c-4a1a-af01-77433b4107df tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:00.446530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45a881c8-b85c-44c4-a0c0-661b2f4c5f47 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:00.446787 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9aba1127-395c-4a1a-af01-77433b4107df tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:00.446965 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45a881c8-b85c-44c4-a0c0-661b2f4c5f47 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:00.447658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45a881c8-b85c-44c4-a0c0-661b2f4c5f47 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:00.448110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45a881c8-b85c-44c4-a0c0-661b2f4c5f47 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:00.448530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45a881c8-b85c-44c4-a0c0-661b2f4c5f47 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:00.537578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9aba1127-395c-4a1a-af01-77433b4107df tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:00.545960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9aba1127-395c-4a1a-af01-77433b4107df tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.117984 Aug 28 01:20:00.546534 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 932/2088] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:00 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:00.613839 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-45a881c8-b85c-44c4-a0c0-661b2f4c5f47 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:00.622865 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-45a881c8-b85c-44c4-a0c0-661b2f4c5f47 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:00.624290 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-45a881c8-b85c-44c4-a0c0-661b2f4c5f47 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:00.625484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-45a881c8-b85c-44c4-a0c0-661b2f4c5f47 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.193729 Aug 28 01:20:00.626037 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1157/2089] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:00 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:01.461099 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-601ddeba-51e4-40e5-af0a-7c173271f761 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:01.461786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-601ddeba-51e4-40e5-af0a-7c173271f761 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:01.464167 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=98 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:20:01.466869 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-af69cfed-410c-4807-9b96-c22d58aa4515 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:01.467499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-af69cfed-410c-4807-9b96-c22d58aa4515 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:01.469539 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-601ddeba-51e4-40e5-af0a-7c173271f761 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:01.470395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-601ddeba-51e4-40e5-af0a-7c173271f761 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:01.470556 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-601ddeba-51e4-40e5-af0a-7c173271f761 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:01.471133 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-601ddeba-51e4-40e5-af0a-7c173271f761 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:01.471569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-601ddeba-51e4-40e5-af0a-7c173271f761 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:01.471998 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-601ddeba-51e4-40e5-af0a-7c173271f761 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:01.473610 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-af69cfed-410c-4807-9b96-c22d58aa4515 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:01.474125 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-af69cfed-410c-4807-9b96-c22d58aa4515 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:01.474708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-af69cfed-410c-4807-9b96-c22d58aa4515 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:01.475232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-af69cfed-410c-4807-9b96-c22d58aa4515 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:01.475678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-af69cfed-410c-4807-9b96-c22d58aa4515 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:01.476099 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-af69cfed-410c-4807-9b96-c22d58aa4515 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:01.665399 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-601ddeba-51e4-40e5-af0a-7c173271f761 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:01.668409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-af69cfed-410c-4807-9b96-c22d58aa4515 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:01.673954 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-601ddeba-51e4-40e5-af0a-7c173271f761 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.216793 Aug 28 01:20:01.674630 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 933/2090] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:01 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:01.676688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-af69cfed-410c-4807-9b96-c22d58aa4515 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.214114 Aug 28 01:20:01.677514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1158/2091] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:01 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:01.681968 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5205915c-dcd0-4cad-be68-477d1cad39a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:01.682617 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5205915c-dcd0-4cad-be68-477d1cad39a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:01.688377 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5205915c-dcd0-4cad-be68-477d1cad39a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:01.688858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5205915c-dcd0-4cad-be68-477d1cad39a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:01.689310 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c3bfdd56-52db-4149-a583-e310f2b9b067 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:01.689452 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5205915c-dcd0-4cad-be68-477d1cad39a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:01.689991 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5205915c-dcd0-4cad-be68-477d1cad39a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:01.690108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c3bfdd56-52db-4149-a583-e310f2b9b067 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:01.690429 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5205915c-dcd0-4cad-be68-477d1cad39a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:01.690842 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5205915c-dcd0-4cad-be68-477d1cad39a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:01.696797 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3bfdd56-52db-4149-a583-e310f2b9b067 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:01.697464 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3bfdd56-52db-4149-a583-e310f2b9b067 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:01.697960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3bfdd56-52db-4149-a583-e310f2b9b067 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:01.698710 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3bfdd56-52db-4149-a583-e310f2b9b067 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:01.699160 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3bfdd56-52db-4149-a583-e310f2b9b067 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:01.699582 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3bfdd56-52db-4149-a583-e310f2b9b067 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:01.789657 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c3bfdd56-52db-4149-a583-e310f2b9b067 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:01.798118 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c3bfdd56-52db-4149-a583-e310f2b9b067 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.122419 Aug 28 01:20:01.798683 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 934/2092] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:01 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:01.846718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5205915c-dcd0-4cad-be68-477d1cad39a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:01.855532 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5205915c-dcd0-4cad-be68-477d1cad39a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:01.856953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5205915c-dcd0-4cad-be68-477d1cad39a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:01.858085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5205915c-dcd0-4cad-be68-477d1cad39a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.179845 Aug 28 01:20:01.858672 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1159/2093] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:01 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:02.696602 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-af0a46f4-b0e4-44f8-bdd6-8860622b0712 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:02.697284 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-af0a46f4-b0e4-44f8-bdd6-8860622b0712 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:02.697391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a7bdee20-c35b-4559-ad90-3edcc1157d27 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:02.697991 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a7bdee20-c35b-4559-ad90-3edcc1157d27 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:02.700294 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=98 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:20:02.704267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af0a46f4-b0e4-44f8-bdd6-8860622b0712 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:02.704750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af0a46f4-b0e4-44f8-bdd6-8860622b0712 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:02.705221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af0a46f4-b0e4-44f8-bdd6-8860622b0712 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:02.705907 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af0a46f4-b0e4-44f8-bdd6-8860622b0712 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:02.706044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7bdee20-c35b-4559-ad90-3edcc1157d27 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:02.706363 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af0a46f4-b0e4-44f8-bdd6-8860622b0712 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:02.706484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7bdee20-c35b-4559-ad90-3edcc1157d27 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:02.706778 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af0a46f4-b0e4-44f8-bdd6-8860622b0712 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:02.706894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7bdee20-c35b-4559-ad90-3edcc1157d27 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:02.707588 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7bdee20-c35b-4559-ad90-3edcc1157d27 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:02.708035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7bdee20-c35b-4559-ad90-3edcc1157d27 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:02.708446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7bdee20-c35b-4559-ad90-3edcc1157d27 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:02.905407 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-af0a46f4-b0e4-44f8-bdd6-8860622b0712 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:02.908281 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a7bdee20-c35b-4559-ad90-3edcc1157d27 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:02.914188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-af0a46f4-b0e4-44f8-bdd6-8860622b0712 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.221825 Aug 28 01:20:02.914753 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 935/2094] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:02 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:02.916598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a7bdee20-c35b-4559-ad90-3edcc1157d27 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.223037 Aug 28 01:20:02.917331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1160/2095] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:02 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:02.919853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a744bc81-2224-4b3a-b8f8-bd567ea5757b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:02.920465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a744bc81-2224-4b3a-b8f8-bd567ea5757b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:02.928010 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a744bc81-2224-4b3a-b8f8-bd567ea5757b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:02.928497 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a744bc81-2224-4b3a-b8f8-bd567ea5757b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:02.928934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a744bc81-2224-4b3a-b8f8-bd567ea5757b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:02.929755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a744bc81-2224-4b3a-b8f8-bd567ea5757b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:02.930323 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a744bc81-2224-4b3a-b8f8-bd567ea5757b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:02.930846 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a744bc81-2224-4b3a-b8f8-bd567ea5757b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:02.931180 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ed2caa57-8794-4b69-b6a8-fce95ea2b55a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:02.931821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ed2caa57-8794-4b69-b6a8-fce95ea2b55a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:02.938046 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed2caa57-8794-4b69-b6a8-fce95ea2b55a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:02.938303 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed2caa57-8794-4b69-b6a8-fce95ea2b55a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:02.938739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed2caa57-8794-4b69-b6a8-fce95ea2b55a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:02.939470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed2caa57-8794-4b69-b6a8-fce95ea2b55a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:02.940174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed2caa57-8794-4b69-b6a8-fce95ea2b55a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:02.940641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ed2caa57-8794-4b69-b6a8-fce95ea2b55a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:03.003106 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a744bc81-2224-4b3a-b8f8-bd567ea5757b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:03.011592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a744bc81-2224-4b3a-b8f8-bd567ea5757b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.095832 Aug 28 01:20:03.012158 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 936/2096] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:02 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:03.091544 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ed2caa57-8794-4b69-b6a8-fce95ea2b55a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:03.105110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ed2caa57-8794-4b69-b6a8-fce95ea2b55a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:03.107420 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ed2caa57-8794-4b69-b6a8-fce95ea2b55a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:03.109191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ed2caa57-8794-4b69-b6a8-fce95ea2b55a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.183005 Aug 28 01:20:03.109824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1161/2097] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:02 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:03.938502 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8e1a7cce-264d-4a1f-b0cb-057eb2bbe375 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:03.939227 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8e1a7cce-264d-4a1f-b0cb-057eb2bbe375 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:03.939955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d5d93db1-2a55-4655-a65a-942ac47c7be3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:03.940672 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d5d93db1-2a55-4655-a65a-942ac47c7be3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:03.946966 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e1a7cce-264d-4a1f-b0cb-057eb2bbe375 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:03.947535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e1a7cce-264d-4a1f-b0cb-057eb2bbe375 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:03.948037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e1a7cce-264d-4a1f-b0cb-057eb2bbe375 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:03.948840 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e1a7cce-264d-4a1f-b0cb-057eb2bbe375 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:03.949062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5d93db1-2a55-4655-a65a-942ac47c7be3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:03.949399 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e1a7cce-264d-4a1f-b0cb-057eb2bbe375 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:03.949579 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5d93db1-2a55-4655-a65a-942ac47c7be3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:03.949884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e1a7cce-264d-4a1f-b0cb-057eb2bbe375 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:03.950063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5d93db1-2a55-4655-a65a-942ac47c7be3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:03.950821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5d93db1-2a55-4655-a65a-942ac47c7be3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:03.951350 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5d93db1-2a55-4655-a65a-942ac47c7be3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:03.951868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5d93db1-2a55-4655-a65a-942ac47c7be3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:04.145780 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8e1a7cce-264d-4a1f-b0cb-057eb2bbe375 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:04.148804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d5d93db1-2a55-4655-a65a-942ac47c7be3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:04.154453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8e1a7cce-264d-4a1f-b0cb-057eb2bbe375 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.221150 Aug 28 01:20:04.154999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 937/2098] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:03 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:04.158600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d5d93db1-2a55-4655-a65a-942ac47c7be3 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.223067 Aug 28 01:20:04.159578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1162/2099] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:03 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:04.159704 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-909bf592-96cd-4e29-b411-1360d8d5a407 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:04.160251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-909bf592-96cd-4e29-b411-1360d8d5a407 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:04.164552 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9b4c5376-e6f3-4a27-9aeb-17b010200365 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:04.165660 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9b4c5376-e6f3-4a27-9aeb-17b010200365 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:04.171579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-909bf592-96cd-4e29-b411-1360d8d5a407 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:04.172076 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-909bf592-96cd-4e29-b411-1360d8d5a407 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:04.172514 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-909bf592-96cd-4e29-b411-1360d8d5a407 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:04.173035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b4c5376-e6f3-4a27-9aeb-17b010200365 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:04.173247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-909bf592-96cd-4e29-b411-1360d8d5a407 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:04.173642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b4c5376-e6f3-4a27-9aeb-17b010200365 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:04.173752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-909bf592-96cd-4e29-b411-1360d8d5a407 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:04.174108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-909bf592-96cd-4e29-b411-1360d8d5a407 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:04.174214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b4c5376-e6f3-4a27-9aeb-17b010200365 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:04.174975 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b4c5376-e6f3-4a27-9aeb-17b010200365 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:04.175526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b4c5376-e6f3-4a27-9aeb-17b010200365 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:04.175943 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b4c5376-e6f3-4a27-9aeb-17b010200365 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:04.265502 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-909bf592-96cd-4e29-b411-1360d8d5a407 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:04.275282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-909bf592-96cd-4e29-b411-1360d8d5a407 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.119051 Aug 28 01:20:04.275898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 938/2100] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:04 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:04.334975 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9b4c5376-e6f3-4a27-9aeb-17b010200365 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:04.348515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9b4c5376-e6f3-4a27-9aeb-17b010200365 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:04.350744 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9b4c5376-e6f3-4a27-9aeb-17b010200365 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:04.352435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9b4c5376-e6f3-4a27-9aeb-17b010200365 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.191119 Aug 28 01:20:04.353144 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1163/2101] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:04 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:05.176167 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2d64990d-26ba-4c64-b59f-6170fc84e454 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:05.176787 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2d64990d-26ba-4c64-b59f-6170fc84e454 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:05.181005 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ab790571-fe43-418e-beed-4cc9cbe989bd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:05.181651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ab790571-fe43-418e-beed-4cc9cbe989bd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:05.184084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d64990d-26ba-4c64-b59f-6170fc84e454 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:05.184576 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d64990d-26ba-4c64-b59f-6170fc84e454 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:05.184995 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d64990d-26ba-4c64-b59f-6170fc84e454 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:05.185731 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d64990d-26ba-4c64-b59f-6170fc84e454 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:05.186183 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d64990d-26ba-4c64-b59f-6170fc84e454 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:05.186721 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d64990d-26ba-4c64-b59f-6170fc84e454 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:05.188304 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab790571-fe43-418e-beed-4cc9cbe989bd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:05.188787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab790571-fe43-418e-beed-4cc9cbe989bd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:05.189248 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab790571-fe43-418e-beed-4cc9cbe989bd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:05.189982 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab790571-fe43-418e-beed-4cc9cbe989bd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:05.190433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab790571-fe43-418e-beed-4cc9cbe989bd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:05.190863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab790571-fe43-418e-beed-4cc9cbe989bd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:05.383404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2d64990d-26ba-4c64-b59f-6170fc84e454 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:05.386969 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ab790571-fe43-418e-beed-4cc9cbe989bd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:05.392209 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2d64990d-26ba-4c64-b59f-6170fc84e454 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.219965 Aug 28 01:20:05.392800 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 939/2102] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:05 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:05.395653 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ab790571-fe43-418e-beed-4cc9cbe989bd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.218737 Aug 28 01:20:05.396397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1164/2103] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:05 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:05.397345 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3ca08880-488d-4deb-9414-737c1fb82717 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:05.397950 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3ca08880-488d-4deb-9414-737c1fb82717 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:05.406065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ca08880-488d-4deb-9414-737c1fb82717 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:05.406549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ca08880-488d-4deb-9414-737c1fb82717 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:05.406980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ca08880-488d-4deb-9414-737c1fb82717 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:05.408225 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ca08880-488d-4deb-9414-737c1fb82717 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:05.408717 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ca08880-488d-4deb-9414-737c1fb82717 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:05.409348 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-13eb0ee8-7213-428d-b8ca-5a331404168d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:05.409477 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ca08880-488d-4deb-9414-737c1fb82717 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:05.409589 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-13eb0ee8-7213-428d-b8ca-5a331404168d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:05.415916 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-13eb0ee8-7213-428d-b8ca-5a331404168d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:05.416399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-13eb0ee8-7213-428d-b8ca-5a331404168d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:05.416827 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-13eb0ee8-7213-428d-b8ca-5a331404168d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:05.417578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-13eb0ee8-7213-428d-b8ca-5a331404168d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:05.418023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-13eb0ee8-7213-428d-b8ca-5a331404168d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:05.418437 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-13eb0ee8-7213-428d-b8ca-5a331404168d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:05.482436 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3ca08880-488d-4deb-9414-737c1fb82717 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:05.490968 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3ca08880-488d-4deb-9414-737c1fb82717 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.097174 Aug 28 01:20:05.491547 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 940/2104] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:05 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:05.570711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-13eb0ee8-7213-428d-b8ca-5a331404168d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:05.579458 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-13eb0ee8-7213-428d-b8ca-5a331404168d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:05.580870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-13eb0ee8-7213-428d-b8ca-5a331404168d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:05.582041 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-13eb0ee8-7213-428d-b8ca-5a331404168d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.177555 Aug 28 01:20:05.582596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1165/2105] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:05 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:06.414194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1c20a384-d0bd-44fa-9d04-5b12981296e0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:06.414821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1c20a384-d0bd-44fa-9d04-5b12981296e0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:06.417737 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4346ee76-eedc-4c42-95c4-9edc3632dafa tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:06.418534 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4346ee76-eedc-4c42-95c4-9edc3632dafa tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:06.422455 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c20a384-d0bd-44fa-9d04-5b12981296e0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:06.423215 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c20a384-d0bd-44fa-9d04-5b12981296e0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:06.423514 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c20a384-d0bd-44fa-9d04-5b12981296e0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:06.424184 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c20a384-d0bd-44fa-9d04-5b12981296e0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:06.424639 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c20a384-d0bd-44fa-9d04-5b12981296e0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:06.425082 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c20a384-d0bd-44fa-9d04-5b12981296e0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:06.425801 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4346ee76-eedc-4c42-95c4-9edc3632dafa tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:06.426435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4346ee76-eedc-4c42-95c4-9edc3632dafa tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:06.426937 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4346ee76-eedc-4c42-95c4-9edc3632dafa tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:06.427924 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4346ee76-eedc-4c42-95c4-9edc3632dafa tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:06.428588 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4346ee76-eedc-4c42-95c4-9edc3632dafa tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:06.429364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4346ee76-eedc-4c42-95c4-9edc3632dafa tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:06.631027 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4346ee76-eedc-4c42-95c4-9edc3632dafa tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:06.633659 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1c20a384-d0bd-44fa-9d04-5b12981296e0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:06.639442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4346ee76-eedc-4c42-95c4-9edc3632dafa tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.225963 Aug 28 01:20:06.640024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1166/2106] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:06 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:06.642373 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1c20a384-d0bd-44fa-9d04-5b12981296e0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.232339 Aug 28 01:20:06.650008 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 941/2107] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:06 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:06.650233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-423ea652-33a4-4a55-8dce-efda9bdb532e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:06.650233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-423ea652-33a4-4a55-8dce-efda9bdb532e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:06.650443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a25401f5-de4d-4a5c-be59-ba765a6e928b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:06.650443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a25401f5-de4d-4a5c-be59-ba765a6e928b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:06.651339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-423ea652-33a4-4a55-8dce-efda9bdb532e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:06.651652 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-423ea652-33a4-4a55-8dce-efda9bdb532e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:06.652108 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-423ea652-33a4-4a55-8dce-efda9bdb532e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:06.652779 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-423ea652-33a4-4a55-8dce-efda9bdb532e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:06.653302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-423ea652-33a4-4a55-8dce-efda9bdb532e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:06.653689 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-423ea652-33a4-4a55-8dce-efda9bdb532e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:06.656663 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a25401f5-de4d-4a5c-be59-ba765a6e928b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:06.657182 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a25401f5-de4d-4a5c-be59-ba765a6e928b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:06.657612 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a25401f5-de4d-4a5c-be59-ba765a6e928b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:06.658307 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a25401f5-de4d-4a5c-be59-ba765a6e928b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:06.658751 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a25401f5-de4d-4a5c-be59-ba765a6e928b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:06.659210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a25401f5-de4d-4a5c-be59-ba765a6e928b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:06.738456 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-423ea652-33a4-4a55-8dce-efda9bdb532e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:06.747083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-423ea652-33a4-4a55-8dce-efda9bdb532e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.106026 Aug 28 01:20:06.747662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1167/2108] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:06 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:06.806151 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a25401f5-de4d-4a5c-be59-ba765a6e928b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:06.815530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a25401f5-de4d-4a5c-be59-ba765a6e928b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:06.817417 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a25401f5-de4d-4a5c-be59-ba765a6e928b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:06.818587 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a25401f5-de4d-4a5c-be59-ba765a6e928b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.174339 Aug 28 01:20:06.819135 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 942/2109] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:06 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:07.660031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-386f83b9-6a70-4f71-a46f-14a0cd10f2c2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:07.660646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-386f83b9-6a70-4f71-a46f-14a0cd10f2c2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:07.663603 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e141ca16-6ff5-4105-bbb5-d5efdfa40511 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:07.664199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e141ca16-6ff5-4105-bbb5-d5efdfa40511 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:07.667559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-386f83b9-6a70-4f71-a46f-14a0cd10f2c2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:07.668041 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-386f83b9-6a70-4f71-a46f-14a0cd10f2c2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:07.668467 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-386f83b9-6a70-4f71-a46f-14a0cd10f2c2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:07.669224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-386f83b9-6a70-4f71-a46f-14a0cd10f2c2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:07.669665 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-386f83b9-6a70-4f71-a46f-14a0cd10f2c2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:07.670119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-386f83b9-6a70-4f71-a46f-14a0cd10f2c2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:07.670959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e141ca16-6ff5-4105-bbb5-d5efdfa40511 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:07.671442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e141ca16-6ff5-4105-bbb5-d5efdfa40511 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:07.671873 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e141ca16-6ff5-4105-bbb5-d5efdfa40511 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:07.672566 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e141ca16-6ff5-4105-bbb5-d5efdfa40511 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:07.673152 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e141ca16-6ff5-4105-bbb5-d5efdfa40511 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:07.673548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e141ca16-6ff5-4105-bbb5-d5efdfa40511 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:07.869350 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-386f83b9-6a70-4f71-a46f-14a0cd10f2c2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:07.872603 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e141ca16-6ff5-4105-bbb5-d5efdfa40511 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:07.877892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-386f83b9-6a70-4f71-a46f-14a0cd10f2c2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.221741 Aug 28 01:20:07.878442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1168/2110] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:07 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:07.881228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e141ca16-6ff5-4105-bbb5-d5efdfa40511 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.221493 Aug 28 01:20:07.892592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 943/2111] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:07 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:07.892828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c6fb5e16-ddef-4b70-af97-9b506d1859c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:07.892828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c6fb5e16-ddef-4b70-af97-9b506d1859c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:07.893045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ab268925-ddbe-407a-b843-388ad280bf1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:07.893045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ab268925-ddbe-407a-b843-388ad280bf1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:07.893242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6fb5e16-ddef-4b70-af97-9b506d1859c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:07.893242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6fb5e16-ddef-4b70-af97-9b506d1859c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:07.893242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6fb5e16-ddef-4b70-af97-9b506d1859c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:07.893242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6fb5e16-ddef-4b70-af97-9b506d1859c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:07.893669 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6fb5e16-ddef-4b70-af97-9b506d1859c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:07.893669 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6fb5e16-ddef-4b70-af97-9b506d1859c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:07.893832 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab268925-ddbe-407a-b843-388ad280bf1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:07.893832 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab268925-ddbe-407a-b843-388ad280bf1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:07.893832 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab268925-ddbe-407a-b843-388ad280bf1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:07.894324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab268925-ddbe-407a-b843-388ad280bf1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:07.894821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab268925-ddbe-407a-b843-388ad280bf1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:07.895217 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab268925-ddbe-407a-b843-388ad280bf1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:07.899309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:20:07.978144 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c6fb5e16-ddef-4b70-af97-9b506d1859c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:07.986833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c6fb5e16-ddef-4b70-af97-9b506d1859c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.107372 Aug 28 01:20:07.987425 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1169/2112] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:07 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:08.053228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ab268925-ddbe-407a-b843-388ad280bf1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:08.062244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ab268925-ddbe-407a-b843-388ad280bf1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:08.063676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ab268925-ddbe-407a-b843-388ad280bf1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:08.064838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ab268925-ddbe-407a-b843-388ad280bf1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.182109 Aug 28 01:20:08.065412 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 944/2113] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:07 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:08.903822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c0a4673a-d6d7-4bf9-9e19-609523f417b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:08.904757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c0a4673a-d6d7-4bf9-9e19-609523f417b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:08.906418 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1ea900a7-3aa9-4693-8217-e95e133204be tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:08.907393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1ea900a7-3aa9-4693-8217-e95e133204be tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:08.914798 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0a4673a-d6d7-4bf9-9e19-609523f417b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:08.915604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0a4673a-d6d7-4bf9-9e19-609523f417b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:08.916293 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0a4673a-d6d7-4bf9-9e19-609523f417b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:08.917245 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1ea900a7-3aa9-4693-8217-e95e133204be tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:08.917386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0a4673a-d6d7-4bf9-9e19-609523f417b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:08.918051 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1ea900a7-3aa9-4693-8217-e95e133204be tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:08.918181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0a4673a-d6d7-4bf9-9e19-609523f417b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:08.918739 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1ea900a7-3aa9-4693-8217-e95e133204be tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:08.918883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0a4673a-d6d7-4bf9-9e19-609523f417b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:08.919827 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1ea900a7-3aa9-4693-8217-e95e133204be tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:08.920551 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1ea900a7-3aa9-4693-8217-e95e133204be tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:08.921268 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1ea900a7-3aa9-4693-8217-e95e133204be tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:09.141902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c0a4673a-d6d7-4bf9-9e19-609523f417b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:09.145531 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1ea900a7-3aa9-4693-8217-e95e133204be tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:09.150411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c0a4673a-d6d7-4bf9-9e19-609523f417b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.253382 Aug 28 01:20:09.150957 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1170/2114] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:08 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:09.154595 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1ea900a7-3aa9-4693-8217-e95e133204be tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.254462 Aug 28 01:20:09.155386 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 945/2115] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:08 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:09.155903 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-01d0b0a7-d532-4afa-bbce-5dbf968e7379 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:09.156461 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-01d0b0a7-d532-4afa-bbce-5dbf968e7379 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:09.163560 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-01d0b0a7-d532-4afa-bbce-5dbf968e7379 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:09.163823 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-01d0b0a7-d532-4afa-bbce-5dbf968e7379 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:09.164264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-01d0b0a7-d532-4afa-bbce-5dbf968e7379 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:09.164996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-01d0b0a7-d532-4afa-bbce-5dbf968e7379 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:09.165452 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-01d0b0a7-d532-4afa-bbce-5dbf968e7379 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:09.165870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-01d0b0a7-d532-4afa-bbce-5dbf968e7379 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:09.169849 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4414c42c-d643-4c57-97b0-b2ff60610a12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:09.170462 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4414c42c-d643-4c57-97b0-b2ff60610a12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:09.176621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4414c42c-d643-4c57-97b0-b2ff60610a12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:09.177171 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4414c42c-d643-4c57-97b0-b2ff60610a12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:09.177583 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4414c42c-d643-4c57-97b0-b2ff60610a12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:09.178276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4414c42c-d643-4c57-97b0-b2ff60610a12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:09.178712 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4414c42c-d643-4c57-97b0-b2ff60610a12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:09.179129 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4414c42c-d643-4c57-97b0-b2ff60610a12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:09.247724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-01d0b0a7-d532-4afa-bbce-5dbf968e7379 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:09.256729 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-01d0b0a7-d532-4afa-bbce-5dbf968e7379 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.104738 Aug 28 01:20:09.257310 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1171/2116] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:09 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:09.336256 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4414c42c-d643-4c57-97b0-b2ff60610a12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:09.345491 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4414c42c-d643-4c57-97b0-b2ff60610a12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:09.346910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4414c42c-d643-4c57-97b0-b2ff60610a12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:09.348067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4414c42c-d643-4c57-97b0-b2ff60610a12 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.182014 Aug 28 01:20:09.348596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 946/2117] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:09 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:10.171397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b721f71a-a2e6-4d76-a221-d9559724aee7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:10.171958 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b721f71a-a2e6-4d76-a221-d9559724aee7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:10.178925 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b721f71a-a2e6-4d76-a221-d9559724aee7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:10.179389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b721f71a-a2e6-4d76-a221-d9559724aee7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:10.179615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c6899aad-8b72-40d9-a99d-d0247c315502 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:10.180138 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b721f71a-a2e6-4d76-a221-d9559724aee7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:10.180248 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c6899aad-8b72-40d9-a99d-d0247c315502 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:10.180527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b721f71a-a2e6-4d76-a221-d9559724aee7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:10.181000 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b721f71a-a2e6-4d76-a221-d9559724aee7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:10.181435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b721f71a-a2e6-4d76-a221-d9559724aee7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:10.186984 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6899aad-8b72-40d9-a99d-d0247c315502 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:10.187463 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6899aad-8b72-40d9-a99d-d0247c315502 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:10.187884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6899aad-8b72-40d9-a99d-d0247c315502 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:10.188557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6899aad-8b72-40d9-a99d-d0247c315502 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:10.189007 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6899aad-8b72-40d9-a99d-d0247c315502 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:10.189437 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6899aad-8b72-40d9-a99d-d0247c315502 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:10.375893 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b721f71a-a2e6-4d76-a221-d9559724aee7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:10.378176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:20:10.378853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c6899aad-8b72-40d9-a99d-d0247c315502 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:10.386129 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b721f71a-a2e6-4d76-a221-d9559724aee7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.218671 Aug 28 01:20:10.386710 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1172/2118] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:10 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:10.387639 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c6899aad-8b72-40d9-a99d-d0247c315502 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.211793 Aug 28 01:20:10.388505 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 947/2119] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:10 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:10.391231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-baa6acb9-5604-44c5-9363-513781352787 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:10.391847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-baa6acb9-5604-44c5-9363-513781352787 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:10.392756 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d26dce11-7171-459a-af5e-4eee7905c22e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:10.393392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d26dce11-7171-459a-af5e-4eee7905c22e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:10.398005 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-baa6acb9-5604-44c5-9363-513781352787 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:10.398489 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-baa6acb9-5604-44c5-9363-513781352787 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:10.398936 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-baa6acb9-5604-44c5-9363-513781352787 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:10.399630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-baa6acb9-5604-44c5-9363-513781352787 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:10.400071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-baa6acb9-5604-44c5-9363-513781352787 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:10.400497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-baa6acb9-5604-44c5-9363-513781352787 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:10.407917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d26dce11-7171-459a-af5e-4eee7905c22e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:10.408413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d26dce11-7171-459a-af5e-4eee7905c22e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:10.408835 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d26dce11-7171-459a-af5e-4eee7905c22e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:10.409589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d26dce11-7171-459a-af5e-4eee7905c22e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:10.410032 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d26dce11-7171-459a-af5e-4eee7905c22e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:10.410497 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d26dce11-7171-459a-af5e-4eee7905c22e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:10.479753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-baa6acb9-5604-44c5-9363-513781352787 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:10.488126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-baa6acb9-5604-44c5-9363-513781352787 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.100405 Aug 28 01:20:10.488651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1173/2120] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:10 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:10.563000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d26dce11-7171-459a-af5e-4eee7905c22e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:10.572278 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d26dce11-7171-459a-af5e-4eee7905c22e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:10.573768 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d26dce11-7171-459a-af5e-4eee7905c22e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:10.574918 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d26dce11-7171-459a-af5e-4eee7905c22e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.185693 Aug 28 01:20:10.575504 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 948/2121] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:10 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:11.414615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-785f7ac6-1bba-4d29-9723-33889d8f4ce1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:11.415395 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8e2af6b8-e24c-4f60-834b-9f8aae19be31 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:11.415782 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-785f7ac6-1bba-4d29-9723-33889d8f4ce1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:11.415895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8e2af6b8-e24c-4f60-834b-9f8aae19be31 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:11.424501 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-785f7ac6-1bba-4d29-9723-33889d8f4ce1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:11.424724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e2af6b8-e24c-4f60-834b-9f8aae19be31 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:11.425017 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-785f7ac6-1bba-4d29-9723-33889d8f4ce1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:11.425145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e2af6b8-e24c-4f60-834b-9f8aae19be31 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:11.425510 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-785f7ac6-1bba-4d29-9723-33889d8f4ce1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:11.425654 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e2af6b8-e24c-4f60-834b-9f8aae19be31 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:11.426529 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-785f7ac6-1bba-4d29-9723-33889d8f4ce1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:11.426651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e2af6b8-e24c-4f60-834b-9f8aae19be31 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:11.427062 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-785f7ac6-1bba-4d29-9723-33889d8f4ce1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:11.427190 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e2af6b8-e24c-4f60-834b-9f8aae19be31 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:11.427639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e2af6b8-e24c-4f60-834b-9f8aae19be31 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:11.427760 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-785f7ac6-1bba-4d29-9723-33889d8f4ce1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:11.631994 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-785f7ac6-1bba-4d29-9723-33889d8f4ce1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:11.639599 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8e2af6b8-e24c-4f60-834b-9f8aae19be31 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:11.641400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-785f7ac6-1bba-4d29-9723-33889d8f4ce1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.231981 Aug 28 01:20:11.641977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 949/2122] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:11 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:11.646535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a77ad9f1-6e87-40e0-bff2-f7b5ff2204ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:11.647137 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a77ad9f1-6e87-40e0-bff2-f7b5ff2204ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:11.648380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8e2af6b8-e24c-4f60-834b-9f8aae19be31 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.238841 Aug 28 01:20:11.654001 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1174/2123] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:11 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:11.654001 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-304d5847-ba9f-483f-9d9f-680ba8a0cb24 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:11.654309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-304d5847-ba9f-483f-9d9f-680ba8a0cb24 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:11.657843 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a77ad9f1-6e87-40e0-bff2-f7b5ff2204ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:11.658313 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a77ad9f1-6e87-40e0-bff2-f7b5ff2204ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:11.658757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a77ad9f1-6e87-40e0-bff2-f7b5ff2204ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:11.659455 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a77ad9f1-6e87-40e0-bff2-f7b5ff2204ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:11.659902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a77ad9f1-6e87-40e0-bff2-f7b5ff2204ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:11.660122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-304d5847-ba9f-483f-9d9f-680ba8a0cb24 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:11.660321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a77ad9f1-6e87-40e0-bff2-f7b5ff2204ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:11.660605 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-304d5847-ba9f-483f-9d9f-680ba8a0cb24 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:11.661083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-304d5847-ba9f-483f-9d9f-680ba8a0cb24 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:11.661770 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-304d5847-ba9f-483f-9d9f-680ba8a0cb24 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:11.662212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-304d5847-ba9f-483f-9d9f-680ba8a0cb24 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:11.662642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-304d5847-ba9f-483f-9d9f-680ba8a0cb24 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:11.749650 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a77ad9f1-6e87-40e0-bff2-f7b5ff2204ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:11.758089 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a77ad9f1-6e87-40e0-bff2-f7b5ff2204ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.115026 Aug 28 01:20:11.758680 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 950/2124] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:11 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:11.819074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-304d5847-ba9f-483f-9d9f-680ba8a0cb24 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:11.833022 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-304d5847-ba9f-483f-9d9f-680ba8a0cb24 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:11.835450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-304d5847-ba9f-483f-9d9f-680ba8a0cb24 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:11.837312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-304d5847-ba9f-483f-9d9f-680ba8a0cb24 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.186979 Aug 28 01:20:11.838166 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1175/2125] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:11 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:12.665267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f7a7ee5e-d68b-4fa4-a232-e890b9e0566a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:12.666052 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f7a7ee5e-d68b-4fa4-a232-e890b9e0566a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:12.669953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0f4170d7-ac0e-4134-9fd3-7f65fd0e8ce4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:12.670547 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0f4170d7-ac0e-4134-9fd3-7f65fd0e8ce4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:12.672995 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f7a7ee5e-d68b-4fa4-a232-e890b9e0566a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:12.673479 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f7a7ee5e-d68b-4fa4-a232-e890b9e0566a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:12.674199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f7a7ee5e-d68b-4fa4-a232-e890b9e0566a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:12.674908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f7a7ee5e-d68b-4fa4-a232-e890b9e0566a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:12.675356 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f7a7ee5e-d68b-4fa4-a232-e890b9e0566a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:12.675769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f7a7ee5e-d68b-4fa4-a232-e890b9e0566a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:12.677122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f4170d7-ac0e-4134-9fd3-7f65fd0e8ce4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:12.677603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f4170d7-ac0e-4134-9fd3-7f65fd0e8ce4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:12.678040 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f4170d7-ac0e-4134-9fd3-7f65fd0e8ce4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:12.678724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f4170d7-ac0e-4134-9fd3-7f65fd0e8ce4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:12.679156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f4170d7-ac0e-4134-9fd3-7f65fd0e8ce4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:12.679996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f4170d7-ac0e-4134-9fd3-7f65fd0e8ce4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:12.866422 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f7a7ee5e-d68b-4fa4-a232-e890b9e0566a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:12.870614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0f4170d7-ac0e-4134-9fd3-7f65fd0e8ce4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:12.876386 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f7a7ee5e-d68b-4fa4-a232-e890b9e0566a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.216092 Aug 28 01:20:12.877311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 951/2126] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:12 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:12.879382 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0f4170d7-ac0e-4134-9fd3-7f65fd0e8ce4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.213454 Aug 28 01:20:12.880088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1176/2127] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:12 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:12.881630 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8a75b5a5-e79c-4c02-bfe8-8f3d14aa8ad2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:12.882243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8a75b5a5-e79c-4c02-bfe8-8f3d14aa8ad2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:12.888270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a75b5a5-e79c-4c02-bfe8-8f3d14aa8ad2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:12.888817 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a75b5a5-e79c-4c02-bfe8-8f3d14aa8ad2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:12.889356 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a75b5a5-e79c-4c02-bfe8-8f3d14aa8ad2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:12.890119 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a75b5a5-e79c-4c02-bfe8-8f3d14aa8ad2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:12.890621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a75b5a5-e79c-4c02-bfe8-8f3d14aa8ad2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:12.891125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a75b5a5-e79c-4c02-bfe8-8f3d14aa8ad2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:12.894400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0d927fe9-1cb9-4118-bcac-22cea5cfa8e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:12.895002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0d927fe9-1cb9-4118-bcac-22cea5cfa8e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:12.901020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0d927fe9-1cb9-4118-bcac-22cea5cfa8e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:12.901525 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0d927fe9-1cb9-4118-bcac-22cea5cfa8e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:12.902024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0d927fe9-1cb9-4118-bcac-22cea5cfa8e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:12.902703 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0d927fe9-1cb9-4118-bcac-22cea5cfa8e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:12.903146 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0d927fe9-1cb9-4118-bcac-22cea5cfa8e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:12.903556 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0d927fe9-1cb9-4118-bcac-22cea5cfa8e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:12.962860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8a75b5a5-e79c-4c02-bfe8-8f3d14aa8ad2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:12.972329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8a75b5a5-e79c-4c02-bfe8-8f3d14aa8ad2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.094076 Aug 28 01:20:12.972762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 952/2128] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:12 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:13.047367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0d927fe9-1cb9-4118-bcac-22cea5cfa8e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:13.060068 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0d927fe9-1cb9-4118-bcac-22cea5cfa8e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:13.062211 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0d927fe9-1cb9-4118-bcac-22cea5cfa8e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:13.063950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0d927fe9-1cb9-4118-bcac-22cea5cfa8e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.172997 Aug 28 01:20:13.064683 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1177/2129] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:12 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:13.898159 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ee6acb6a-521d-48fb-b47e-094074f8805b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:13.898806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ee6acb6a-521d-48fb-b47e-094074f8805b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:13.899869 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0210085b-7823-483d-b3f2-6acc7d64446c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:13.900473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0210085b-7823-483d-b3f2-6acc7d64446c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:13.905940 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ee6acb6a-521d-48fb-b47e-094074f8805b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:13.906428 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ee6acb6a-521d-48fb-b47e-094074f8805b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:13.906854 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ee6acb6a-521d-48fb-b47e-094074f8805b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:13.907030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0210085b-7823-483d-b3f2-6acc7d64446c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:13.907449 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0210085b-7823-483d-b3f2-6acc7d64446c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:13.907617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ee6acb6a-521d-48fb-b47e-094074f8805b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:13.907874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0210085b-7823-483d-b3f2-6acc7d64446c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:13.908042 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ee6acb6a-521d-48fb-b47e-094074f8805b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:13.908397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ee6acb6a-521d-48fb-b47e-094074f8805b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:13.908549 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0210085b-7823-483d-b3f2-6acc7d64446c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:13.908981 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0210085b-7823-483d-b3f2-6acc7d64446c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:13.909416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0210085b-7823-483d-b3f2-6acc7d64446c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:14.338940 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ee6acb6a-521d-48fb-b47e-094074f8805b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:14.342476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0210085b-7823-483d-b3f2-6acc7d64446c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:14.347523 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ee6acb6a-521d-48fb-b47e-094074f8805b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.453320 Aug 28 01:20:14.348244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 953/2130] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:13 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:14.350961 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0210085b-7823-483d-b3f2-6acc7d64446c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.454732 Aug 28 01:20:14.354480 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1178/2131] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:13 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:14.354654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f0d64a1b-979e-40d7-a2ef-9a0917717b6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:14.354654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f0d64a1b-979e-40d7-a2ef-9a0917717b6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:14.359752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0d64a1b-979e-40d7-a2ef-9a0917717b6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:14.360356 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0d64a1b-979e-40d7-a2ef-9a0917717b6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:14.360876 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0d64a1b-979e-40d7-a2ef-9a0917717b6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:14.361699 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0d64a1b-979e-40d7-a2ef-9a0917717b6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:14.362237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0d64a1b-979e-40d7-a2ef-9a0917717b6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:14.362649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0d64a1b-979e-40d7-a2ef-9a0917717b6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:14.363095 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b3532254-fac8-4848-90ec-ffba3ec2893f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:14.363704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b3532254-fac8-4848-90ec-ffba3ec2893f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:14.370156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b3532254-fac8-4848-90ec-ffba3ec2893f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:14.370720 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b3532254-fac8-4848-90ec-ffba3ec2893f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:14.371158 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b3532254-fac8-4848-90ec-ffba3ec2893f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:14.371840 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b3532254-fac8-4848-90ec-ffba3ec2893f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:14.372311 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b3532254-fac8-4848-90ec-ffba3ec2893f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:14.372730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b3532254-fac8-4848-90ec-ffba3ec2893f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:14.448816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f0d64a1b-979e-40d7-a2ef-9a0917717b6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:14.457615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f0d64a1b-979e-40d7-a2ef-9a0917717b6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.108373 Aug 28 01:20:14.458195 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 954/2132] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:14 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:14.517783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b3532254-fac8-4848-90ec-ffba3ec2893f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:14.526568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b3532254-fac8-4848-90ec-ffba3ec2893f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:14.528001 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b3532254-fac8-4848-90ec-ffba3ec2893f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:14.529191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b3532254-fac8-4848-90ec-ffba3ec2893f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.176718 Aug 28 01:20:14.529751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1179/2133] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:14 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:15.369670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e32e82d0-fe80-4b75-a56b-c440541377f4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:15.370481 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e32e82d0-fe80-4b75-a56b-c440541377f4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:15.371780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-18352e7a-de6d-44b2-9716-8fc29337fd34 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:15.372309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-18352e7a-de6d-44b2-9716-8fc29337fd34 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:15.377446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e32e82d0-fe80-4b75-a56b-c440541377f4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:15.377925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e32e82d0-fe80-4b75-a56b-c440541377f4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:15.378349 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e32e82d0-fe80-4b75-a56b-c440541377f4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:15.379039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e32e82d0-fe80-4b75-a56b-c440541377f4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:15.379161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-18352e7a-de6d-44b2-9716-8fc29337fd34 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:15.379481 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e32e82d0-fe80-4b75-a56b-c440541377f4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:15.379671 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-18352e7a-de6d-44b2-9716-8fc29337fd34 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:15.379890 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e32e82d0-fe80-4b75-a56b-c440541377f4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:15.380099 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-18352e7a-de6d-44b2-9716-8fc29337fd34 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:15.380802 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-18352e7a-de6d-44b2-9716-8fc29337fd34 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:15.381278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-18352e7a-de6d-44b2-9716-8fc29337fd34 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:15.381702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-18352e7a-de6d-44b2-9716-8fc29337fd34 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:15.573218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e32e82d0-fe80-4b75-a56b-c440541377f4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:15.576253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-18352e7a-de6d-44b2-9716-8fc29337fd34 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:15.582588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e32e82d0-fe80-4b75-a56b-c440541377f4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.216716 Aug 28 01:20:15.583385 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 955/2134] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:15 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:15.584740 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-18352e7a-de6d-44b2-9716-8fc29337fd34 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.216815 Aug 28 01:20:15.597168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1180/2135] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:15 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:15.597396 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f71a5a43-7819-4b94-9ebb-e80b1e226647 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:15.597396 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f71a5a43-7819-4b94-9ebb-e80b1e226647 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:15.597600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1909f898-a16b-4b6b-95c0-1df2fc72f0ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:15.597600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1909f898-a16b-4b6b-95c0-1df2fc72f0ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:15.597789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f71a5a43-7819-4b94-9ebb-e80b1e226647 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:15.597789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f71a5a43-7819-4b94-9ebb-e80b1e226647 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:15.597789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f71a5a43-7819-4b94-9ebb-e80b1e226647 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:15.598144 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1909f898-a16b-4b6b-95c0-1df2fc72f0ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:15.598245 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f71a5a43-7819-4b94-9ebb-e80b1e226647 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:15.598331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1909f898-a16b-4b6b-95c0-1df2fc72f0ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:15.598438 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f71a5a43-7819-4b94-9ebb-e80b1e226647 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:15.598532 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1909f898-a16b-4b6b-95c0-1df2fc72f0ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:15.598635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f71a5a43-7819-4b94-9ebb-e80b1e226647 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:15.598721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1909f898-a16b-4b6b-95c0-1df2fc72f0ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:15.598721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1909f898-a16b-4b6b-95c0-1df2fc72f0ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:15.598721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1909f898-a16b-4b6b-95c0-1df2fc72f0ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:15.679588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f71a5a43-7819-4b94-9ebb-e80b1e226647 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:15.688070 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f71a5a43-7819-4b94-9ebb-e80b1e226647 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.103839 Aug 28 01:20:15.688645 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 956/2136] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:15 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:15.746962 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1909f898-a16b-4b6b-95c0-1df2fc72f0ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:15.756147 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1909f898-a16b-4b6b-95c0-1df2fc72f0ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:15.757606 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1909f898-a16b-4b6b-95c0-1df2fc72f0ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:15.758748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1909f898-a16b-4b6b-95c0-1df2fc72f0ef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.172520 Aug 28 01:20:15.759335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1181/2137] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:15 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:16.604024 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-07fa734a-7a8e-4fae-b94f-8d65fa15e6d9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:16.604654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-07fa734a-7a8e-4fae-b94f-8d65fa15e6d9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:16.608452 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-50ad4f27-bb70-4569-aa60-1dd86abf8cfe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:16.609172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-50ad4f27-bb70-4569-aa60-1dd86abf8cfe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:16.611624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07fa734a-7a8e-4fae-b94f-8d65fa15e6d9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:16.612107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07fa734a-7a8e-4fae-b94f-8d65fa15e6d9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:16.612537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07fa734a-7a8e-4fae-b94f-8d65fa15e6d9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:16.613301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07fa734a-7a8e-4fae-b94f-8d65fa15e6d9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:16.613729 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07fa734a-7a8e-4fae-b94f-8d65fa15e6d9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:16.614169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07fa734a-7a8e-4fae-b94f-8d65fa15e6d9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:16.616815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-50ad4f27-bb70-4569-aa60-1dd86abf8cfe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:16.617348 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-50ad4f27-bb70-4569-aa60-1dd86abf8cfe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:16.617833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-50ad4f27-bb70-4569-aa60-1dd86abf8cfe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:16.618665 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-50ad4f27-bb70-4569-aa60-1dd86abf8cfe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:16.619173 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-50ad4f27-bb70-4569-aa60-1dd86abf8cfe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:16.619714 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-50ad4f27-bb70-4569-aa60-1dd86abf8cfe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:16.821125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-07fa734a-7a8e-4fae-b94f-8d65fa15e6d9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:16.824887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-50ad4f27-bb70-4569-aa60-1dd86abf8cfe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:16.831047 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-07fa734a-7a8e-4fae-b94f-8d65fa15e6d9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.230965 Aug 28 01:20:16.845649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 957/2138] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:16 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:16.845920 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-50ad4f27-bb70-4569-aa60-1dd86abf8cfe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.230589 Aug 28 01:20:16.846037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-189f6a0e-865f-44d0-bfc4-65b025830094 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:16.846037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-189f6a0e-865f-44d0-bfc4-65b025830094 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:16.846037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-189f6a0e-865f-44d0-bfc4-65b025830094 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:16.846037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-189f6a0e-865f-44d0-bfc4-65b025830094 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:16.846424 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-189f6a0e-865f-44d0-bfc4-65b025830094 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:16.846531 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1182/2139] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:16 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:16.846636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-189f6a0e-865f-44d0-bfc4-65b025830094 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:16.846994 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-189f6a0e-865f-44d0-bfc4-65b025830094 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:16.847474 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-189f6a0e-865f-44d0-bfc4-65b025830094 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:16.854205 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7b5824ff-7311-4067-b535-85c490ef0256 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:16.854950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7b5824ff-7311-4067-b535-85c490ef0256 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:16.861696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b5824ff-7311-4067-b535-85c490ef0256 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:16.862191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b5824ff-7311-4067-b535-85c490ef0256 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:16.862626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b5824ff-7311-4067-b535-85c490ef0256 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:16.863926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b5824ff-7311-4067-b535-85c490ef0256 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:16.864489 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b5824ff-7311-4067-b535-85c490ef0256 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:16.865336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b5824ff-7311-4067-b535-85c490ef0256 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:16.926306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-189f6a0e-865f-44d0-bfc4-65b025830094 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:16.934826 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-189f6a0e-865f-44d0-bfc4-65b025830094 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.102070 Aug 28 01:20:16.935412 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 958/2140] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:16 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:17.016654 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7b5824ff-7311-4067-b535-85c490ef0256 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:17.029022 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7b5824ff-7311-4067-b535-85c490ef0256 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:17.031486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7b5824ff-7311-4067-b535-85c490ef0256 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:17.031679 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7b5824ff-7311-4067-b535-85c490ef0256 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.185111 Aug 28 01:20:17.032222 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1183/2141] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:16 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:17.853830 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e3c6782b-acac-47a6-be94-6800c9ab3809 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:17.854470 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e3c6782b-acac-47a6-be94-6800c9ab3809 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:17.860191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-90c26f05-8631-4dee-b59a-216b2091a14d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:17.860875 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-90c26f05-8631-4dee-b59a-216b2091a14d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:17.861313 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3c6782b-acac-47a6-be94-6800c9ab3809 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:17.861800 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3c6782b-acac-47a6-be94-6800c9ab3809 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:17.862228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3c6782b-acac-47a6-be94-6800c9ab3809 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:17.862897 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3c6782b-acac-47a6-be94-6800c9ab3809 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:17.863375 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3c6782b-acac-47a6-be94-6800c9ab3809 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:17.863768 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3c6782b-acac-47a6-be94-6800c9ab3809 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:17.867221 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90c26f05-8631-4dee-b59a-216b2091a14d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:17.867697 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90c26f05-8631-4dee-b59a-216b2091a14d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:17.868126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90c26f05-8631-4dee-b59a-216b2091a14d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:17.868809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90c26f05-8631-4dee-b59a-216b2091a14d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:17.869278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90c26f05-8631-4dee-b59a-216b2091a14d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:17.869692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90c26f05-8631-4dee-b59a-216b2091a14d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:18.056330 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e3c6782b-acac-47a6-be94-6800c9ab3809 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:18.060153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-90c26f05-8631-4dee-b59a-216b2091a14d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:18.065006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e3c6782b-acac-47a6-be94-6800c9ab3809 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.215105 Aug 28 01:20:18.065580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 959/2142] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:17 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:18.069334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-90c26f05-8631-4dee-b59a-216b2091a14d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.212836 Aug 28 01:20:18.069886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1184/2143] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:17 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:18.070468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5fc4f664-74a0-485f-8d80-9f974eb27a73 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:18.071071 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5fc4f664-74a0-485f-8d80-9f974eb27a73 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:18.075034 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e769864c-0c47-42dc-836c-b093f58daee4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:18.075866 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e769864c-0c47-42dc-836c-b093f58daee4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:18.077265 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5fc4f664-74a0-485f-8d80-9f974eb27a73 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:18.077740 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5fc4f664-74a0-485f-8d80-9f974eb27a73 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:18.078385 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5fc4f664-74a0-485f-8d80-9f974eb27a73 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:18.079255 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5fc4f664-74a0-485f-8d80-9f974eb27a73 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:18.079887 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5fc4f664-74a0-485f-8d80-9f974eb27a73 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:18.080307 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5fc4f664-74a0-485f-8d80-9f974eb27a73 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:18.082061 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e769864c-0c47-42dc-836c-b093f58daee4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:18.082668 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e769864c-0c47-42dc-836c-b093f58daee4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:18.082978 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e769864c-0c47-42dc-836c-b093f58daee4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:18.083661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e769864c-0c47-42dc-836c-b093f58daee4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:18.084128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e769864c-0c47-42dc-836c-b093f58daee4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:18.084536 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e769864c-0c47-42dc-836c-b093f58daee4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:18.148933 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5fc4f664-74a0-485f-8d80-9f974eb27a73 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:18.151324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:20:18.159342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5fc4f664-74a0-485f-8d80-9f974eb27a73 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.092428 Aug 28 01:20:18.159701 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 960/2144] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:18 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:18.236354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e769864c-0c47-42dc-836c-b093f58daee4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:18.249472 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e769864c-0c47-42dc-836c-b093f58daee4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:18.251676 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e769864c-0c47-42dc-836c-b093f58daee4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:18.253518 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e769864c-0c47-42dc-836c-b093f58daee4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.181913 Aug 28 01:20:18.254336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1185/2145] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:18 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:19.087459 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0c039a63-1062-4435-99c4-977f8f7fd509 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:19.088042 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0c039a63-1062-4435-99c4-977f8f7fd509 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:19.090484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-047a9a2f-c526-4ffe-b22e-29e7fbf135fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:19.091117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-047a9a2f-c526-4ffe-b22e-29e7fbf135fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:19.095001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0c039a63-1062-4435-99c4-977f8f7fd509 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:19.095484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0c039a63-1062-4435-99c4-977f8f7fd509 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:19.096596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0c039a63-1062-4435-99c4-977f8f7fd509 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:19.097329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0c039a63-1062-4435-99c4-977f8f7fd509 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:19.097780 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0c039a63-1062-4435-99c4-977f8f7fd509 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:19.097925 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-047a9a2f-c526-4ffe-b22e-29e7fbf135fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:19.098199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0c039a63-1062-4435-99c4-977f8f7fd509 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:19.098310 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-047a9a2f-c526-4ffe-b22e-29e7fbf135fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:19.098723 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-047a9a2f-c526-4ffe-b22e-29e7fbf135fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:19.099407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-047a9a2f-c526-4ffe-b22e-29e7fbf135fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:19.099853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-047a9a2f-c526-4ffe-b22e-29e7fbf135fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:19.100268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-047a9a2f-c526-4ffe-b22e-29e7fbf135fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:19.295514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-047a9a2f-c526-4ffe-b22e-29e7fbf135fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:19.298848 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0c039a63-1062-4435-99c4-977f8f7fd509 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:19.304402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-047a9a2f-c526-4ffe-b22e-29e7fbf135fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.217655 Aug 28 01:20:19.305026 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1186/2146] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:19 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:19.307556 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0c039a63-1062-4435-99c4-977f8f7fd509 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.224090 Aug 28 01:20:19.309597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-635bab41-2571-4faf-a45e-cca868bdc85c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:19.310196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-635bab41-2571-4faf-a45e-cca868bdc85c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:19.316128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-635bab41-2571-4faf-a45e-cca868bdc85c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:19.316592 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-635bab41-2571-4faf-a45e-cca868bdc85c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:19.317578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-635bab41-2571-4faf-a45e-cca868bdc85c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:19.318262 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-635bab41-2571-4faf-a45e-cca868bdc85c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:19.318699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-635bab41-2571-4faf-a45e-cca868bdc85c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:19.319119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-635bab41-2571-4faf-a45e-cca868bdc85c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:19.321244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 961/2147] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:19 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:19.326961 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f45ae38b-ee58-4be9-acf2-b9b6f4bb87bd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:19.327775 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f45ae38b-ee58-4be9-acf2-b9b6f4bb87bd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:19.334063 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f45ae38b-ee58-4be9-acf2-b9b6f4bb87bd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:19.334634 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f45ae38b-ee58-4be9-acf2-b9b6f4bb87bd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:19.335135 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f45ae38b-ee58-4be9-acf2-b9b6f4bb87bd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:19.335905 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f45ae38b-ee58-4be9-acf2-b9b6f4bb87bd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:19.336414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f45ae38b-ee58-4be9-acf2-b9b6f4bb87bd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:19.336903 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f45ae38b-ee58-4be9-acf2-b9b6f4bb87bd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:19.383249 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-635bab41-2571-4faf-a45e-cca868bdc85c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:19.392107 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-635bab41-2571-4faf-a45e-cca868bdc85c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.086051 Aug 28 01:20:19.392699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1187/2148] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:19 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:19.481448 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f45ae38b-ee58-4be9-acf2-b9b6f4bb87bd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:19.493442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f45ae38b-ee58-4be9-acf2-b9b6f4bb87bd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:19.495479 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f45ae38b-ee58-4be9-acf2-b9b6f4bb87bd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:19.497107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f45ae38b-ee58-4be9-acf2-b9b6f4bb87bd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.174630 Aug 28 01:20:19.497673 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 962/2149] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:19 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:20.329581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fa2abdbd-e5b7-4125-b6b1-8e2ce70c6db3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:20.330222 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fa2abdbd-e5b7-4125-b6b1-8e2ce70c6db3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:20.334055 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5d3bebb1-5c66-4c57-aede-046269a8216d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:20.334758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5d3bebb1-5c66-4c57-aede-046269a8216d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:20.339202 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fa2abdbd-e5b7-4125-b6b1-8e2ce70c6db3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:20.339775 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fa2abdbd-e5b7-4125-b6b1-8e2ce70c6db3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:20.340283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fa2abdbd-e5b7-4125-b6b1-8e2ce70c6db3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:20.340989 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fa2abdbd-e5b7-4125-b6b1-8e2ce70c6db3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:20.341476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fa2abdbd-e5b7-4125-b6b1-8e2ce70c6db3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:20.341891 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fa2abdbd-e5b7-4125-b6b1-8e2ce70c6db3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:20.343095 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5d3bebb1-5c66-4c57-aede-046269a8216d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:20.343576 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5d3bebb1-5c66-4c57-aede-046269a8216d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:20.344006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5d3bebb1-5c66-4c57-aede-046269a8216d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:20.344724 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5d3bebb1-5c66-4c57-aede-046269a8216d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:20.345229 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5d3bebb1-5c66-4c57-aede-046269a8216d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:20.345649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5d3bebb1-5c66-4c57-aede-046269a8216d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:20.547300 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5d3bebb1-5c66-4c57-aede-046269a8216d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:20.550441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fa2abdbd-e5b7-4125-b6b1-8e2ce70c6db3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:20.555837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5d3bebb1-5c66-4c57-aede-046269a8216d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.227266 Aug 28 01:20:20.556413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 963/2150] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:20 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:20.559402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fa2abdbd-e5b7-4125-b6b1-8e2ce70c6db3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.236008 Aug 28 01:20:20.560171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1188/2151] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:20 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:20.560808 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1c5554ee-1ebd-4068-900d-57e8356b53a8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:20.561463 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1c5554ee-1ebd-4068-900d-57e8356b53a8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:20.564654 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7c3092fe-f8d4-459a-b87f-b29f608d5626 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:20.565447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7c3092fe-f8d4-459a-b87f-b29f608d5626 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:20.567901 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c5554ee-1ebd-4068-900d-57e8356b53a8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:20.568384 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c5554ee-1ebd-4068-900d-57e8356b53a8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:20.568816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c5554ee-1ebd-4068-900d-57e8356b53a8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:20.569674 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c5554ee-1ebd-4068-900d-57e8356b53a8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:20.570318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c5554ee-1ebd-4068-900d-57e8356b53a8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:20.570701 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c5554ee-1ebd-4068-900d-57e8356b53a8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:20.572367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c3092fe-f8d4-459a-b87f-b29f608d5626 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:20.572857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c3092fe-f8d4-459a-b87f-b29f608d5626 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:20.573402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c3092fe-f8d4-459a-b87f-b29f608d5626 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:20.574023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c3092fe-f8d4-459a-b87f-b29f608d5626 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:20.574506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c3092fe-f8d4-459a-b87f-b29f608d5626 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:20.575206 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c3092fe-f8d4-459a-b87f-b29f608d5626 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:20.590625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:20:20.639723 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1c5554ee-1ebd-4068-900d-57e8356b53a8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:20.648407 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1c5554ee-1ebd-4068-900d-57e8356b53a8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.091007 Aug 28 01:20:20.649014 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 964/2152] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:20 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:20.740991 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7c3092fe-f8d4-459a-b87f-b29f608d5626 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:20.753498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7c3092fe-f8d4-459a-b87f-b29f608d5626 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:20.755724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7c3092fe-f8d4-459a-b87f-b29f608d5626 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:20.757612 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7c3092fe-f8d4-459a-b87f-b29f608d5626 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.196198 Aug 28 01:20:20.758330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1189/2153] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:20 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:21.585781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6ab076eb-7dc3-4589-af27-b52b28e78b14 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:21.586436 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6ab076eb-7dc3-4589-af27-b52b28e78b14 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:21.593776 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6ab076eb-7dc3-4589-af27-b52b28e78b14 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:21.594262 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6ab076eb-7dc3-4589-af27-b52b28e78b14 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:21.594688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6ab076eb-7dc3-4589-af27-b52b28e78b14 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:21.595781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e761c6c8-18da-4124-900f-fdf68e2f2157 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:21.596234 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6ab076eb-7dc3-4589-af27-b52b28e78b14 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:21.596234 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6ab076eb-7dc3-4589-af27-b52b28e78b14 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:21.596418 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e761c6c8-18da-4124-900f-fdf68e2f2157 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:21.596684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6ab076eb-7dc3-4589-af27-b52b28e78b14 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:21.603325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e761c6c8-18da-4124-900f-fdf68e2f2157 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:21.603813 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e761c6c8-18da-4124-900f-fdf68e2f2157 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:21.604238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e761c6c8-18da-4124-900f-fdf68e2f2157 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:21.604910 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e761c6c8-18da-4124-900f-fdf68e2f2157 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:21.605388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e761c6c8-18da-4124-900f-fdf68e2f2157 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:21.605826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e761c6c8-18da-4124-900f-fdf68e2f2157 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:21.791767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6ab076eb-7dc3-4589-af27-b52b28e78b14 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:21.795957 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e761c6c8-18da-4124-900f-fdf68e2f2157 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:21.804584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6ab076eb-7dc3-4589-af27-b52b28e78b14 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.222420 Aug 28 01:20:21.805395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 965/2154] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:21 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:21.807348 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e761c6c8-18da-4124-900f-fdf68e2f2157 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.215332 Aug 28 01:20:21.808076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1190/2155] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:21 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:21.809948 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c97add5f-b206-4aeb-905b-148b11922c98 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:21.810539 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c97add5f-b206-4aeb-905b-148b11922c98 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:21.814611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-49128da9-ae48-449f-871f-5ae957532975 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:21.815485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-49128da9-ae48-449f-871f-5ae957532975 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:21.816395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c97add5f-b206-4aeb-905b-148b11922c98 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:21.817244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c97add5f-b206-4aeb-905b-148b11922c98 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:21.817654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c97add5f-b206-4aeb-905b-148b11922c98 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:21.818477 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c97add5f-b206-4aeb-905b-148b11922c98 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:21.818782 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c97add5f-b206-4aeb-905b-148b11922c98 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:21.819197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c97add5f-b206-4aeb-905b-148b11922c98 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:21.823683 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49128da9-ae48-449f-871f-5ae957532975 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:21.824189 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49128da9-ae48-449f-871f-5ae957532975 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:21.824638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49128da9-ae48-449f-871f-5ae957532975 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:21.825398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49128da9-ae48-449f-871f-5ae957532975 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:21.825838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49128da9-ae48-449f-871f-5ae957532975 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:21.826250 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49128da9-ae48-449f-871f-5ae957532975 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:21.903827 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c97add5f-b206-4aeb-905b-148b11922c98 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:21.912297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c97add5f-b206-4aeb-905b-148b11922c98 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.105899 Aug 28 01:20:21.912843 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 966/2156] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:21 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:21.974174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-49128da9-ae48-449f-871f-5ae957532975 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:21.989465 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-49128da9-ae48-449f-871f-5ae957532975 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:21.990894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-49128da9-ae48-449f-871f-5ae957532975 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:21.992062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-49128da9-ae48-449f-871f-5ae957532975 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.182311 Aug 28 01:20:21.992604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1191/2157] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:21 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:22.827577 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fe1b773c-5b4b-411f-9c41-4101dac2d986 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:22.828244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fe1b773c-5b4b-411f-9c41-4101dac2d986 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:22.829049 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b9e7c17a-3920-4551-a814-7a5a618ef5b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:22.829664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b9e7c17a-3920-4551-a814-7a5a618ef5b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:22.835291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fe1b773c-5b4b-411f-9c41-4101dac2d986 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:22.835786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fe1b773c-5b4b-411f-9c41-4101dac2d986 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:22.836211 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fe1b773c-5b4b-411f-9c41-4101dac2d986 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:22.836901 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fe1b773c-5b4b-411f-9c41-4101dac2d986 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:22.837206 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9e7c17a-3920-4551-a814-7a5a618ef5b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:22.837377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fe1b773c-5b4b-411f-9c41-4101dac2d986 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:22.837689 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9e7c17a-3920-4551-a814-7a5a618ef5b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:22.837799 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fe1b773c-5b4b-411f-9c41-4101dac2d986 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:22.838122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9e7c17a-3920-4551-a814-7a5a618ef5b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:22.838817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9e7c17a-3920-4551-a814-7a5a618ef5b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:22.839267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9e7c17a-3920-4551-a814-7a5a618ef5b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:22.839686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9e7c17a-3920-4551-a814-7a5a618ef5b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:23.030065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fe1b773c-5b4b-411f-9c41-4101dac2d986 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:23.034472 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b9e7c17a-3920-4551-a814-7a5a618ef5b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:23.038500 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fe1b773c-5b4b-411f-9c41-4101dac2d986 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.214903 Aug 28 01:20:23.039081 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 967/2158] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:22 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:23.042863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b9e7c17a-3920-4551-a814-7a5a618ef5b3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.217716 Aug 28 01:20:23.043452 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1192/2159] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:22 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:23.047860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-830a0e42-26aa-43f5-9057-87391409cd38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:23.048643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-830a0e42-26aa-43f5-9057-87391409cd38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:23.054402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-830a0e42-26aa-43f5-9057-87391409cd38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:23.054884 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-830a0e42-26aa-43f5-9057-87391409cd38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:23.055315 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-830a0e42-26aa-43f5-9057-87391409cd38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:23.055999 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-830a0e42-26aa-43f5-9057-87391409cd38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:23.056389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-add872ed-68dd-404b-a13c-d67c41d6a398 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:23.056508 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-830a0e42-26aa-43f5-9057-87391409cd38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:23.056853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-830a0e42-26aa-43f5-9057-87391409cd38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:23.056986 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-add872ed-68dd-404b-a13c-d67c41d6a398 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:23.063038 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-add872ed-68dd-404b-a13c-d67c41d6a398 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:23.063523 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-add872ed-68dd-404b-a13c-d67c41d6a398 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:23.063957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-add872ed-68dd-404b-a13c-d67c41d6a398 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:23.064652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-add872ed-68dd-404b-a13c-d67c41d6a398 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:23.065137 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-add872ed-68dd-404b-a13c-d67c41d6a398 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:23.065559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-add872ed-68dd-404b-a13c-d67c41d6a398 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:23.146022 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-add872ed-68dd-404b-a13c-d67c41d6a398 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:23.155069 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-add872ed-68dd-404b-a13c-d67c41d6a398 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.102315 Aug 28 01:20:23.155782 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 968/2160] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:23 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:23.212124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-830a0e42-26aa-43f5-9057-87391409cd38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:23.220922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-830a0e42-26aa-43f5-9057-87391409cd38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:23.222379 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-830a0e42-26aa-43f5-9057-87391409cd38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:23.223539 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-830a0e42-26aa-43f5-9057-87391409cd38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.179151 Aug 28 01:20:23.224108 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1193/2161] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:23 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:24.059559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9b688039-86b0-473a-b1e4-c8ef8a263213 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:24.060159 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9b688039-86b0-473a-b1e4-c8ef8a263213 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:24.063919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-08a8fcb0-86bb-4720-ab9d-3d637cd60bb1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:24.064684 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-08a8fcb0-86bb-4720-ab9d-3d637cd60bb1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:24.067366 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b688039-86b0-473a-b1e4-c8ef8a263213 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:24.067844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b688039-86b0-473a-b1e4-c8ef8a263213 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:24.068278 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b688039-86b0-473a-b1e4-c8ef8a263213 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:24.068980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b688039-86b0-473a-b1e4-c8ef8a263213 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:24.069436 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b688039-86b0-473a-b1e4-c8ef8a263213 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:24.069844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b688039-86b0-473a-b1e4-c8ef8a263213 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:24.071738 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08a8fcb0-86bb-4720-ab9d-3d637cd60bb1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:24.072230 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08a8fcb0-86bb-4720-ab9d-3d637cd60bb1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:24.072666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08a8fcb0-86bb-4720-ab9d-3d637cd60bb1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:24.073387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08a8fcb0-86bb-4720-ab9d-3d637cd60bb1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:24.073910 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08a8fcb0-86bb-4720-ab9d-3d637cd60bb1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:24.074344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08a8fcb0-86bb-4720-ab9d-3d637cd60bb1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:24.263557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9b688039-86b0-473a-b1e4-c8ef8a263213 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:24.266516 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-08a8fcb0-86bb-4720-ab9d-3d637cd60bb1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:24.273012 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9b688039-86b0-473a-b1e4-c8ef8a263213 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.217492 Aug 28 01:20:24.273593 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 969/2162] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:24 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:24.275234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-08a8fcb0-86bb-4720-ab9d-3d637cd60bb1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.215176 Aug 28 01:20:24.287710 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1194/2163] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:24 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:24.287937 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-94b99030-1560-4721-ba13-fb2a32165157 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:24.287937 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-94b99030-1560-4721-ba13-fb2a32165157 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:24.288168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-287393ae-1016-477d-aab5-d513b0b9e7e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:24.288168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-287393ae-1016-477d-aab5-d513b0b9e7e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:24.288342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94b99030-1560-4721-ba13-fb2a32165157 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:24.288342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94b99030-1560-4721-ba13-fb2a32165157 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:24.288342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94b99030-1560-4721-ba13-fb2a32165157 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:24.288342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94b99030-1560-4721-ba13-fb2a32165157 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:24.288674 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94b99030-1560-4721-ba13-fb2a32165157 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:24.288674 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94b99030-1560-4721-ba13-fb2a32165157 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:24.288839 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-287393ae-1016-477d-aab5-d513b0b9e7e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:24.288839 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-287393ae-1016-477d-aab5-d513b0b9e7e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:24.288839 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-287393ae-1016-477d-aab5-d513b0b9e7e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:24.289226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-287393ae-1016-477d-aab5-d513b0b9e7e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:24.289523 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-287393ae-1016-477d-aab5-d513b0b9e7e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:24.290002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-287393ae-1016-477d-aab5-d513b0b9e7e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:24.372924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-94b99030-1560-4721-ba13-fb2a32165157 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:24.386003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-94b99030-1560-4721-ba13-fb2a32165157 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.111067 Aug 28 01:20:24.386888 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 970/2164] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:24 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:24.462369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-287393ae-1016-477d-aab5-d513b0b9e7e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:24.471198 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-287393ae-1016-477d-aab5-d513b0b9e7e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:24.472615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-287393ae-1016-477d-aab5-d513b0b9e7e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:24.473780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-287393ae-1016-477d-aab5-d513b0b9e7e4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.196940 Aug 28 01:20:24.474333 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1195/2165] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:24 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:25.295588 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3d84f2bf-0edb-47ab-a713-72fc05759724 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:25.296220 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a4af23aa-cc41-41a5-818b-d3288211f825 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:25.296596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3d84f2bf-0edb-47ab-a713-72fc05759724 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:25.296703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a4af23aa-cc41-41a5-818b-d3288211f825 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:25.303121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a4af23aa-cc41-41a5-818b-d3288211f825 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:25.303298 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d84f2bf-0edb-47ab-a713-72fc05759724 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:25.303598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d84f2bf-0edb-47ab-a713-72fc05759724 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:25.303752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a4af23aa-cc41-41a5-818b-d3288211f825 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:25.304048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d84f2bf-0edb-47ab-a713-72fc05759724 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:25.304214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a4af23aa-cc41-41a5-818b-d3288211f825 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:25.304740 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d84f2bf-0edb-47ab-a713-72fc05759724 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:25.304913 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a4af23aa-cc41-41a5-818b-d3288211f825 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:25.305224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d84f2bf-0edb-47ab-a713-72fc05759724 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:25.305401 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a4af23aa-cc41-41a5-818b-d3288211f825 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:25.305645 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d84f2bf-0edb-47ab-a713-72fc05759724 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:25.305794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a4af23aa-cc41-41a5-818b-d3288211f825 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:25.520061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a4af23aa-cc41-41a5-818b-d3288211f825 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:25.523423 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3d84f2bf-0edb-47ab-a713-72fc05759724 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:25.528496 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a4af23aa-cc41-41a5-818b-d3288211f825 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.236722 Aug 28 01:20:25.529114 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 971/2166] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:25 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:25.533785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0090fee8-4b92-44a6-be50-d323a1312fb8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:25.534388 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0090fee8-4b92-44a6-be50-d323a1312fb8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:25.540484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0090fee8-4b92-44a6-be50-d323a1312fb8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:25.541000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0090fee8-4b92-44a6-be50-d323a1312fb8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:25.541465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0090fee8-4b92-44a6-be50-d323a1312fb8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:25.541625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3d84f2bf-0edb-47ab-a713-72fc05759724 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.249564 Aug 28 01:20:25.542157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0090fee8-4b92-44a6-be50-d323a1312fb8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:25.542504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1196/2167] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:25 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:25.542631 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0090fee8-4b92-44a6-be50-d323a1312fb8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:25.543134 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0090fee8-4b92-44a6-be50-d323a1312fb8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:25.547325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e5ea3d94-6c4a-447b-88bb-a658ea66b812 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:25.548000 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e5ea3d94-6c4a-447b-88bb-a658ea66b812 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:25.554006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e5ea3d94-6c4a-447b-88bb-a658ea66b812 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:25.554533 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e5ea3d94-6c4a-447b-88bb-a658ea66b812 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:25.555044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e5ea3d94-6c4a-447b-88bb-a658ea66b812 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:25.555744 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e5ea3d94-6c4a-447b-88bb-a658ea66b812 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:25.556184 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e5ea3d94-6c4a-447b-88bb-a658ea66b812 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:25.556598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e5ea3d94-6c4a-447b-88bb-a658ea66b812 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:25.606612 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0090fee8-4b92-44a6-be50-d323a1312fb8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:25.614955 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0090fee8-4b92-44a6-be50-d323a1312fb8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.084779 Aug 28 01:20:25.615542 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 972/2168] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:25 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:25.707269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e5ea3d94-6c4a-447b-88bb-a658ea66b812 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:25.715974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e5ea3d94-6c4a-447b-88bb-a658ea66b812 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:25.717441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e5ea3d94-6c4a-447b-88bb-a658ea66b812 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:25.718610 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e5ea3d94-6c4a-447b-88bb-a658ea66b812 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.175089 Aug 28 01:20:25.719160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1197/2169] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:25 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:26.549410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cb8fb384-ad89-45d5-8c70-4c23d5890b75 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:26.550081 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cb8fb384-ad89-45d5-8c70-4c23d5890b75 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:26.556835 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb8fb384-ad89-45d5-8c70-4c23d5890b75 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:26.557353 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb8fb384-ad89-45d5-8c70-4c23d5890b75 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:26.557864 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb8fb384-ad89-45d5-8c70-4c23d5890b75 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:26.558558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb8fb384-ad89-45d5-8c70-4c23d5890b75 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:26.559003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb8fb384-ad89-45d5-8c70-4c23d5890b75 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:26.559442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cb8fb384-ad89-45d5-8c70-4c23d5890b75 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:26.562756 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-60a37fc3-52ea-4889-9dc6-7907f22494a7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:26.563392 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-60a37fc3-52ea-4889-9dc6-7907f22494a7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:26.570560 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60a37fc3-52ea-4889-9dc6-7907f22494a7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:26.571048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60a37fc3-52ea-4889-9dc6-7907f22494a7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:26.571482 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60a37fc3-52ea-4889-9dc6-7907f22494a7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:26.572180 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60a37fc3-52ea-4889-9dc6-7907f22494a7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:26.572650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60a37fc3-52ea-4889-9dc6-7907f22494a7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:26.573090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-60a37fc3-52ea-4889-9dc6-7907f22494a7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:26.758246 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-cb8fb384-ad89-45d5-8c70-4c23d5890b75 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:26.766012 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-60a37fc3-52ea-4889-9dc6-7907f22494a7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:26.766698 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cb8fb384-ad89-45d5-8c70-4c23d5890b75 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.221162 Aug 28 01:20:26.767276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 973/2170] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:26 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:26.771738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-693e824b-ba39-4884-b6e3-6ca41d064fe0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:26.772332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-693e824b-ba39-4884-b6e3-6ca41d064fe0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:26.775809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-60a37fc3-52ea-4889-9dc6-7907f22494a7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.216656 Aug 28 01:20:26.780654 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1198/2171] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:26 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:26.780868 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-693e824b-ba39-4884-b6e3-6ca41d064fe0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:26.780868 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-693e824b-ba39-4884-b6e3-6ca41d064fe0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:26.780868 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-693e824b-ba39-4884-b6e3-6ca41d064fe0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:26.780868 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-693e824b-ba39-4884-b6e3-6ca41d064fe0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:26.781256 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-693e824b-ba39-4884-b6e3-6ca41d064fe0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:26.781256 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-693e824b-ba39-4884-b6e3-6ca41d064fe0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:26.781432 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ea93a3bf-6c91-4b45-b27d-bdface8ba472 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:26.781921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ea93a3bf-6c91-4b45-b27d-bdface8ba472 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:26.788907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea93a3bf-6c91-4b45-b27d-bdface8ba472 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:26.789468 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea93a3bf-6c91-4b45-b27d-bdface8ba472 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:26.789862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea93a3bf-6c91-4b45-b27d-bdface8ba472 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:26.790539 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea93a3bf-6c91-4b45-b27d-bdface8ba472 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:26.790986 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea93a3bf-6c91-4b45-b27d-bdface8ba472 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:26.791402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea93a3bf-6c91-4b45-b27d-bdface8ba472 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:26.866597 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-693e824b-ba39-4884-b6e3-6ca41d064fe0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:26.874861 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-693e824b-ba39-4884-b6e3-6ca41d064fe0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.106616 Aug 28 01:20:26.875656 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 974/2172] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:26 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:26.937242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ea93a3bf-6c91-4b45-b27d-bdface8ba472 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:26.950688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ea93a3bf-6c91-4b45-b27d-bdface8ba472 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:26.952886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ea93a3bf-6c91-4b45-b27d-bdface8ba472 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:26.954689 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ea93a3bf-6c91-4b45-b27d-bdface8ba472 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.176774 Aug 28 01:20:26.955493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1199/2173] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:26 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:27.788563 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-52575d1d-65fe-4190-8cdf-59ade7791456 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:27.789230 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-52575d1d-65fe-4190-8cdf-59ade7791456 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:27.796332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-52575d1d-65fe-4190-8cdf-59ade7791456 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:27.796809 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-52575d1d-65fe-4190-8cdf-59ade7791456 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:27.797270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-52575d1d-65fe-4190-8cdf-59ade7791456 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:27.797988 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-52575d1d-65fe-4190-8cdf-59ade7791456 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:27.798429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-52575d1d-65fe-4190-8cdf-59ade7791456 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:27.798550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d48e7db3-0897-4b1c-83e3-d9014db15720 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:27.799006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-52575d1d-65fe-4190-8cdf-59ade7791456 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:27.799112 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d48e7db3-0897-4b1c-83e3-d9014db15720 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:27.805786 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d48e7db3-0897-4b1c-83e3-d9014db15720 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:27.806263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d48e7db3-0897-4b1c-83e3-d9014db15720 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:27.806690 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d48e7db3-0897-4b1c-83e3-d9014db15720 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:27.807383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d48e7db3-0897-4b1c-83e3-d9014db15720 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:27.807821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d48e7db3-0897-4b1c-83e3-d9014db15720 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:27.808236 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d48e7db3-0897-4b1c-83e3-d9014db15720 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:28.000192 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-52575d1d-65fe-4190-8cdf-59ade7791456 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:28.008932 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-52575d1d-65fe-4190-8cdf-59ade7791456 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.224254 Aug 28 01:20:28.009530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 975/2174] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:27 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:28.013585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d48e7db3-0897-4b1c-83e3-d9014db15720 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:28.014778 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-26057dba-8959-40ee-a96f-53ce02c21572 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:28.015389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-26057dba-8959-40ee-a96f-53ce02c21572 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:28.022018 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26057dba-8959-40ee-a96f-53ce02c21572 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:28.022515 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26057dba-8959-40ee-a96f-53ce02c21572 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:28.022948 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26057dba-8959-40ee-a96f-53ce02c21572 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:28.023256 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d48e7db3-0897-4b1c-83e3-d9014db15720 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.228478 Aug 28 01:20:28.023685 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26057dba-8959-40ee-a96f-53ce02c21572 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:28.024033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1200/2175] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:27 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:28.024192 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26057dba-8959-40ee-a96f-53ce02c21572 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:28.024693 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-26057dba-8959-40ee-a96f-53ce02c21572 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:28.029403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-933313a6-4422-47c4-b508-04996fa40ed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:28.029983 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-933313a6-4422-47c4-b508-04996fa40ed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:28.036359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-933313a6-4422-47c4-b508-04996fa40ed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:28.037168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-933313a6-4422-47c4-b508-04996fa40ed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:28.037605 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-933313a6-4422-47c4-b508-04996fa40ed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:28.038309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-933313a6-4422-47c4-b508-04996fa40ed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:28.038902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-933313a6-4422-47c4-b508-04996fa40ed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:28.039232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-933313a6-4422-47c4-b508-04996fa40ed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:28.102826 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-26057dba-8959-40ee-a96f-53ce02c21572 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:28.111720 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-26057dba-8959-40ee-a96f-53ce02c21572 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.101085 Aug 28 01:20:28.112581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 976/2176] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:28 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:28.194929 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-933313a6-4422-47c4-b508-04996fa40ed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:28.205828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-933313a6-4422-47c4-b508-04996fa40ed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:28.207266 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-933313a6-4422-47c4-b508-04996fa40ed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:28.208434 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-933313a6-4422-47c4-b508-04996fa40ed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.183006 Aug 28 01:20:28.209023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1201/2177] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:28 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:29.030620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b7c79921-bc69-4a81-8bdc-bcb72128523e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:29.031284 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b7c79921-bc69-4a81-8bdc-bcb72128523e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:29.038016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b7c79921-bc69-4a81-8bdc-bcb72128523e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:29.038503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b7c79921-bc69-4a81-8bdc-bcb72128523e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:29.039032 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b7c79921-bc69-4a81-8bdc-bcb72128523e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:29.039760 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b7c79921-bc69-4a81-8bdc-bcb72128523e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:29.040190 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b7c79921-bc69-4a81-8bdc-bcb72128523e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:29.040697 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b7c79921-bc69-4a81-8bdc-bcb72128523e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:29.044330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-13ab341a-0550-463d-8de9-b393fd2717fd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:29.045779 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-13ab341a-0550-463d-8de9-b393fd2717fd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:29.054317 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-13ab341a-0550-463d-8de9-b393fd2717fd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:29.054810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-13ab341a-0550-463d-8de9-b393fd2717fd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:29.055232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-13ab341a-0550-463d-8de9-b393fd2717fd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:29.055940 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-13ab341a-0550-463d-8de9-b393fd2717fd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:29.056414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-13ab341a-0550-463d-8de9-b393fd2717fd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:29.056712 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:20:29.056841 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-13ab341a-0550-463d-8de9-b393fd2717fd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:29.244075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b7c79921-bc69-4a81-8bdc-bcb72128523e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:29.251969 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-13ab341a-0550-463d-8de9-b393fd2717fd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:29.253076 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b7c79921-bc69-4a81-8bdc-bcb72128523e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.226467 Aug 28 01:20:29.253662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 977/2178] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:29 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:29.258380 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b6b11c00-aa39-4a40-b3fc-09e7f02925a7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:29.258989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b6b11c00-aa39-4a40-b3fc-09e7f02925a7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:29.265260 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6b11c00-aa39-4a40-b3fc-09e7f02925a7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:29.265760 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6b11c00-aa39-4a40-b3fc-09e7f02925a7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:29.266168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6b11c00-aa39-4a40-b3fc-09e7f02925a7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:29.266327 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-13ab341a-0550-463d-8de9-b393fd2717fd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.225483 Aug 28 01:20:29.267181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1202/2179] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:29 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:20:29.267332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6b11c00-aa39-4a40-b3fc-09e7f02925a7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:29.267777 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6b11c00-aa39-4a40-b3fc-09e7f02925a7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:29.268206 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6b11c00-aa39-4a40-b3fc-09e7f02925a7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:29.271770 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6803d80f-f42d-42e2-8fc1-93cc2e24006d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:29.272407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6803d80f-f42d-42e2-8fc1-93cc2e24006d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:29.280339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6803d80f-f42d-42e2-8fc1-93cc2e24006d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:29.280887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6803d80f-f42d-42e2-8fc1-93cc2e24006d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:29.281312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6803d80f-f42d-42e2-8fc1-93cc2e24006d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:29.282258 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6803d80f-f42d-42e2-8fc1-93cc2e24006d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:29.282707 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6803d80f-f42d-42e2-8fc1-93cc2e24006d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:29.283134 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6803d80f-f42d-42e2-8fc1-93cc2e24006d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:29.343925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b6b11c00-aa39-4a40-b3fc-09e7f02925a7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:29.352597 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b6b11c00-aa39-4a40-b3fc-09e7f02925a7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.097875 Aug 28 01:20:29.361039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 978/2180] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:29 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:29.437375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6803d80f-f42d-42e2-8fc1-93cc2e24006d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:29.446742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6803d80f-f42d-42e2-8fc1-93cc2e24006d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:29.448634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6803d80f-f42d-42e2-8fc1-93cc2e24006d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:29.450336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6803d80f-f42d-42e2-8fc1-93cc2e24006d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.182172 Aug 28 01:20:29.450986 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1203/2181] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:29 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:30.274960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c44a3f71-1572-4413-add5-1ac59b363c50 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:30.275623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c44a3f71-1572-4413-add5-1ac59b363c50 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:30.282480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c44a3f71-1572-4413-add5-1ac59b363c50 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:30.282999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c44a3f71-1572-4413-add5-1ac59b363c50 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:30.283426 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c44a3f71-1572-4413-add5-1ac59b363c50 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:30.284124 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c44a3f71-1572-4413-add5-1ac59b363c50 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:30.284573 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c44a3f71-1572-4413-add5-1ac59b363c50 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:30.284991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c44a3f71-1572-4413-add5-1ac59b363c50 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:30.290327 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8c1b3c27-2ff5-4619-ae56-71e149bdaa81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:30.291012 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8c1b3c27-2ff5-4619-ae56-71e149bdaa81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:30.298424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8c1b3c27-2ff5-4619-ae56-71e149bdaa81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:30.298746 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8c1b3c27-2ff5-4619-ae56-71e149bdaa81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:30.299168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8c1b3c27-2ff5-4619-ae56-71e149bdaa81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:30.299873 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8c1b3c27-2ff5-4619-ae56-71e149bdaa81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:30.300330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8c1b3c27-2ff5-4619-ae56-71e149bdaa81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:30.300751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8c1b3c27-2ff5-4619-ae56-71e149bdaa81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:30.487728 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c44a3f71-1572-4413-add5-1ac59b363c50 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:30.497831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8c1b3c27-2ff5-4619-ae56-71e149bdaa81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:30.499322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c44a3f71-1572-4413-add5-1ac59b363c50 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.228180 Aug 28 01:20:30.500278 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 979/2182] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:30 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:30.504954 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-303e4a75-8e52-4d0d-8e7d-3f5192e11b2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:30.505548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-303e4a75-8e52-4d0d-8e7d-3f5192e11b2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:30.507117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8c1b3c27-2ff5-4619-ae56-71e149bdaa81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.221051 Aug 28 01:20:30.516303 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1204/2183] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:30 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:30.516530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-303e4a75-8e52-4d0d-8e7d-3f5192e11b2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:30.516530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-303e4a75-8e52-4d0d-8e7d-3f5192e11b2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:30.516530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-303e4a75-8e52-4d0d-8e7d-3f5192e11b2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:30.516835 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a9417617-09e4-43ac-82e0-795ad8d9f3ee tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:30.516944 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-303e4a75-8e52-4d0d-8e7d-3f5192e11b2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:30.517082 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a9417617-09e4-43ac-82e0-795ad8d9f3ee tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:30.517183 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-303e4a75-8e52-4d0d-8e7d-3f5192e11b2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:30.517183 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-303e4a75-8e52-4d0d-8e7d-3f5192e11b2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:30.520319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9417617-09e4-43ac-82e0-795ad8d9f3ee tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:30.520975 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9417617-09e4-43ac-82e0-795ad8d9f3ee tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:30.521472 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9417617-09e4-43ac-82e0-795ad8d9f3ee tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:30.522320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9417617-09e4-43ac-82e0-795ad8d9f3ee tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:30.522889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9417617-09e4-43ac-82e0-795ad8d9f3ee tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:30.523433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9417617-09e4-43ac-82e0-795ad8d9f3ee tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:30.585861 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-303e4a75-8e52-4d0d-8e7d-3f5192e11b2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:30.594294 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-303e4a75-8e52-4d0d-8e7d-3f5192e11b2c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.093029 Aug 28 01:20:30.594862 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 980/2184] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:30 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:30.669598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a9417617-09e4-43ac-82e0-795ad8d9f3ee tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:30.678317 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a9417617-09e4-43ac-82e0-795ad8d9f3ee tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:30.679769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a9417617-09e4-43ac-82e0-795ad8d9f3ee tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:30.680905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a9417617-09e4-43ac-82e0-795ad8d9f3ee tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.172179 Aug 28 01:20:30.681503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1205/2185] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:30 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:31.520908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d5649ab8-b8ac-4a46-b892-31d4bd45bb2e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:31.521592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d5649ab8-b8ac-4a46-b892-31d4bd45bb2e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:31.523867 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=96 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:20:31.527956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-18819cb8-c02f-4a65-9fde-106ae6589753 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:31.528559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-18819cb8-c02f-4a65-9fde-106ae6589753 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:31.530140 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d5649ab8-b8ac-4a46-b892-31d4bd45bb2e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:31.530696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d5649ab8-b8ac-4a46-b892-31d4bd45bb2e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:31.531130 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d5649ab8-b8ac-4a46-b892-31d4bd45bb2e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:31.531818 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d5649ab8-b8ac-4a46-b892-31d4bd45bb2e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:31.532258 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d5649ab8-b8ac-4a46-b892-31d4bd45bb2e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:31.532663 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d5649ab8-b8ac-4a46-b892-31d4bd45bb2e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:31.535237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-18819cb8-c02f-4a65-9fde-106ae6589753 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:31.535713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-18819cb8-c02f-4a65-9fde-106ae6589753 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:31.536139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-18819cb8-c02f-4a65-9fde-106ae6589753 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:31.536859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-18819cb8-c02f-4a65-9fde-106ae6589753 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:31.537288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-18819cb8-c02f-4a65-9fde-106ae6589753 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:31.537702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-18819cb8-c02f-4a65-9fde-106ae6589753 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:31.540398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:20:31.726470 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d5649ab8-b8ac-4a46-b892-31d4bd45bb2e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:31.729306 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-18819cb8-c02f-4a65-9fde-106ae6589753 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:31.735806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d5649ab8-b8ac-4a46-b892-31d4bd45bb2e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.218747 Aug 28 01:20:31.749194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 981/2186] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:31 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:31.749435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-18819cb8-c02f-4a65-9fde-106ae6589753 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.213892 Aug 28 01:20:31.749435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1206/2187] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:31 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:31.749623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-693ea3fd-68a1-43d7-8cd3-4a0da603733b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:31.749623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-693ea3fd-68a1-43d7-8cd3-4a0da603733b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:31.749795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3b55209e-7be2-4874-98e6-d3df822fe513 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:31.749795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3b55209e-7be2-4874-98e6-d3df822fe513 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:31.749962 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-693ea3fd-68a1-43d7-8cd3-4a0da603733b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:31.749962 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-693ea3fd-68a1-43d7-8cd3-4a0da603733b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:31.749962 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-693ea3fd-68a1-43d7-8cd3-4a0da603733b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:31.750197 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3b55209e-7be2-4874-98e6-d3df822fe513 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:31.750197 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3b55209e-7be2-4874-98e6-d3df822fe513 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:31.750373 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-693ea3fd-68a1-43d7-8cd3-4a0da603733b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:31.750459 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3b55209e-7be2-4874-98e6-d3df822fe513 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:31.750548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-693ea3fd-68a1-43d7-8cd3-4a0da603733b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:31.750548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-693ea3fd-68a1-43d7-8cd3-4a0da603733b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:31.750712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3b55209e-7be2-4874-98e6-d3df822fe513 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:31.751692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3b55209e-7be2-4874-98e6-d3df822fe513 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:31.751814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3b55209e-7be2-4874-98e6-d3df822fe513 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:31.836524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-693ea3fd-68a1-43d7-8cd3-4a0da603733b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:31.845180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-693ea3fd-68a1-43d7-8cd3-4a0da603733b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.107722 Aug 28 01:20:31.845761 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 982/2188] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:31 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:31.908323 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3b55209e-7be2-4874-98e6-d3df822fe513 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:31.921043 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3b55209e-7be2-4874-98e6-d3df822fe513 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:31.922476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3b55209e-7be2-4874-98e6-d3df822fe513 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:31.923627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3b55209e-7be2-4874-98e6-d3df822fe513 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.184425 Aug 28 01:20:31.924184 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1207/2189] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:31 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:32.759992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a18de8d3-e54a-4b43-9bb3-81c46a752c17 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:32.760675 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-722a1be5-2859-4b71-9607-62f8d4c71117 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:32.760675 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-722a1be5-2859-4b71-9607-62f8d4c71117 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:32.761172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a18de8d3-e54a-4b43-9bb3-81c46a752c17 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:32.762865 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=96 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:20:32.767685 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-722a1be5-2859-4b71-9607-62f8d4c71117 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:32.768170 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-722a1be5-2859-4b71-9607-62f8d4c71117 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:32.768433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a18de8d3-e54a-4b43-9bb3-81c46a752c17 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:32.768624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-722a1be5-2859-4b71-9607-62f8d4c71117 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:32.768921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a18de8d3-e54a-4b43-9bb3-81c46a752c17 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:32.769314 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-722a1be5-2859-4b71-9607-62f8d4c71117 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:32.769462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a18de8d3-e54a-4b43-9bb3-81c46a752c17 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:32.769762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-722a1be5-2859-4b71-9607-62f8d4c71117 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:32.770042 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a18de8d3-e54a-4b43-9bb3-81c46a752c17 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:32.770157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-722a1be5-2859-4b71-9607-62f8d4c71117 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:32.770492 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a18de8d3-e54a-4b43-9bb3-81c46a752c17 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:32.770912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a18de8d3-e54a-4b43-9bb3-81c46a752c17 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:32.967885 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-722a1be5-2859-4b71-9607-62f8d4c71117 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:32.968046 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a18de8d3-e54a-4b43-9bb3-81c46a752c17 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:32.972779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-722a1be5-2859-4b71-9607-62f8d4c71117 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.216760 Aug 28 01:20:32.973359 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 983/2190] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:32 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:32.977690 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a18de8d3-e54a-4b43-9bb3-81c46a752c17 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.221660 Aug 28 01:20:32.977942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-af4430c9-ce59-48c8-80d5-a862f3846afb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:32.989179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1208/2191] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:32 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:32.989432 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-af4430c9-ce59-48c8-80d5-a862f3846afb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:32.989541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a4d501d6-dd81-4d43-bc65-415af532bedf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:32.989541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a4d501d6-dd81-4d43-bc65-415af532bedf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:32.989714 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af4430c9-ce59-48c8-80d5-a862f3846afb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:32.989714 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af4430c9-ce59-48c8-80d5-a862f3846afb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:32.989714 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af4430c9-ce59-48c8-80d5-a862f3846afb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:32.989714 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af4430c9-ce59-48c8-80d5-a862f3846afb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:32.990049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af4430c9-ce59-48c8-80d5-a862f3846afb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:32.990049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-af4430c9-ce59-48c8-80d5-a862f3846afb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:32.991469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4d501d6-dd81-4d43-bc65-415af532bedf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:32.992036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4d501d6-dd81-4d43-bc65-415af532bedf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:32.992528 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4d501d6-dd81-4d43-bc65-415af532bedf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:32.993274 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4d501d6-dd81-4d43-bc65-415af532bedf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:32.993741 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4d501d6-dd81-4d43-bc65-415af532bedf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:32.994184 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4d501d6-dd81-4d43-bc65-415af532bedf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:33.139139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a4d501d6-dd81-4d43-bc65-415af532bedf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:33.148744 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a4d501d6-dd81-4d43-bc65-415af532bedf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:33.150273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a4d501d6-dd81-4d43-bc65-415af532bedf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:33.151458 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a4d501d6-dd81-4d43-bc65-415af532bedf tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.172027 Aug 28 01:20:33.152013 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1209/2192] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:32 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:33.206903 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-af4430c9-ce59-48c8-80d5-a862f3846afb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:33.220239 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-af4430c9-ce59-48c8-80d5-a862f3846afb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.245608 Aug 28 01:20:33.221036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 984/2193] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:32 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:33.997299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-851de8f1-22da-4b4f-a020-6abe3a6f70f0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:33.997951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-851de8f1-22da-4b4f-a020-6abe3a6f70f0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:34.000185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3f30d62c-e635-41e2-93b5-e314fce7664b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:34.000833 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3f30d62c-e635-41e2-93b5-e314fce7664b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:34.004767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-851de8f1-22da-4b4f-a020-6abe3a6f70f0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:34.005281 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-851de8f1-22da-4b4f-a020-6abe3a6f70f0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:34.005707 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-851de8f1-22da-4b4f-a020-6abe3a6f70f0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:34.006373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-851de8f1-22da-4b4f-a020-6abe3a6f70f0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:34.006846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-851de8f1-22da-4b4f-a020-6abe3a6f70f0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:34.007259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-851de8f1-22da-4b4f-a020-6abe3a6f70f0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:34.007594 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3f30d62c-e635-41e2-93b5-e314fce7664b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:34.008099 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3f30d62c-e635-41e2-93b5-e314fce7664b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:34.008527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3f30d62c-e635-41e2-93b5-e314fce7664b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:34.009249 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3f30d62c-e635-41e2-93b5-e314fce7664b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:34.009697 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3f30d62c-e635-41e2-93b5-e314fce7664b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:34.010106 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3f30d62c-e635-41e2-93b5-e314fce7664b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:34.206027 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-851de8f1-22da-4b4f-a020-6abe3a6f70f0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:34.209267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3f30d62c-e635-41e2-93b5-e314fce7664b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:34.214831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-851de8f1-22da-4b4f-a020-6abe3a6f70f0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.221399 Aug 28 01:20:34.215447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1210/2194] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:33 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:34.218241 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3f30d62c-e635-41e2-93b5-e314fce7664b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.222095 Aug 28 01:20:34.218927 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 985/2195] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:33 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:34.220087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-48c24f1d-b701-4ede-8e0d-bb488b01f22a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:34.220695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-48c24f1d-b701-4ede-8e0d-bb488b01f22a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:34.226630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48c24f1d-b701-4ede-8e0d-bb488b01f22a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:34.227108 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48c24f1d-b701-4ede-8e0d-bb488b01f22a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:34.227538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48c24f1d-b701-4ede-8e0d-bb488b01f22a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:34.228227 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48c24f1d-b701-4ede-8e0d-bb488b01f22a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:34.228672 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48c24f1d-b701-4ede-8e0d-bb488b01f22a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:34.229156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48c24f1d-b701-4ede-8e0d-bb488b01f22a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:34.245580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ac7438cb-9a55-40c5-8ded-d213c9aab9ab tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:34.246181 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ac7438cb-9a55-40c5-8ded-d213c9aab9ab tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:34.252317 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac7438cb-9a55-40c5-8ded-d213c9aab9ab tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:34.252722 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac7438cb-9a55-40c5-8ded-d213c9aab9ab tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:34.253178 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac7438cb-9a55-40c5-8ded-d213c9aab9ab tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:34.253860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac7438cb-9a55-40c5-8ded-d213c9aab9ab tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:34.254308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac7438cb-9a55-40c5-8ded-d213c9aab9ab tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:34.254696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac7438cb-9a55-40c5-8ded-d213c9aab9ab tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:34.326370 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ac7438cb-9a55-40c5-8ded-d213c9aab9ab tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:34.335346 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ac7438cb-9a55-40c5-8ded-d213c9aab9ab tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.093557 Aug 28 01:20:34.335884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 986/2196] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:34 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:20:34.390982 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-48c24f1d-b701-4ede-8e0d-bb488b01f22a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:34.403836 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-48c24f1d-b701-4ede-8e0d-bb488b01f22a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:34.405296 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-48c24f1d-b701-4ede-8e0d-bb488b01f22a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:34.406450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-48c24f1d-b701-4ede-8e0d-bb488b01f22a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.190101 Aug 28 01:20:34.407052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1211/2197] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:34 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:35.236803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e6a203d1-7125-4b2f-901c-1816ebdfc312 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:35.237424 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e6a203d1-7125-4b2f-901c-1816ebdfc312 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:35.240922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-906e7f62-0126-4aea-b6e9-9e9f589efbdd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:35.241567 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-906e7f62-0126-4aea-b6e9-9e9f589efbdd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:35.244205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6a203d1-7125-4b2f-901c-1816ebdfc312 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:35.244689 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6a203d1-7125-4b2f-901c-1816ebdfc312 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:35.245145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6a203d1-7125-4b2f-901c-1816ebdfc312 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:35.245851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6a203d1-7125-4b2f-901c-1816ebdfc312 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:35.246304 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6a203d1-7125-4b2f-901c-1816ebdfc312 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:35.246719 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6a203d1-7125-4b2f-901c-1816ebdfc312 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:35.248175 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-906e7f62-0126-4aea-b6e9-9e9f589efbdd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:35.248678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-906e7f62-0126-4aea-b6e9-9e9f589efbdd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:35.249108 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-906e7f62-0126-4aea-b6e9-9e9f589efbdd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:35.249890 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-906e7f62-0126-4aea-b6e9-9e9f589efbdd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:35.250336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-906e7f62-0126-4aea-b6e9-9e9f589efbdd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:35.250755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-906e7f62-0126-4aea-b6e9-9e9f589efbdd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:35.448626 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e6a203d1-7125-4b2f-901c-1816ebdfc312 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:35.452035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-906e7f62-0126-4aea-b6e9-9e9f589efbdd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:35.457469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e6a203d1-7125-4b2f-901c-1816ebdfc312 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.224504 Aug 28 01:20:35.458041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 987/2198] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:35 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:35.471160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-906e7f62-0126-4aea-b6e9-9e9f589efbdd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.225346 Aug 28 01:20:35.471160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1212/2199] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:35 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:35.471350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-98766b01-f5fb-46b4-8f6a-07c74fc9a3f0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:35.471350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-98766b01-f5fb-46b4-8f6a-07c74fc9a3f0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:35.471531 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a6f1f17d-868f-479b-a554-d0f1e25d0c4e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:35.471531 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a6f1f17d-868f-479b-a554-d0f1e25d0c4e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:35.471698 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98766b01-f5fb-46b4-8f6a-07c74fc9a3f0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:35.471698 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98766b01-f5fb-46b4-8f6a-07c74fc9a3f0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:35.471698 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98766b01-f5fb-46b4-8f6a-07c74fc9a3f0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:35.471698 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98766b01-f5fb-46b4-8f6a-07c74fc9a3f0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:35.472029 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98766b01-f5fb-46b4-8f6a-07c74fc9a3f0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:35.472029 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98766b01-f5fb-46b4-8f6a-07c74fc9a3f0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:35.472760 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6f1f17d-868f-479b-a554-d0f1e25d0c4e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:35.473857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6f1f17d-868f-479b-a554-d0f1e25d0c4e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:35.473857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6f1f17d-868f-479b-a554-d0f1e25d0c4e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:35.474322 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6f1f17d-868f-479b-a554-d0f1e25d0c4e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:35.474787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6f1f17d-868f-479b-a554-d0f1e25d0c4e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:35.475366 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6f1f17d-868f-479b-a554-d0f1e25d0c4e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:35.542128 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-98766b01-f5fb-46b4-8f6a-07c74fc9a3f0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:35.551044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-98766b01-f5fb-46b4-8f6a-07c74fc9a3f0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.091997 Aug 28 01:20:35.551582 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 988/2200] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:35 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:35.630622 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a6f1f17d-868f-479b-a554-d0f1e25d0c4e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:35.639801 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a6f1f17d-868f-479b-a554-d0f1e25d0c4e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:35.641241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a6f1f17d-868f-479b-a554-d0f1e25d0c4e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:35.642422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a6f1f17d-868f-479b-a554-d0f1e25d0c4e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.180078 Aug 28 01:20:35.642985 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1213/2201] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:35 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:36.479074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5b72f422-98b8-4188-829e-81d0f09a3ed7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:36.479743 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5b72f422-98b8-4188-829e-81d0f09a3ed7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:36.482962 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-debd498b-f45f-4b18-8438-843b3619bd39 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:36.483567 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-debd498b-f45f-4b18-8438-843b3619bd39 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:36.486679 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b72f422-98b8-4188-829e-81d0f09a3ed7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:36.487187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b72f422-98b8-4188-829e-81d0f09a3ed7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:36.487613 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b72f422-98b8-4188-829e-81d0f09a3ed7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:36.488314 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b72f422-98b8-4188-829e-81d0f09a3ed7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:36.488764 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b72f422-98b8-4188-829e-81d0f09a3ed7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:36.489195 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b72f422-98b8-4188-829e-81d0f09a3ed7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:36.490262 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-debd498b-f45f-4b18-8438-843b3619bd39 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:36.490752 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-debd498b-f45f-4b18-8438-843b3619bd39 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:36.491187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-debd498b-f45f-4b18-8438-843b3619bd39 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:36.491885 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-debd498b-f45f-4b18-8438-843b3619bd39 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:36.492345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-debd498b-f45f-4b18-8438-843b3619bd39 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:36.492768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-debd498b-f45f-4b18-8438-843b3619bd39 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:36.678258 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5b72f422-98b8-4188-829e-81d0f09a3ed7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:36.682744 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-debd498b-f45f-4b18-8438-843b3619bd39 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:36.689164 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5b72f422-98b8-4188-829e-81d0f09a3ed7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.213997 Aug 28 01:20:36.689699 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 989/2202] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:36 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:36.692329 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-debd498b-f45f-4b18-8438-843b3619bd39 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.213253 Aug 28 01:20:36.704171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1214/2203] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:36 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:36.704377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1db8e1c8-e766-4668-9139-6e2169e525f6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:36.704377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1db8e1c8-e766-4668-9139-6e2169e525f6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:36.704575 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-74275841-9639-4a99-81f0-89e746f560c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:36.704575 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-74275841-9639-4a99-81f0-89e746f560c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:36.704755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1db8e1c8-e766-4668-9139-6e2169e525f6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:36.704755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1db8e1c8-e766-4668-9139-6e2169e525f6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:36.704755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1db8e1c8-e766-4668-9139-6e2169e525f6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:36.704755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1db8e1c8-e766-4668-9139-6e2169e525f6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:36.705107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1db8e1c8-e766-4668-9139-6e2169e525f6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:36.705107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1db8e1c8-e766-4668-9139-6e2169e525f6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:36.705276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-74275841-9639-4a99-81f0-89e746f560c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:36.705276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-74275841-9639-4a99-81f0-89e746f560c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:36.705276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-74275841-9639-4a99-81f0-89e746f560c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:36.705929 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-74275841-9639-4a99-81f0-89e746f560c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:36.706157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-74275841-9639-4a99-81f0-89e746f560c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:36.706607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-74275841-9639-4a99-81f0-89e746f560c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:36.777683 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1db8e1c8-e766-4668-9139-6e2169e525f6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:36.786187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1db8e1c8-e766-4668-9139-6e2169e525f6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.095458 Aug 28 01:20:36.786778 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 990/2204] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:36 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:36.867461 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-74275841-9639-4a99-81f0-89e746f560c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:36.876425 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-74275841-9639-4a99-81f0-89e746f560c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:36.877892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-74275841-9639-4a99-81f0-89e746f560c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:36.879688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-74275841-9639-4a99-81f0-89e746f560c4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.185618 Aug 28 01:20:36.880249 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1215/2205] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:36 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:37.714436 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9be42078-b7ea-405e-9a56-12b8db67a479 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:37.715501 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9be42078-b7ea-405e-9a56-12b8db67a479 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:37.716619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-44d85cc4-7dea-4e7c-9539-145931dce972 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:37.717602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-44d85cc4-7dea-4e7c-9539-145931dce972 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:37.724765 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9be42078-b7ea-405e-9a56-12b8db67a479 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:37.725540 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9be42078-b7ea-405e-9a56-12b8db67a479 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:37.726837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9be42078-b7ea-405e-9a56-12b8db67a479 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:37.727225 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9be42078-b7ea-405e-9a56-12b8db67a479 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:37.727536 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-44d85cc4-7dea-4e7c-9539-145931dce972 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:37.727934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9be42078-b7ea-405e-9a56-12b8db67a479 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:37.728267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-44d85cc4-7dea-4e7c-9539-145931dce972 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:37.728564 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9be42078-b7ea-405e-9a56-12b8db67a479 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:37.729104 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-44d85cc4-7dea-4e7c-9539-145931dce972 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:37.730011 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-44d85cc4-7dea-4e7c-9539-145931dce972 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:37.730824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-44d85cc4-7dea-4e7c-9539-145931dce972 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:37.731473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-44d85cc4-7dea-4e7c-9539-145931dce972 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:37.922306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9be42078-b7ea-405e-9a56-12b8db67a479 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:37.925548 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-44d85cc4-7dea-4e7c-9539-145931dce972 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:37.931234 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9be42078-b7ea-405e-9a56-12b8db67a479 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.222682 Aug 28 01:20:37.931775 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 991/2206] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:37 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:37.934254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-44d85cc4-7dea-4e7c-9539-145931dce972 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.223504 Aug 28 01:20:37.935033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1216/2207] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:37 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:37.936327 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7ddbc1bb-0446-41c8-9edf-1b253d809659 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:37.936926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7ddbc1bb-0446-41c8-9edf-1b253d809659 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:37.939915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-db5f9791-b39f-4a7b-a9df-7430a71d02fb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:37.940746 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-db5f9791-b39f-4a7b-a9df-7430a71d02fb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:37.943207 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ddbc1bb-0446-41c8-9edf-1b253d809659 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:37.943687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ddbc1bb-0446-41c8-9edf-1b253d809659 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:37.944601 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ddbc1bb-0446-41c8-9edf-1b253d809659 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:37.945387 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ddbc1bb-0446-41c8-9edf-1b253d809659 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:37.945840 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ddbc1bb-0446-41c8-9edf-1b253d809659 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:37.946289 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ddbc1bb-0446-41c8-9edf-1b253d809659 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:37.948604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db5f9791-b39f-4a7b-a9df-7430a71d02fb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:37.948604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db5f9791-b39f-4a7b-a9df-7430a71d02fb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:37.949069 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db5f9791-b39f-4a7b-a9df-7430a71d02fb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:37.949789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db5f9791-b39f-4a7b-a9df-7430a71d02fb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:37.950245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db5f9791-b39f-4a7b-a9df-7430a71d02fb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:37.950715 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db5f9791-b39f-4a7b-a9df-7430a71d02fb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:38.033771 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7ddbc1bb-0446-41c8-9edf-1b253d809659 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:38.042246 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7ddbc1bb-0446-41c8-9edf-1b253d809659 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.109434 Aug 28 01:20:38.042855 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 992/2208] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:37 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:38.109915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-db5f9791-b39f-4a7b-a9df-7430a71d02fb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:38.119264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-db5f9791-b39f-4a7b-a9df-7430a71d02fb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:38.120691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-db5f9791-b39f-4a7b-a9df-7430a71d02fb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:38.121897 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-db5f9791-b39f-4a7b-a9df-7430a71d02fb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.185798 Aug 28 01:20:38.122592 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1217/2209] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:37 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:38.958033 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-06b2ce90-a05a-42a3-9564-9a611642e24d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:38.959029 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b94ee7ca-b02f-446c-a19b-f723931857dc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:38.959705 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-06b2ce90-a05a-42a3-9564-9a611642e24d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:38.959882 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b94ee7ca-b02f-446c-a19b-f723931857dc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:38.969170 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-06b2ce90-a05a-42a3-9564-9a611642e24d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:38.969384 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b94ee7ca-b02f-446c-a19b-f723931857dc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:38.969653 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-06b2ce90-a05a-42a3-9564-9a611642e24d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:38.969821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b94ee7ca-b02f-446c-a19b-f723931857dc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:38.970075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-06b2ce90-a05a-42a3-9564-9a611642e24d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:38.970231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b94ee7ca-b02f-446c-a19b-f723931857dc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:38.970751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b94ee7ca-b02f-446c-a19b-f723931857dc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:38.970903 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-06b2ce90-a05a-42a3-9564-9a611642e24d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:38.971213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b94ee7ca-b02f-446c-a19b-f723931857dc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:38.971383 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-06b2ce90-a05a-42a3-9564-9a611642e24d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:38.971641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b94ee7ca-b02f-446c-a19b-f723931857dc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:38.971815 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-06b2ce90-a05a-42a3-9564-9a611642e24d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:39.164408 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-06b2ce90-a05a-42a3-9564-9a611642e24d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:39.167394 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b94ee7ca-b02f-446c-a19b-f723931857dc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:39.174394 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-06b2ce90-a05a-42a3-9564-9a611642e24d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.223028 Aug 28 01:20:39.174998 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 993/2210] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:38 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:39.175991 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b94ee7ca-b02f-446c-a19b-f723931857dc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.223058 Aug 28 01:20:39.188990 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1218/2211] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:38 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:39.189258 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8b8aa46a-052e-4263-a642-bf1b502cfd18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:39.189258 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8b8aa46a-052e-4263-a642-bf1b502cfd18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:39.189432 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-88a4522b-5c29-4fe2-9fe1-dadc85e6c779 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:39.189432 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-88a4522b-5c29-4fe2-9fe1-dadc85e6c779 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:39.189636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b8aa46a-052e-4263-a642-bf1b502cfd18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:39.189636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b8aa46a-052e-4263-a642-bf1b502cfd18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:39.189636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b8aa46a-052e-4263-a642-bf1b502cfd18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:39.189636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b8aa46a-052e-4263-a642-bf1b502cfd18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:39.190020 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b8aa46a-052e-4263-a642-bf1b502cfd18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:39.190124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88a4522b-5c29-4fe2-9fe1-dadc85e6c779 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:39.190309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b8aa46a-052e-4263-a642-bf1b502cfd18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:39.190483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88a4522b-5c29-4fe2-9fe1-dadc85e6c779 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:39.190712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88a4522b-5c29-4fe2-9fe1-dadc85e6c779 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:39.191442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88a4522b-5c29-4fe2-9fe1-dadc85e6c779 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:39.192227 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88a4522b-5c29-4fe2-9fe1-dadc85e6c779 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:39.192643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88a4522b-5c29-4fe2-9fe1-dadc85e6c779 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:39.193066 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:20:39.276647 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8b8aa46a-052e-4263-a642-bf1b502cfd18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:39.288736 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8b8aa46a-052e-4263-a642-bf1b502cfd18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.112610 Aug 28 01:20:39.289346 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 994/2212] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:39 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:39.350571 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-88a4522b-5c29-4fe2-9fe1-dadc85e6c779 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:39.361845 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-88a4522b-5c29-4fe2-9fe1-dadc85e6c779 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:39.363331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-88a4522b-5c29-4fe2-9fe1-dadc85e6c779 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:39.364459 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-88a4522b-5c29-4fe2-9fe1-dadc85e6c779 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.186669 Aug 28 01:20:39.365064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1219/2213] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:39 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:40.197041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-93921afa-a9a3-40eb-87aa-70a2a95d449b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:40.197610 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b315a5be-2537-4dbc-b5b6-7adfe21a79ef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:40.198001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-93921afa-a9a3-40eb-87aa-70a2a95d449b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:40.198101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b315a5be-2537-4dbc-b5b6-7adfe21a79ef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:40.204421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b315a5be-2537-4dbc-b5b6-7adfe21a79ef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:40.204623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-93921afa-a9a3-40eb-87aa-70a2a95d449b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:40.204907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b315a5be-2537-4dbc-b5b6-7adfe21a79ef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:40.205134 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-93921afa-a9a3-40eb-87aa-70a2a95d449b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:40.205386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b315a5be-2537-4dbc-b5b6-7adfe21a79ef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:40.205542 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-93921afa-a9a3-40eb-87aa-70a2a95d449b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:40.206078 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b315a5be-2537-4dbc-b5b6-7adfe21a79ef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:40.206232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-93921afa-a9a3-40eb-87aa-70a2a95d449b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:40.206527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b315a5be-2537-4dbc-b5b6-7adfe21a79ef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:40.206692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-93921afa-a9a3-40eb-87aa-70a2a95d449b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:40.206948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b315a5be-2537-4dbc-b5b6-7adfe21a79ef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:40.207100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-93921afa-a9a3-40eb-87aa-70a2a95d449b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:40.671699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b315a5be-2537-4dbc-b5b6-7adfe21a79ef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:40.675834 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-93921afa-a9a3-40eb-87aa-70a2a95d449b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:40.682630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b315a5be-2537-4dbc-b5b6-7adfe21a79ef tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.489395 Aug 28 01:20:40.683223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1220/2214] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:40 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:40.693243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-93921afa-a9a3-40eb-87aa-70a2a95d449b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.491868 Aug 28 01:20:40.693243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 995/2215] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:40 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 493 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:40.693482 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-dafb86cc-ec36-43b9-a7f2-eb2fbc6ae897 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:40.693482 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-dafb86cc-ec36-43b9-a7f2-eb2fbc6ae897 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:40.693646 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7d5acf30-3b7c-45c2-bce4-815488ce25aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:40.693646 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7d5acf30-3b7c-45c2-bce4-815488ce25aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:40.701161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dafb86cc-ec36-43b9-a7f2-eb2fbc6ae897 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:40.701161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dafb86cc-ec36-43b9-a7f2-eb2fbc6ae897 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:40.701161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dafb86cc-ec36-43b9-a7f2-eb2fbc6ae897 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:40.701652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7d5acf30-3b7c-45c2-bce4-815488ce25aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:40.701836 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dafb86cc-ec36-43b9-a7f2-eb2fbc6ae897 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:40.702111 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7d5acf30-3b7c-45c2-bce4-815488ce25aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:40.702214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dafb86cc-ec36-43b9-a7f2-eb2fbc6ae897 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:40.702310 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7d5acf30-3b7c-45c2-bce4-815488ce25aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:40.702397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dafb86cc-ec36-43b9-a7f2-eb2fbc6ae897 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:40.702499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7d5acf30-3b7c-45c2-bce4-815488ce25aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:40.702499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7d5acf30-3b7c-45c2-bce4-815488ce25aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:40.702499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7d5acf30-3b7c-45c2-bce4-815488ce25aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:40.786790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dafb86cc-ec36-43b9-a7f2-eb2fbc6ae897 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:40.795150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-dafb86cc-ec36-43b9-a7f2-eb2fbc6ae897 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.110847 Aug 28 01:20:40.795819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1221/2216] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:40 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:40.856610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7d5acf30-3b7c-45c2-bce4-815488ce25aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:40.867899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7d5acf30-3b7c-45c2-bce4-815488ce25aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:40.869276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7d5acf30-3b7c-45c2-bce4-815488ce25aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:40.870459 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7d5acf30-3b7c-45c2-bce4-815488ce25aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.183926 Aug 28 01:20:40.871052 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 996/2217] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:40 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:41.709421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7a0d6e69-0f51-442a-ad8d-830bff3f3431 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:41.710261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7a0d6e69-0f51-442a-ad8d-830bff3f3431 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:41.710546 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b1f884bc-8ea3-44b5-abb4-7b4191a502e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:41.711155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b1f884bc-8ea3-44b5-abb4-7b4191a502e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:41.716913 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7a0d6e69-0f51-442a-ad8d-830bff3f3431 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:41.717435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7a0d6e69-0f51-442a-ad8d-830bff3f3431 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:41.717868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7a0d6e69-0f51-442a-ad8d-830bff3f3431 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:41.718033 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b1f884bc-8ea3-44b5-abb4-7b4191a502e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:41.718480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b1f884bc-8ea3-44b5-abb4-7b4191a502e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:41.718650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7a0d6e69-0f51-442a-ad8d-830bff3f3431 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:41.718885 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b1f884bc-8ea3-44b5-abb4-7b4191a502e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:41.719126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7a0d6e69-0f51-442a-ad8d-830bff3f3431 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:41.719462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7a0d6e69-0f51-442a-ad8d-830bff3f3431 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:41.719632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b1f884bc-8ea3-44b5-abb4-7b4191a502e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:41.720116 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b1f884bc-8ea3-44b5-abb4-7b4191a502e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:41.720529 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b1f884bc-8ea3-44b5-abb4-7b4191a502e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:41.913312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7a0d6e69-0f51-442a-ad8d-830bff3f3431 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:41.915541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:20:41.916589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b1f884bc-8ea3-44b5-abb4-7b4191a502e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:41.923268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7a0d6e69-0f51-442a-ad8d-830bff3f3431 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.217858 Aug 28 01:20:41.923875 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1222/2218] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:41 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:41.924908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b1f884bc-8ea3-44b5-abb4-7b4191a502e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.218151 Aug 28 01:20:41.936884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 997/2219] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:41 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:41.937125 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5001cbbe-28e8-46ee-8998-79871df16207 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:41.937125 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5001cbbe-28e8-46ee-8998-79871df16207 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:41.937331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-14a3b36e-d06b-4b87-b6e6-3986df37f04b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:41.937331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-14a3b36e-d06b-4b87-b6e6-3986df37f04b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:41.937507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5001cbbe-28e8-46ee-8998-79871df16207 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:41.937507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5001cbbe-28e8-46ee-8998-79871df16207 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:41.937507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5001cbbe-28e8-46ee-8998-79871df16207 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:41.937870 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14a3b36e-d06b-4b87-b6e6-3986df37f04b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:41.937966 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5001cbbe-28e8-46ee-8998-79871df16207 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:41.938052 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14a3b36e-d06b-4b87-b6e6-3986df37f04b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:41.938149 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5001cbbe-28e8-46ee-8998-79871df16207 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:41.938249 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14a3b36e-d06b-4b87-b6e6-3986df37f04b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:41.938341 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5001cbbe-28e8-46ee-8998-79871df16207 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:41.938427 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14a3b36e-d06b-4b87-b6e6-3986df37f04b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:41.938511 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14a3b36e-d06b-4b87-b6e6-3986df37f04b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:41.938920 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-14a3b36e-d06b-4b87-b6e6-3986df37f04b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:42.025459 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5001cbbe-28e8-46ee-8998-79871df16207 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:42.033810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5001cbbe-28e8-46ee-8998-79871df16207 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.108937 Aug 28 01:20:42.034384 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1223/2220] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:41 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:42.093624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-14a3b36e-d06b-4b87-b6e6-3986df37f04b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:42.102377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-14a3b36e-d06b-4b87-b6e6-3986df37f04b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:42.103820 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-14a3b36e-d06b-4b87-b6e6-3986df37f04b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:42.104995 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-14a3b36e-d06b-4b87-b6e6-3986df37f04b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.178584 Aug 28 01:20:42.105773 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 998/2221] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:41 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:42.948907 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ea7fccec-5f90-4e82-936b-ea731c6ed617 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:42.948907 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ea7fccec-5f90-4e82-936b-ea731c6ed617 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:42.955745 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea7fccec-5f90-4e82-936b-ea731c6ed617 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:42.956218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea7fccec-5f90-4e82-936b-ea731c6ed617 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:42.956733 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea7fccec-5f90-4e82-936b-ea731c6ed617 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:42.957454 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea7fccec-5f90-4e82-936b-ea731c6ed617 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:42.957898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea7fccec-5f90-4e82-936b-ea731c6ed617 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:42.958293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea7fccec-5f90-4e82-936b-ea731c6ed617 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:43.058784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9c54d97d-cacd-4236-8192-b0f4d583c20b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:43.059394 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9c54d97d-cacd-4236-8192-b0f4d583c20b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:43.065779 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c54d97d-cacd-4236-8192-b0f4d583c20b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:43.066282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c54d97d-cacd-4236-8192-b0f4d583c20b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:43.067065 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c54d97d-cacd-4236-8192-b0f4d583c20b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:43.067765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c54d97d-cacd-4236-8192-b0f4d583c20b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:43.068210 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c54d97d-cacd-4236-8192-b0f4d583c20b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:43.068638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c54d97d-cacd-4236-8192-b0f4d583c20b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:43.088757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ea7fccec-5f90-4e82-936b-ea731c6ed617 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:43.097319 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ea7fccec-5f90-4e82-936b-ea731c6ed617 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.153096 Aug 28 01:20:43.097895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 999/2222] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:42 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:43.102341 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8281c99c-5057-4de1-8ce3-c60b2841c4e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:43.102945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8281c99c-5057-4de1-8ce3-c60b2841c4e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:43.108982 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8281c99c-5057-4de1-8ce3-c60b2841c4e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:43.109483 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8281c99c-5057-4de1-8ce3-c60b2841c4e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:43.109960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8281c99c-5057-4de1-8ce3-c60b2841c4e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:43.110661 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8281c99c-5057-4de1-8ce3-c60b2841c4e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:43.111114 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8281c99c-5057-4de1-8ce3-c60b2841c4e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:43.111537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8281c99c-5057-4de1-8ce3-c60b2841c4e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:43.190512 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8281c99c-5057-4de1-8ce3-c60b2841c4e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:43.199691 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8281c99c-5057-4de1-8ce3-c60b2841c4e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.100732 Aug 28 01:20:43.200297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1000/2223] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:43 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:43.205069 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4621276e-c02a-4d40-87fc-5f2bab17bf79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:43.205843 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4621276e-c02a-4d40-87fc-5f2bab17bf79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:43.211705 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4621276e-c02a-4d40-87fc-5f2bab17bf79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:43.212537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4621276e-c02a-4d40-87fc-5f2bab17bf79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:43.213207 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4621276e-c02a-4d40-87fc-5f2bab17bf79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:43.214261 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4621276e-c02a-4d40-87fc-5f2bab17bf79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:43.214764 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4621276e-c02a-4d40-87fc-5f2bab17bf79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:43.215181 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4621276e-c02a-4d40-87fc-5f2bab17bf79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:43.223917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9c54d97d-cacd-4236-8192-b0f4d583c20b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:43.232324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9c54d97d-cacd-4236-8192-b0f4d583c20b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.291533 Aug 28 01:20:43.233025 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1224/2224] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:42 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:43.353865 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4621276e-c02a-4d40-87fc-5f2bab17bf79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:43.367524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4621276e-c02a-4d40-87fc-5f2bab17bf79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:43.369817 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4621276e-c02a-4d40-87fc-5f2bab17bf79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:43.371311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4621276e-c02a-4d40-87fc-5f2bab17bf79 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.169600 Aug 28 01:20:43.372097 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1001/2225] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:43 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:44.220336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f5406374-6b05-4142-a767-e4c402d16525 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:44.221173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f5406374-6b05-4142-a767-e4c402d16525 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:44.228087 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f5406374-6b05-4142-a767-e4c402d16525 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:44.228568 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f5406374-6b05-4142-a767-e4c402d16525 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:44.229029 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f5406374-6b05-4142-a767-e4c402d16525 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:44.229750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f5406374-6b05-4142-a767-e4c402d16525 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:44.230186 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f5406374-6b05-4142-a767-e4c402d16525 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:44.230607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f5406374-6b05-4142-a767-e4c402d16525 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:44.241758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-32b02a70-9ce3-40c6-a708-9b0930c5b60b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:44.242317 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-32b02a70-9ce3-40c6-a708-9b0930c5b60b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:44.252476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32b02a70-9ce3-40c6-a708-9b0930c5b60b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:44.253332 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32b02a70-9ce3-40c6-a708-9b0930c5b60b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:44.253979 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32b02a70-9ce3-40c6-a708-9b0930c5b60b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:44.255030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32b02a70-9ce3-40c6-a708-9b0930c5b60b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:44.255202 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32b02a70-9ce3-40c6-a708-9b0930c5b60b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:44.255542 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-32b02a70-9ce3-40c6-a708-9b0930c5b60b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:44.295701 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f5406374-6b05-4142-a767-e4c402d16525 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:44.310583 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f5406374-6b05-4142-a767-e4c402d16525 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.093566 Aug 28 01:20:44.311289 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1002/2226] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:44 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:44.327556 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4d94b38d-e474-4184-bca1-9cb8964242a2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:44.329620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4d94b38d-e474-4184-bca1-9cb8964242a2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:44.337425 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d94b38d-e474-4184-bca1-9cb8964242a2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:44.337928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d94b38d-e474-4184-bca1-9cb8964242a2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:44.338348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d94b38d-e474-4184-bca1-9cb8964242a2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:44.339076 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d94b38d-e474-4184-bca1-9cb8964242a2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:44.339504 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d94b38d-e474-4184-bca1-9cb8964242a2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:44.340112 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d94b38d-e474-4184-bca1-9cb8964242a2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:44.431968 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-32b02a70-9ce3-40c6-a708-9b0930c5b60b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:44.440909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-32b02a70-9ce3-40c6-a708-9b0930c5b60b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.324840 Aug 28 01:20:44.442087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1225/2227] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:44 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:44.446740 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5254d1e1-9597-4401-9018-a5801616cbca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:44.447592 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5254d1e1-9597-4401-9018-a5801616cbca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:44.453416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5254d1e1-9597-4401-9018-a5801616cbca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:44.453905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5254d1e1-9597-4401-9018-a5801616cbca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:44.454340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5254d1e1-9597-4401-9018-a5801616cbca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:44.455104 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5254d1e1-9597-4401-9018-a5801616cbca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:44.455559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5254d1e1-9597-4401-9018-a5801616cbca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:44.456185 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5254d1e1-9597-4401-9018-a5801616cbca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:44.529831 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4d94b38d-e474-4184-bca1-9cb8964242a2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:44.539304 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4d94b38d-e474-4184-bca1-9cb8964242a2 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.226644 Aug 28 01:20:44.539974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1003/2228] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:44 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:44.610066 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5254d1e1-9597-4401-9018-a5801616cbca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:44.619210 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5254d1e1-9597-4401-9018-a5801616cbca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:44.620624 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5254d1e1-9597-4401-9018-a5801616cbca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:44.621815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5254d1e1-9597-4401-9018-a5801616cbca tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.178580 Aug 28 01:20:44.622368 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1226/2229] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:44 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:45.333057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a89c18bb-af1e-4fa8-9ddc-b21256491d55 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:45.333657 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a89c18bb-af1e-4fa8-9ddc-b21256491d55 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:45.340569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a89c18bb-af1e-4fa8-9ddc-b21256491d55 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:45.341074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a89c18bb-af1e-4fa8-9ddc-b21256491d55 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:45.341509 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a89c18bb-af1e-4fa8-9ddc-b21256491d55 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:45.342202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a89c18bb-af1e-4fa8-9ddc-b21256491d55 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:45.342656 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a89c18bb-af1e-4fa8-9ddc-b21256491d55 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:45.343077 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a89c18bb-af1e-4fa8-9ddc-b21256491d55 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:45.409545 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a89c18bb-af1e-4fa8-9ddc-b21256491d55 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:45.418386 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a89c18bb-af1e-4fa8-9ddc-b21256491d55 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089391 Aug 28 01:20:45.418954 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1004/2230] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:45 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:45.465117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7de8c5f4-2dc9-4e1c-9d1e-a7dbadda9b7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:45.465767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7de8c5f4-2dc9-4e1c-9d1e-a7dbadda9b7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:45.473100 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7de8c5f4-2dc9-4e1c-9d1e-a7dbadda9b7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:45.473578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7de8c5f4-2dc9-4e1c-9d1e-a7dbadda9b7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:45.474006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7de8c5f4-2dc9-4e1c-9d1e-a7dbadda9b7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:45.474690 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7de8c5f4-2dc9-4e1c-9d1e-a7dbadda9b7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:45.475172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7de8c5f4-2dc9-4e1c-9d1e-a7dbadda9b7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:45.475593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7de8c5f4-2dc9-4e1c-9d1e-a7dbadda9b7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:45.562009 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c6611505-04f1-4ee1-888c-5ed5e97fa304 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:45.562627 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c6611505-04f1-4ee1-888c-5ed5e97fa304 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:45.569362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6611505-04f1-4ee1-888c-5ed5e97fa304 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:45.569851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6611505-04f1-4ee1-888c-5ed5e97fa304 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:45.570287 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6611505-04f1-4ee1-888c-5ed5e97fa304 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:45.570990 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6611505-04f1-4ee1-888c-5ed5e97fa304 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:45.571439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6611505-04f1-4ee1-888c-5ed5e97fa304 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:45.571859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6611505-04f1-4ee1-888c-5ed5e97fa304 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:45.605976 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7de8c5f4-2dc9-4e1c-9d1e-a7dbadda9b7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:45.615034 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7de8c5f4-2dc9-4e1c-9d1e-a7dbadda9b7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1730 microversion: 2.1 time: 0.154322 Aug 28 01:20:45.615867 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1227/2231] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:45 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1730 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:45.644182 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-45c7499f-02df-422e-be18-30462945ee46 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:45.644789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-45c7499f-02df-422e-be18-30462945ee46 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:45.650903 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45c7499f-02df-422e-be18-30462945ee46 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:45.651382 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45c7499f-02df-422e-be18-30462945ee46 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:45.651826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45c7499f-02df-422e-be18-30462945ee46 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:45.652510 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45c7499f-02df-422e-be18-30462945ee46 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:45.653096 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45c7499f-02df-422e-be18-30462945ee46 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:45.653523 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45c7499f-02df-422e-be18-30462945ee46 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:45.735675 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c6611505-04f1-4ee1-888c-5ed5e97fa304 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:45.747725 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c6611505-04f1-4ee1-888c-5ed5e97fa304 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.189490 Aug 28 01:20:45.748480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1005/2232] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:45 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:45.801939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-45c7499f-02df-422e-be18-30462945ee46 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:45.811156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-45c7499f-02df-422e-be18-30462945ee46 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:45.812594 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-45c7499f-02df-422e-be18-30462945ee46 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:45.815215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-45c7499f-02df-422e-be18-30462945ee46 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.173506 Aug 28 01:20:45.815215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1228/2233] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:45 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 01:20:46.439608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c2023dc9-4a11-4eaa-82e3-9d31d5aa8674 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:46.440460 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c2023dc9-4a11-4eaa-82e3-9d31d5aa8674 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:46.448573 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c2023dc9-4a11-4eaa-82e3-9d31d5aa8674 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:46.449917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c2023dc9-4a11-4eaa-82e3-9d31d5aa8674 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:46.450560 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c2023dc9-4a11-4eaa-82e3-9d31d5aa8674 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:46.451292 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c2023dc9-4a11-4eaa-82e3-9d31d5aa8674 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:46.452026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c2023dc9-4a11-4eaa-82e3-9d31d5aa8674 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:46.452571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c2023dc9-4a11-4eaa-82e3-9d31d5aa8674 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:46.536880 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c2023dc9-4a11-4eaa-82e3-9d31d5aa8674 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:46.545489 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c2023dc9-4a11-4eaa-82e3-9d31d5aa8674 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.109821 Aug 28 01:20:46.546069 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1006/2234] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:46 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:46.636881 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-50b9fb5d-6cbf-4b14-a2a2-2d79a592df5c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:46.637495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-50b9fb5d-6cbf-4b14-a2a2-2d79a592df5c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:46.644774 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-50b9fb5d-6cbf-4b14-a2a2-2d79a592df5c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:46.645283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-50b9fb5d-6cbf-4b14-a2a2-2d79a592df5c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:46.645735 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-50b9fb5d-6cbf-4b14-a2a2-2d79a592df5c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:46.646403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-50b9fb5d-6cbf-4b14-a2a2-2d79a592df5c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:46.646850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-50b9fb5d-6cbf-4b14-a2a2-2d79a592df5c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:46.647295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-50b9fb5d-6cbf-4b14-a2a2-2d79a592df5c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:46.715042 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-50b9fb5d-6cbf-4b14-a2a2-2d79a592df5c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:46.723844 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-50b9fb5d-6cbf-4b14-a2a2-2d79a592df5c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1662 microversion: 2.1 time: 0.091568 Aug 28 01:20:46.724519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1229/2235] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:46 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1662 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:46.769586 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7f8adb19-3374-47da-9a86-0d89d5f62161 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:46.770251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7f8adb19-3374-47da-9a86-0d89d5f62161 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:46.778378 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7f8adb19-3374-47da-9a86-0d89d5f62161 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:46.779338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7f8adb19-3374-47da-9a86-0d89d5f62161 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:46.779881 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7f8adb19-3374-47da-9a86-0d89d5f62161 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:46.780818 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7f8adb19-3374-47da-9a86-0d89d5f62161 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:46.781718 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7f8adb19-3374-47da-9a86-0d89d5f62161 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:46.782182 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7f8adb19-3374-47da-9a86-0d89d5f62161 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:46.829524 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-4bf235b5-6d20-493a-a820-f4b2cdfb19ab req-6c368e80-6c79-4c26-9dec-e9b80653e761 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dff36aeb-f0c7-41f0-8ecc-9924adf8289c", "name": "network-vif-deleted", "tag": "072b8279-6084-43c1-8f0e-c1a705ec9b99"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:20:46.842769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-4bf235b5-6d20-493a-a820-f4b2cdfb19ab req-6c368e80-6c79-4c26-9dec-e9b80653e761 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:46.843249 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-4bf235b5-6d20-493a-a820-f4b2cdfb19ab req-6c368e80-6c79-4c26-9dec-e9b80653e761 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:46.843676 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-4bf235b5-6d20-493a-a820-f4b2cdfb19ab req-6c368e80-6c79-4c26-9dec-e9b80653e761 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:46.863520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-4bf235b5-6d20-493a-a820-f4b2cdfb19ab req-6c368e80-6c79-4c26-9dec-e9b80653e761 service nova] Creating event network-vif-deleted:072b8279-6084-43c1-8f0e-c1a705ec9b99 for instance dff36aeb-f0c7-41f0-8ecc-9924adf8289c on hv-cinder-maste Aug 28 01:20:46.871367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-4bf235b5-6d20-493a-a820-f4b2cdfb19ab req-6c368e80-6c79-4c26-9dec-e9b80653e761 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045718 Aug 28 01:20:46.872213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1230/2236] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:20:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:20:46.877905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-701d9ba4-86dd-4de3-974e-aa84d26287e0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:46.879006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-701d9ba4-86dd-4de3-974e-aa84d26287e0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:46.885550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-701d9ba4-86dd-4de3-974e-aa84d26287e0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:46.886375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-701d9ba4-86dd-4de3-974e-aa84d26287e0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:46.887110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-701d9ba4-86dd-4de3-974e-aa84d26287e0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:46.887826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-701d9ba4-86dd-4de3-974e-aa84d26287e0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:46.888348 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-701d9ba4-86dd-4de3-974e-aa84d26287e0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:46.888838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-701d9ba4-86dd-4de3-974e-aa84d26287e0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:47.003378 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7f8adb19-3374-47da-9a86-0d89d5f62161 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:47.020485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7f8adb19-3374-47da-9a86-0d89d5f62161 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.254555 Aug 28 01:20:47.021772 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1007/2237] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:46 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:47.067610 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-701d9ba4-86dd-4de3-974e-aa84d26287e0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:47.076587 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-701d9ba4-86dd-4de3-974e-aa84d26287e0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:47.077985 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-701d9ba4-86dd-4de3-974e-aa84d26287e0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:47.079186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-701d9ba4-86dd-4de3-974e-aa84d26287e0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.205550 Aug 28 01:20:47.079756 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1231/2238] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:46 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:47.569835 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ae27c580-2b9c-4d5f-af14-a88bfdfac406 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:47.571217 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ae27c580-2b9c-4d5f-af14-a88bfdfac406 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:47.584139 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae27c580-2b9c-4d5f-af14-a88bfdfac406 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:47.584635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae27c580-2b9c-4d5f-af14-a88bfdfac406 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:47.585093 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae27c580-2b9c-4d5f-af14-a88bfdfac406 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:47.585816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae27c580-2b9c-4d5f-af14-a88bfdfac406 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:47.586267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae27c580-2b9c-4d5f-af14-a88bfdfac406 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:47.586696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae27c580-2b9c-4d5f-af14-a88bfdfac406 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:47.666108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ae27c580-2b9c-4d5f-af14-a88bfdfac406 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:47.675221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ae27c580-2b9c-4d5f-af14-a88bfdfac406 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.111325 Aug 28 01:20:47.675913 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1008/2239] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:47 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:47.745658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a46919af-64d0-4310-8c53-2d0121460422 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:47.746450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a46919af-64d0-4310-8c53-2d0121460422 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:47.753205 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a46919af-64d0-4310-8c53-2d0121460422 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:47.753805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a46919af-64d0-4310-8c53-2d0121460422 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:47.754298 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a46919af-64d0-4310-8c53-2d0121460422 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:47.755079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a46919af-64d0-4310-8c53-2d0121460422 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:47.755542 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a46919af-64d0-4310-8c53-2d0121460422 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:47.755991 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a46919af-64d0-4310-8c53-2d0121460422 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:47.823107 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a46919af-64d0-4310-8c53-2d0121460422 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:47.831930 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a46919af-64d0-4310-8c53-2d0121460422 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.090087 Aug 28 01:20:47.832523 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1232/2240] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:47 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:48.042631 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ce2a029f-b473-4762-8a8b-2428320f314c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:48.043195 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ce2a029f-b473-4762-8a8b-2428320f314c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:48.049575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce2a029f-b473-4762-8a8b-2428320f314c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:48.050061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce2a029f-b473-4762-8a8b-2428320f314c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:48.050483 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce2a029f-b473-4762-8a8b-2428320f314c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:48.051175 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce2a029f-b473-4762-8a8b-2428320f314c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:48.051616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce2a029f-b473-4762-8a8b-2428320f314c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:48.052028 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce2a029f-b473-4762-8a8b-2428320f314c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:48.102294 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c9dd5a61-edd9-4e0d-bd15-fc6b6b0fe24c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:48.102922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c9dd5a61-edd9-4e0d-bd15-fc6b6b0fe24c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:48.109447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c9dd5a61-edd9-4e0d-bd15-fc6b6b0fe24c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:48.109938 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c9dd5a61-edd9-4e0d-bd15-fc6b6b0fe24c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:48.110406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c9dd5a61-edd9-4e0d-bd15-fc6b6b0fe24c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:48.111229 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c9dd5a61-edd9-4e0d-bd15-fc6b6b0fe24c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:48.111685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c9dd5a61-edd9-4e0d-bd15-fc6b6b0fe24c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:48.112106 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c9dd5a61-edd9-4e0d-bd15-fc6b6b0fe24c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:48.256743 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ce2a029f-b473-4762-8a8b-2428320f314c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:48.265637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ce2a029f-b473-4762-8a8b-2428320f314c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.226947 Aug 28 01:20:48.266222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1009/2241] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:48 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:48.295560 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c9dd5a61-edd9-4e0d-bd15-fc6b6b0fe24c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:48.305495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c9dd5a61-edd9-4e0d-bd15-fc6b6b0fe24c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:48.306841 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c9dd5a61-edd9-4e0d-bd15-fc6b6b0fe24c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:48.308006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c9dd5a61-edd9-4e0d-bd15-fc6b6b0fe24c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.209670 Aug 28 01:20:48.308584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1233/2242] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:48 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 01:20:48.697626 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5644c821-9f6c-4b49-8ec7-91c503a1e346 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:48.698291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5644c821-9f6c-4b49-8ec7-91c503a1e346 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:48.705587 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5644c821-9f6c-4b49-8ec7-91c503a1e346 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:48.706084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5644c821-9f6c-4b49-8ec7-91c503a1e346 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:48.706516 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5644c821-9f6c-4b49-8ec7-91c503a1e346 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:48.707353 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5644c821-9f6c-4b49-8ec7-91c503a1e346 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:48.708050 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5644c821-9f6c-4b49-8ec7-91c503a1e346 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:48.708629 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5644c821-9f6c-4b49-8ec7-91c503a1e346 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:48.779253 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5644c821-9f6c-4b49-8ec7-91c503a1e346 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:48.781507 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:20:48.789375 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5644c821-9f6c-4b49-8ec7-91c503a1e346 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.095751 Aug 28 01:20:48.789934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1010/2243] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:48 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:48.853079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a976b6c2-6cc6-49b5-ae6d-003b58025979 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:48.853695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a976b6c2-6cc6-49b5-ae6d-003b58025979 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:48.860210 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a976b6c2-6cc6-49b5-ae6d-003b58025979 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:48.860742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a976b6c2-6cc6-49b5-ae6d-003b58025979 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:48.861264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a976b6c2-6cc6-49b5-ae6d-003b58025979 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:48.861936 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a976b6c2-6cc6-49b5-ae6d-003b58025979 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:48.862384 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a976b6c2-6cc6-49b5-ae6d-003b58025979 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:48.862793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a976b6c2-6cc6-49b5-ae6d-003b58025979 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:48.933812 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a976b6c2-6cc6-49b5-ae6d-003b58025979 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:48.942878 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a976b6c2-6cc6-49b5-ae6d-003b58025979 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.093758 Aug 28 01:20:48.943434 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1234/2244] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:48 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:49.287564 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c6336ab1-f748-45cd-a8ec-b1e1612ac6db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:49.288173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c6336ab1-f748-45cd-a8ec-b1e1612ac6db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:49.294785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6336ab1-f748-45cd-a8ec-b1e1612ac6db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:49.295287 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6336ab1-f748-45cd-a8ec-b1e1612ac6db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:49.295710 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6336ab1-f748-45cd-a8ec-b1e1612ac6db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:49.296392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6336ab1-f748-45cd-a8ec-b1e1612ac6db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:49.296833 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6336ab1-f748-45cd-a8ec-b1e1612ac6db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:49.297262 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c6336ab1-f748-45cd-a8ec-b1e1612ac6db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:49.339738 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c58e15f2-3b81-40b0-b112-c65812bee596 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:49.340392 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c58e15f2-3b81-40b0-b112-c65812bee596 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:49.346887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c58e15f2-3b81-40b0-b112-c65812bee596 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:49.347374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c58e15f2-3b81-40b0-b112-c65812bee596 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:49.347801 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c58e15f2-3b81-40b0-b112-c65812bee596 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:49.348491 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c58e15f2-3b81-40b0-b112-c65812bee596 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:49.348991 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c58e15f2-3b81-40b0-b112-c65812bee596 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:49.349410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c58e15f2-3b81-40b0-b112-c65812bee596 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:49.484044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c6336ab1-f748-45cd-a8ec-b1e1612ac6db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:49.493091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c6336ab1-f748-45cd-a8ec-b1e1612ac6db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.209428 Aug 28 01:20:49.494022 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1011/2245] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:49 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:49.511527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c58e15f2-3b81-40b0-b112-c65812bee596 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:49.520786 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c58e15f2-3b81-40b0-b112-c65812bee596 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:49.522191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c58e15f2-3b81-40b0-b112-c65812bee596 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:49.523345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c58e15f2-3b81-40b0-b112-c65812bee596 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.187700 Aug 28 01:20:49.523915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1235/2246] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:49 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:49.811376 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c46390d4-afc2-465c-a0f6-a1647705bff9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:49.811972 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c46390d4-afc2-465c-a0f6-a1647705bff9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:49.819207 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c46390d4-afc2-465c-a0f6-a1647705bff9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:49.819689 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c46390d4-afc2-465c-a0f6-a1647705bff9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:49.820115 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c46390d4-afc2-465c-a0f6-a1647705bff9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:49.820799 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c46390d4-afc2-465c-a0f6-a1647705bff9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:49.821276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c46390d4-afc2-465c-a0f6-a1647705bff9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:49.821694 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c46390d4-afc2-465c-a0f6-a1647705bff9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:49.888458 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c46390d4-afc2-465c-a0f6-a1647705bff9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:49.897346 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c46390d4-afc2-465c-a0f6-a1647705bff9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089928 Aug 28 01:20:49.897939 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1012/2247] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:49 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:49.964272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a21985be-ba65-4eba-8fe6-1c4000833ea6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:49.964904 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a21985be-ba65-4eba-8fe6-1c4000833ea6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:49.972262 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a21985be-ba65-4eba-8fe6-1c4000833ea6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:49.972752 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a21985be-ba65-4eba-8fe6-1c4000833ea6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:49.973232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a21985be-ba65-4eba-8fe6-1c4000833ea6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:49.974037 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a21985be-ba65-4eba-8fe6-1c4000833ea6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:49.974566 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a21985be-ba65-4eba-8fe6-1c4000833ea6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:49.974995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a21985be-ba65-4eba-8fe6-1c4000833ea6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:50.046405 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a21985be-ba65-4eba-8fe6-1c4000833ea6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:50.058695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a21985be-ba65-4eba-8fe6-1c4000833ea6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.098231 Aug 28 01:20:50.059473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1236/2248] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:49 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:50.517997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9efba684-62c3-443a-9721-11e8286b74bb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:50.518625 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9efba684-62c3-443a-9721-11e8286b74bb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:50.525687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9efba684-62c3-443a-9721-11e8286b74bb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:50.526174 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9efba684-62c3-443a-9721-11e8286b74bb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:50.526634 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9efba684-62c3-443a-9721-11e8286b74bb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:50.527342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9efba684-62c3-443a-9721-11e8286b74bb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:50.527780 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9efba684-62c3-443a-9721-11e8286b74bb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:50.528205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9efba684-62c3-443a-9721-11e8286b74bb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:50.547422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-22505162-e488-4ea6-a1e3-733507f082a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:50.548096 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-22505162-e488-4ea6-a1e3-733507f082a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:50.554822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-22505162-e488-4ea6-a1e3-733507f082a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:50.555320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-22505162-e488-4ea6-a1e3-733507f082a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:50.555741 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-22505162-e488-4ea6-a1e3-733507f082a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:50.556443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-22505162-e488-4ea6-a1e3-733507f082a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:50.556901 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-22505162-e488-4ea6-a1e3-733507f082a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:50.557358 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-22505162-e488-4ea6-a1e3-733507f082a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:50.726235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9efba684-62c3-443a-9721-11e8286b74bb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:50.736053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9efba684-62c3-443a-9721-11e8286b74bb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.222055 Aug 28 01:20:50.736669 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1013/2249] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:50 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:50.749967 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-22505162-e488-4ea6-a1e3-733507f082a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:50.759282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-22505162-e488-4ea6-a1e3-733507f082a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:50.760739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-22505162-e488-4ea6-a1e3-733507f082a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:50.761926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-22505162-e488-4ea6-a1e3-733507f082a5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.218599 Aug 28 01:20:50.762447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1237/2250] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:50 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:50.919062 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4ab2c581-0fb8-43ab-a2e9-3693f01ab194 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:50.919676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4ab2c581-0fb8-43ab-a2e9-3693f01ab194 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:50.926257 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ab2c581-0fb8-43ab-a2e9-3693f01ab194 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:50.926739 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ab2c581-0fb8-43ab-a2e9-3693f01ab194 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:50.927171 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ab2c581-0fb8-43ab-a2e9-3693f01ab194 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:50.927870 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ab2c581-0fb8-43ab-a2e9-3693f01ab194 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:50.928314 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ab2c581-0fb8-43ab-a2e9-3693f01ab194 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:50.928727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ab2c581-0fb8-43ab-a2e9-3693f01ab194 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:50.999980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4ab2c581-0fb8-43ab-a2e9-3693f01ab194 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:51.011336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4ab2c581-0fb8-43ab-a2e9-3693f01ab194 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.095993 Aug 28 01:20:51.012158 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1014/2251] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:50 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:51.080655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a5c6f756-9ca6-49c2-8dde-4c7d1f41f397 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:51.081324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a5c6f756-9ca6-49c2-8dde-4c7d1f41f397 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:51.088178 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5c6f756-9ca6-49c2-8dde-4c7d1f41f397 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:51.088663 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5c6f756-9ca6-49c2-8dde-4c7d1f41f397 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:51.089143 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5c6f756-9ca6-49c2-8dde-4c7d1f41f397 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:51.089843 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5c6f756-9ca6-49c2-8dde-4c7d1f41f397 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:51.090290 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5c6f756-9ca6-49c2-8dde-4c7d1f41f397 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:51.090708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5c6f756-9ca6-49c2-8dde-4c7d1f41f397 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:51.162008 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a5c6f756-9ca6-49c2-8dde-4c7d1f41f397 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:51.171230 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a5c6f756-9ca6-49c2-8dde-4c7d1f41f397 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.094567 Aug 28 01:20:51.171833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1238/2252] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:51 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:51.757986 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f65e5af6-502c-44a4-9b6c-1ee7a7174254 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:51.758590 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f65e5af6-502c-44a4-9b6c-1ee7a7174254 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:51.765167 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f65e5af6-502c-44a4-9b6c-1ee7a7174254 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:51.765691 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f65e5af6-502c-44a4-9b6c-1ee7a7174254 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:51.766118 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f65e5af6-502c-44a4-9b6c-1ee7a7174254 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:51.766876 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f65e5af6-502c-44a4-9b6c-1ee7a7174254 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:51.767307 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f65e5af6-502c-44a4-9b6c-1ee7a7174254 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:51.767722 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f65e5af6-502c-44a4-9b6c-1ee7a7174254 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:51.788421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-23614653-b241-4b9f-ac3f-46c5faf892b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:51.789748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-23614653-b241-4b9f-ac3f-46c5faf892b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:51.800771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23614653-b241-4b9f-ac3f-46c5faf892b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:51.801795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23614653-b241-4b9f-ac3f-46c5faf892b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:51.802724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23614653-b241-4b9f-ac3f-46c5faf892b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:51.803959 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23614653-b241-4b9f-ac3f-46c5faf892b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:51.805151 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23614653-b241-4b9f-ac3f-46c5faf892b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:51.805900 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23614653-b241-4b9f-ac3f-46c5faf892b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:51.962858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f65e5af6-502c-44a4-9b6c-1ee7a7174254 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:51.971591 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f65e5af6-502c-44a4-9b6c-1ee7a7174254 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.217526 Aug 28 01:20:51.972199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1015/2253] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:51 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:51.987790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-23614653-b241-4b9f-ac3f-46c5faf892b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:51.996679 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-23614653-b241-4b9f-ac3f-46c5faf892b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:51.998171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-23614653-b241-4b9f-ac3f-46c5faf892b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:51.999343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-23614653-b241-4b9f-ac3f-46c5faf892b4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.218537 Aug 28 01:20:52.000083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1239/2254] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:51 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:52.035123 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-73d3f28d-b5c5-42a4-a245-7326ed822b69 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:52.035652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-73d3f28d-b5c5-42a4-a245-7326ed822b69 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:52.041765 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73d3f28d-b5c5-42a4-a245-7326ed822b69 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:52.042266 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73d3f28d-b5c5-42a4-a245-7326ed822b69 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:52.042703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73d3f28d-b5c5-42a4-a245-7326ed822b69 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:52.043400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73d3f28d-b5c5-42a4-a245-7326ed822b69 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:52.043852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73d3f28d-b5c5-42a4-a245-7326ed822b69 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:52.044273 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73d3f28d-b5c5-42a4-a245-7326ed822b69 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:52.110116 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-73d3f28d-b5c5-42a4-a245-7326ed822b69 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:52.119173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-73d3f28d-b5c5-42a4-a245-7326ed822b69 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.087920 Aug 28 01:20:52.120045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1016/2255] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:52 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:52.193696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7bfed604-7d80-4f9d-b862-96dd13214710 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:52.194352 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7bfed604-7d80-4f9d-b862-96dd13214710 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:52.203244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7bfed604-7d80-4f9d-b862-96dd13214710 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:52.203730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7bfed604-7d80-4f9d-b862-96dd13214710 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:52.204164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7bfed604-7d80-4f9d-b862-96dd13214710 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:52.204860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7bfed604-7d80-4f9d-b862-96dd13214710 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:52.205359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7bfed604-7d80-4f9d-b862-96dd13214710 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:52.205781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7bfed604-7d80-4f9d-b862-96dd13214710 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:52.208616 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:20:52.280237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7bfed604-7d80-4f9d-b862-96dd13214710 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:52.290802 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7bfed604-7d80-4f9d-b862-96dd13214710 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.101389 Aug 28 01:20:52.291471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1240/2256] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:52 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:52.993678 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ae2b89e4-8c9f-4642-8b4f-f83365c76ff4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:52.994323 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ae2b89e4-8c9f-4642-8b4f-f83365c76ff4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:53.000989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae2b89e4-8c9f-4642-8b4f-f83365c76ff4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:53.001548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae2b89e4-8c9f-4642-8b4f-f83365c76ff4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:53.002000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae2b89e4-8c9f-4642-8b4f-f83365c76ff4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:53.002702 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae2b89e4-8c9f-4642-8b4f-f83365c76ff4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:53.003157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae2b89e4-8c9f-4642-8b4f-f83365c76ff4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:53.003584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae2b89e4-8c9f-4642-8b4f-f83365c76ff4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:53.022528 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-aa40e0ed-849e-46b6-80d7-7383b41d1583 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:53.023230 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-aa40e0ed-849e-46b6-80d7-7383b41d1583 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:53.030305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa40e0ed-849e-46b6-80d7-7383b41d1583 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:53.030818 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa40e0ed-849e-46b6-80d7-7383b41d1583 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:53.031242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa40e0ed-849e-46b6-80d7-7383b41d1583 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:53.031930 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa40e0ed-849e-46b6-80d7-7383b41d1583 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:53.032687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa40e0ed-849e-46b6-80d7-7383b41d1583 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:53.033296 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aa40e0ed-849e-46b6-80d7-7383b41d1583 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:53.203382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ae2b89e4-8c9f-4642-8b4f-f83365c76ff4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:53.212122 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ae2b89e4-8c9f-4642-8b4f-f83365c76ff4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.222433 Aug 28 01:20:53.212679 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1017/2257] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:52 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:53.215869 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-aa40e0ed-849e-46b6-80d7-7383b41d1583 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:53.224072 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-058d2e83-1feb-401f-a08b-8b96ed78d4d9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:53.224863 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-058d2e83-1feb-401f-a08b-8b96ed78d4d9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:53.229370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-aa40e0ed-849e-46b6-80d7-7383b41d1583 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:53.231281 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-aa40e0ed-849e-46b6-80d7-7383b41d1583 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:53.231454 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-058d2e83-1feb-401f-a08b-8b96ed78d4d9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:53.231853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-058d2e83-1feb-401f-a08b-8b96ed78d4d9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:53.232324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-058d2e83-1feb-401f-a08b-8b96ed78d4d9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:53.233064 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-058d2e83-1feb-401f-a08b-8b96ed78d4d9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:53.233219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-aa40e0ed-849e-46b6-80d7-7383b41d1583 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.214393 Aug 28 01:20:53.233495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-058d2e83-1feb-401f-a08b-8b96ed78d4d9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:53.233881 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1241/2258] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:53 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:53.234023 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-058d2e83-1feb-401f-a08b-8b96ed78d4d9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:53.301538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-058d2e83-1feb-401f-a08b-8b96ed78d4d9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:53.310437 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-058d2e83-1feb-401f-a08b-8b96ed78d4d9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.090208 Aug 28 01:20:53.311000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1018/2259] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:53 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:53.311544 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a105d57f-e98a-452c-bc66-cfceee7c5f93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:53.312161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a105d57f-e98a-452c-bc66-cfceee7c5f93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:53.319298 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a105d57f-e98a-452c-bc66-cfceee7c5f93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:53.319784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a105d57f-e98a-452c-bc66-cfceee7c5f93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:53.320219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a105d57f-e98a-452c-bc66-cfceee7c5f93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:53.320918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a105d57f-e98a-452c-bc66-cfceee7c5f93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:53.321403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a105d57f-e98a-452c-bc66-cfceee7c5f93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:53.321831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a105d57f-e98a-452c-bc66-cfceee7c5f93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:53.384430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a105d57f-e98a-452c-bc66-cfceee7c5f93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:53.392902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a105d57f-e98a-452c-bc66-cfceee7c5f93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.085184 Aug 28 01:20:53.393538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1242/2260] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:53 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:20:54.234765 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0976e5c0-84fe-43dc-a9d8-beca1722b268 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:54.235431 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0976e5c0-84fe-43dc-a9d8-beca1722b268 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:54.243290 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0976e5c0-84fe-43dc-a9d8-beca1722b268 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:54.243772 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0976e5c0-84fe-43dc-a9d8-beca1722b268 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:54.244205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0976e5c0-84fe-43dc-a9d8-beca1722b268 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:54.244910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0976e5c0-84fe-43dc-a9d8-beca1722b268 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:54.245390 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0976e5c0-84fe-43dc-a9d8-beca1722b268 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:54.245812 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0976e5c0-84fe-43dc-a9d8-beca1722b268 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:54.256504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2f09b1ce-eb5d-4192-b953-5eb0324ac94f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:54.257238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2f09b1ce-eb5d-4192-b953-5eb0324ac94f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:54.263402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f09b1ce-eb5d-4192-b953-5eb0324ac94f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:54.263888 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f09b1ce-eb5d-4192-b953-5eb0324ac94f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:54.264318 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f09b1ce-eb5d-4192-b953-5eb0324ac94f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:54.265075 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f09b1ce-eb5d-4192-b953-5eb0324ac94f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:54.266089 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f09b1ce-eb5d-4192-b953-5eb0324ac94f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:54.266089 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f09b1ce-eb5d-4192-b953-5eb0324ac94f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:54.445666 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0976e5c0-84fe-43dc-a9d8-beca1722b268 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:54.454870 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0976e5c0-84fe-43dc-a9d8-beca1722b268 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.224149 Aug 28 01:20:54.455498 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1019/2261] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:54 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:54.460592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3709ad37-d13a-4b45-b76b-52c44720addb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:54.461574 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3709ad37-d13a-4b45-b76b-52c44720addb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:54.464685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2f09b1ce-eb5d-4192-b953-5eb0324ac94f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:54.471081 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3709ad37-d13a-4b45-b76b-52c44720addb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:54.471576 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3709ad37-d13a-4b45-b76b-52c44720addb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:54.472013 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3709ad37-d13a-4b45-b76b-52c44720addb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:54.472729 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3709ad37-d13a-4b45-b76b-52c44720addb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:54.473213 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3709ad37-d13a-4b45-b76b-52c44720addb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:54.473639 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3709ad37-d13a-4b45-b76b-52c44720addb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:54.475602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2f09b1ce-eb5d-4192-b953-5eb0324ac94f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:54.477059 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2f09b1ce-eb5d-4192-b953-5eb0324ac94f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:54.478292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2f09b1ce-eb5d-4192-b953-5eb0324ac94f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.226307 Aug 28 01:20:54.478847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1243/2262] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:54 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:54.484155 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e20e4c68-d010-488b-9c53-f1380d824be5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:54.484757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e20e4c68-d010-488b-9c53-f1380d824be5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:54.491369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e20e4c68-d010-488b-9c53-f1380d824be5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:54.491860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e20e4c68-d010-488b-9c53-f1380d824be5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:54.492284 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e20e4c68-d010-488b-9c53-f1380d824be5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:54.492996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e20e4c68-d010-488b-9c53-f1380d824be5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:54.493942 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e20e4c68-d010-488b-9c53-f1380d824be5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:54.493942 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e20e4c68-d010-488b-9c53-f1380d824be5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:54.553006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3709ad37-d13a-4b45-b76b-52c44720addb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:54.562726 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3709ad37-d13a-4b45-b76b-52c44720addb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.106197 Aug 28 01:20:54.563299 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1020/2263] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:54 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:54.567278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e20e4c68-d010-488b-9c53-f1380d824be5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:54.575987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e20e4c68-d010-488b-9c53-f1380d824be5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.095789 Aug 28 01:20:54.576602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1244/2264] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:54 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:55.480528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-444503f8-1606-4bcc-8156-aeda3c0b571c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:55.481254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-444503f8-1606-4bcc-8156-aeda3c0b571c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:55.488172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-444503f8-1606-4bcc-8156-aeda3c0b571c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:55.488643 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-444503f8-1606-4bcc-8156-aeda3c0b571c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:55.489106 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-444503f8-1606-4bcc-8156-aeda3c0b571c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:55.489794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-444503f8-1606-4bcc-8156-aeda3c0b571c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:55.490238 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-444503f8-1606-4bcc-8156-aeda3c0b571c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:55.490652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-444503f8-1606-4bcc-8156-aeda3c0b571c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:55.502148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bc6e5699-38ef-4276-8253-5635b9350945 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:55.502815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bc6e5699-38ef-4276-8253-5635b9350945 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:55.509484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc6e5699-38ef-4276-8253-5635b9350945 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:55.509962 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc6e5699-38ef-4276-8253-5635b9350945 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:55.510403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc6e5699-38ef-4276-8253-5635b9350945 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:55.511103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc6e5699-38ef-4276-8253-5635b9350945 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:55.511553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc6e5699-38ef-4276-8253-5635b9350945 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:55.511977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc6e5699-38ef-4276-8253-5635b9350945 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:55.693032 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-444503f8-1606-4bcc-8156-aeda3c0b571c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:55.702196 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-444503f8-1606-4bcc-8156-aeda3c0b571c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.225521 Aug 28 01:20:55.703081 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1021/2265] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:55 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:55.708173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a42cf1ed-e1ce-4ca0-a22b-c41de0936ad4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:55.709090 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a42cf1ed-e1ce-4ca0-a22b-c41de0936ad4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:55.711316 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bc6e5699-38ef-4276-8253-5635b9350945 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:55.715880 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a42cf1ed-e1ce-4ca0-a22b-c41de0936ad4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:55.716560 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a42cf1ed-e1ce-4ca0-a22b-c41de0936ad4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:55.717134 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a42cf1ed-e1ce-4ca0-a22b-c41de0936ad4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:55.717859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a42cf1ed-e1ce-4ca0-a22b-c41de0936ad4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:55.718592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a42cf1ed-e1ce-4ca0-a22b-c41de0936ad4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:55.719000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a42cf1ed-e1ce-4ca0-a22b-c41de0936ad4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:55.721248 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bc6e5699-38ef-4276-8253-5635b9350945 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:55.722704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bc6e5699-38ef-4276-8253-5635b9350945 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:55.723900 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bc6e5699-38ef-4276-8253-5635b9350945 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.225752 Aug 28 01:20:55.724477 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1245/2266] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:55 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:55.729330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6fa43782-60a3-4926-9258-786f7185b81a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:55.729923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6fa43782-60a3-4926-9258-786f7185b81a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:55.739124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6fa43782-60a3-4926-9258-786f7185b81a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:55.739531 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6fa43782-60a3-4926-9258-786f7185b81a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:55.739957 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6fa43782-60a3-4926-9258-786f7185b81a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:55.740625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6fa43782-60a3-4926-9258-786f7185b81a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:55.741332 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6fa43782-60a3-4926-9258-786f7185b81a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:55.741749 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6fa43782-60a3-4926-9258-786f7185b81a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:55.796758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a42cf1ed-e1ce-4ca0-a22b-c41de0936ad4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:55.805701 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a42cf1ed-e1ce-4ca0-a22b-c41de0936ad4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.101630 Aug 28 01:20:55.806287 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1022/2267] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:55 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:55.815971 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6fa43782-60a3-4926-9258-786f7185b81a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:55.828017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6fa43782-60a3-4926-9258-786f7185b81a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.102319 Aug 28 01:20:55.828819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1246/2268] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:55 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:56.727743 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fd81ff8f-39cf-4e0d-8eea-e09ab4110123 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:56.728535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fd81ff8f-39cf-4e0d-8eea-e09ab4110123 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:56.741263 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd81ff8f-39cf-4e0d-8eea-e09ab4110123 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:56.741998 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd81ff8f-39cf-4e0d-8eea-e09ab4110123 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:56.742651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd81ff8f-39cf-4e0d-8eea-e09ab4110123 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:56.743883 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd81ff8f-39cf-4e0d-8eea-e09ab4110123 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:56.744611 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd81ff8f-39cf-4e0d-8eea-e09ab4110123 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:56.745379 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd81ff8f-39cf-4e0d-8eea-e09ab4110123 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:56.749830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-79f4115d-50a0-4bf0-9ae8-a02c407f0eac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:56.750412 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-79f4115d-50a0-4bf0-9ae8-a02c407f0eac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:56.758381 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79f4115d-50a0-4bf0-9ae8-a02c407f0eac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:56.758871 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79f4115d-50a0-4bf0-9ae8-a02c407f0eac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:56.759298 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79f4115d-50a0-4bf0-9ae8-a02c407f0eac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:56.760028 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79f4115d-50a0-4bf0-9ae8-a02c407f0eac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:56.760478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79f4115d-50a0-4bf0-9ae8-a02c407f0eac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:56.760904 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79f4115d-50a0-4bf0-9ae8-a02c407f0eac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:56.951294 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fd81ff8f-39cf-4e0d-8eea-e09ab4110123 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:56.959983 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fd81ff8f-39cf-4e0d-8eea-e09ab4110123 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.238641 Aug 28 01:20:56.960553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1023/2269] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:56 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:20:56.965181 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-175c337f-0482-4edf-a577-f92855b95b17 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:56.965793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-175c337f-0482-4edf-a577-f92855b95b17 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:56.966584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-79f4115d-50a0-4bf0-9ae8-a02c407f0eac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:56.972127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-175c337f-0482-4edf-a577-f92855b95b17 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:56.972619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-175c337f-0482-4edf-a577-f92855b95b17 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:56.973085 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-175c337f-0482-4edf-a577-f92855b95b17 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:56.973781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-175c337f-0482-4edf-a577-f92855b95b17 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:56.974231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-175c337f-0482-4edf-a577-f92855b95b17 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:56.974648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-175c337f-0482-4edf-a577-f92855b95b17 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:56.976383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-79f4115d-50a0-4bf0-9ae8-a02c407f0eac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:56.977973 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-79f4115d-50a0-4bf0-9ae8-a02c407f0eac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:56.979082 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-79f4115d-50a0-4bf0-9ae8-a02c407f0eac tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.233950 Aug 28 01:20:56.979681 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1247/2270] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:56 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:56.984292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f54c8527-1fa7-48f2-abe9-fb96f0ef596b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:56.984908 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f54c8527-1fa7-48f2-abe9-fb96f0ef596b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:56.994356 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f54c8527-1fa7-48f2-abe9-fb96f0ef596b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:56.995302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f54c8527-1fa7-48f2-abe9-fb96f0ef596b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:56.995302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f54c8527-1fa7-48f2-abe9-fb96f0ef596b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:56.995955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f54c8527-1fa7-48f2-abe9-fb96f0ef596b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:56.996594 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f54c8527-1fa7-48f2-abe9-fb96f0ef596b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:56.996813 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f54c8527-1fa7-48f2-abe9-fb96f0ef596b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:57.068649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-175c337f-0482-4edf-a577-f92855b95b17 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:57.078695 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-175c337f-0482-4edf-a577-f92855b95b17 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.117001 Aug 28 01:20:57.079398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1024/2271] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:56 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:57.092087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f54c8527-1fa7-48f2-abe9-fb96f0ef596b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:57.105650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f54c8527-1fa7-48f2-abe9-fb96f0ef596b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.124893 Aug 28 01:20:57.106371 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1248/2272] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:56 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:57.983546 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9b501e9e-db11-4174-a437-30dbe4eed7a5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:57.984210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9b501e9e-db11-4174-a437-30dbe4eed7a5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:57.991427 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b501e9e-db11-4174-a437-30dbe4eed7a5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:57.991907 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b501e9e-db11-4174-a437-30dbe4eed7a5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:57.992334 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b501e9e-db11-4174-a437-30dbe4eed7a5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:57.993262 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b501e9e-db11-4174-a437-30dbe4eed7a5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:57.993706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b501e9e-db11-4174-a437-30dbe4eed7a5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:57.994122 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b501e9e-db11-4174-a437-30dbe4eed7a5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:58.002819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e2827ee7-e5a0-4238-8826-9eded7ad077b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:58.003423 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e2827ee7-e5a0-4238-8826-9eded7ad077b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:58.010328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2827ee7-e5a0-4238-8826-9eded7ad077b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:58.010809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2827ee7-e5a0-4238-8826-9eded7ad077b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:58.011233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2827ee7-e5a0-4238-8826-9eded7ad077b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:58.011936 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2827ee7-e5a0-4238-8826-9eded7ad077b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:58.012379 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2827ee7-e5a0-4238-8826-9eded7ad077b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:58.012795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e2827ee7-e5a0-4238-8826-9eded7ad077b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:58.195833 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9b501e9e-db11-4174-a437-30dbe4eed7a5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:58.207662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9b501e9e-db11-4174-a437-30dbe4eed7a5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.227622 Aug 28 01:20:58.207892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e2827ee7-e5a0-4238-8826-9eded7ad077b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:58.208573 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1025/2273] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:57 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:58.217943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ae54db6e-65f1-49ff-a7fc-2b38a2f3d48c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:58.217943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ae54db6e-65f1-49ff-a7fc-2b38a2f3d48c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:58.218173 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e2827ee7-e5a0-4238-8826-9eded7ad077b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:58.218934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e2827ee7-e5a0-4238-8826-9eded7ad077b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:58.220219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e2827ee7-e5a0-4238-8826-9eded7ad077b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.221417 Aug 28 01:20:58.220781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1249/2274] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:57 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:58.233155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae54db6e-65f1-49ff-a7fc-2b38a2f3d48c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:58.233155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae54db6e-65f1-49ff-a7fc-2b38a2f3d48c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:58.233155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae54db6e-65f1-49ff-a7fc-2b38a2f3d48c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:58.233155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae54db6e-65f1-49ff-a7fc-2b38a2f3d48c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:58.233617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae54db6e-65f1-49ff-a7fc-2b38a2f3d48c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:58.233617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae54db6e-65f1-49ff-a7fc-2b38a2f3d48c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:58.233786 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-17a8db1d-4fff-4243-9650-a5efaf00910d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:58.233786 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-17a8db1d-4fff-4243-9650-a5efaf00910d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:58.239326 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-17a8db1d-4fff-4243-9650-a5efaf00910d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:58.239814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-17a8db1d-4fff-4243-9650-a5efaf00910d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:58.240243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-17a8db1d-4fff-4243-9650-a5efaf00910d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:58.240993 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-17a8db1d-4fff-4243-9650-a5efaf00910d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:58.241487 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-17a8db1d-4fff-4243-9650-a5efaf00910d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:58.241853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-17a8db1d-4fff-4243-9650-a5efaf00910d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:58.311822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-17a8db1d-4fff-4243-9650-a5efaf00910d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:58.322615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-17a8db1d-4fff-4243-9650-a5efaf00910d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.095418 Aug 28 01:20:58.323293 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1250/2275] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:58 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:58.340719 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ae54db6e-65f1-49ff-a7fc-2b38a2f3d48c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:58.349600 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ae54db6e-65f1-49ff-a7fc-2b38a2f3d48c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.139657 Aug 28 01:20:58.350194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1026/2276] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:58 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:59.230747 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-165abbbc-04b0-4ca0-942c-32fa8b2f77e5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:59.231388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-165abbbc-04b0-4ca0-942c-32fa8b2f77e5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:59.238925 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-165abbbc-04b0-4ca0-942c-32fa8b2f77e5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:59.239414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-165abbbc-04b0-4ca0-942c-32fa8b2f77e5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:59.239847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-165abbbc-04b0-4ca0-942c-32fa8b2f77e5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:59.240550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-165abbbc-04b0-4ca0-942c-32fa8b2f77e5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:59.241049 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-165abbbc-04b0-4ca0-942c-32fa8b2f77e5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:59.241475 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-165abbbc-04b0-4ca0-942c-32fa8b2f77e5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:59.243917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9fd0d8c8-de42-419f-aabf-f04e518e530f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:59.244537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9fd0d8c8-de42-419f-aabf-f04e518e530f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:59.251423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9fd0d8c8-de42-419f-aabf-f04e518e530f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:59.251894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9fd0d8c8-de42-419f-aabf-f04e518e530f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:59.252343 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9fd0d8c8-de42-419f-aabf-f04e518e530f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:59.253086 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9fd0d8c8-de42-419f-aabf-f04e518e530f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:59.253549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9fd0d8c8-de42-419f-aabf-f04e518e530f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:59.254065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9fd0d8c8-de42-419f-aabf-f04e518e530f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:59.271702 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:20:59.444579 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-165abbbc-04b0-4ca0-942c-32fa8b2f77e5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:59.450361 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9fd0d8c8-de42-419f-aabf-f04e518e530f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:59.454460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-165abbbc-04b0-4ca0-942c-32fa8b2f77e5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.227700 Aug 28 01:20:59.455059 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1251/2277] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:59 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:59.459732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8393001d-f28b-4ecd-b15b-3d2aedefb949 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:59.460066 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9fd0d8c8-de42-419f-aabf-f04e518e530f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:59.460330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8393001d-f28b-4ecd-b15b-3d2aedefb949 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:59.461551 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9fd0d8c8-de42-419f-aabf-f04e518e530f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:59.462732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9fd0d8c8-de42-419f-aabf-f04e518e530f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.222752 Aug 28 01:20:59.463522 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1027/2278] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:20:59 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:20:59.466573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8393001d-f28b-4ecd-b15b-3d2aedefb949 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:59.467054 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8393001d-f28b-4ecd-b15b-3d2aedefb949 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:59.467479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8393001d-f28b-4ecd-b15b-3d2aedefb949 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:59.468168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8393001d-f28b-4ecd-b15b-3d2aedefb949 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:59.468609 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8393001d-f28b-4ecd-b15b-3d2aedefb949 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:59.469080 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8393001d-f28b-4ecd-b15b-3d2aedefb949 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:59.479553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-42f78ccf-63f1-4d66-b598-5caea846706b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:20:59.480264 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-42f78ccf-63f1-4d66-b598-5caea846706b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:20:59.488800 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42f78ccf-63f1-4d66-b598-5caea846706b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:59.489540 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42f78ccf-63f1-4d66-b598-5caea846706b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:59.490141 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42f78ccf-63f1-4d66-b598-5caea846706b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:59.490934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42f78ccf-63f1-4d66-b598-5caea846706b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:20:59.491487 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42f78ccf-63f1-4d66-b598-5caea846706b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:20:59.491957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42f78ccf-63f1-4d66-b598-5caea846706b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:20:59.541138 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8393001d-f28b-4ecd-b15b-3d2aedefb949 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:59.550124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8393001d-f28b-4ecd-b15b-3d2aedefb949 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.094054 Aug 28 01:20:59.550728 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1252/2279] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:59 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:20:59.560741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-42f78ccf-63f1-4d66-b598-5caea846706b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:20:59.569877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-42f78ccf-63f1-4d66-b598-5caea846706b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.094206 Aug 28 01:20:59.570447 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1028/2280] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:20:59 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:00.477252 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2bf9e48c-ff10-47ef-b46e-a6dd95f75a75 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:00.477936 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2bf9e48c-ff10-47ef-b46e-a6dd95f75a75 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:00.485295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2bf9e48c-ff10-47ef-b46e-a6dd95f75a75 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:00.485793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2bf9e48c-ff10-47ef-b46e-a6dd95f75a75 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:00.486220 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2bf9e48c-ff10-47ef-b46e-a6dd95f75a75 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:00.486781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-315f7707-4b5a-4825-9776-3b0912e4278e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:00.487253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2bf9e48c-ff10-47ef-b46e-a6dd95f75a75 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:00.487354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2bf9e48c-ff10-47ef-b46e-a6dd95f75a75 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:00.487447 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-315f7707-4b5a-4825-9776-3b0912e4278e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:00.487790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2bf9e48c-ff10-47ef-b46e-a6dd95f75a75 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:00.494765 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-315f7707-4b5a-4825-9776-3b0912e4278e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:00.495286 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-315f7707-4b5a-4825-9776-3b0912e4278e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:00.495723 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-315f7707-4b5a-4825-9776-3b0912e4278e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:00.496441 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-315f7707-4b5a-4825-9776-3b0912e4278e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:00.496888 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-315f7707-4b5a-4825-9776-3b0912e4278e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:00.497358 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-315f7707-4b5a-4825-9776-3b0912e4278e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:00.691205 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2bf9e48c-ff10-47ef-b46e-a6dd95f75a75 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:00.694672 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-315f7707-4b5a-4825-9776-3b0912e4278e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:00.700231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2bf9e48c-ff10-47ef-b46e-a6dd95f75a75 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.226873 Aug 28 01:21:00.700793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1253/2281] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:00 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:00.703673 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-315f7707-4b5a-4825-9776-3b0912e4278e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:00.705155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-315f7707-4b5a-4825-9776-3b0912e4278e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:00.705526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-09e06ed3-c8e2-4e8f-a540-57a1432385b2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:00.706128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-09e06ed3-c8e2-4e8f-a540-57a1432385b2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:00.706347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-315f7707-4b5a-4825-9776-3b0912e4278e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.223523 Aug 28 01:21:00.707064 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1029/2282] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:00 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:00.712390 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09e06ed3-c8e2-4e8f-a540-57a1432385b2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:00.712878 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09e06ed3-c8e2-4e8f-a540-57a1432385b2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:00.713481 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09e06ed3-c8e2-4e8f-a540-57a1432385b2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:00.714079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09e06ed3-c8e2-4e8f-a540-57a1432385b2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:00.714536 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09e06ed3-c8e2-4e8f-a540-57a1432385b2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:00.714954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09e06ed3-c8e2-4e8f-a540-57a1432385b2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:00.719290 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-dbb2f092-9e9f-4768-b05e-a3bb621168f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:00.719901 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-dbb2f092-9e9f-4768-b05e-a3bb621168f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:00.730283 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dbb2f092-9e9f-4768-b05e-a3bb621168f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:00.730693 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dbb2f092-9e9f-4768-b05e-a3bb621168f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:00.731137 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dbb2f092-9e9f-4768-b05e-a3bb621168f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:00.731864 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dbb2f092-9e9f-4768-b05e-a3bb621168f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:00.732325 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dbb2f092-9e9f-4768-b05e-a3bb621168f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:00.732784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dbb2f092-9e9f-4768-b05e-a3bb621168f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:00.783218 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-09e06ed3-c8e2-4e8f-a540-57a1432385b2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:00.792099 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-09e06ed3-c8e2-4e8f-a540-57a1432385b2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.090190 Aug 28 01:21:00.792690 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1254/2283] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:00 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:00.812885 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-dbb2f092-9e9f-4768-b05e-a3bb621168f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:00.821791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-dbb2f092-9e9f-4768-b05e-a3bb621168f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.106243 Aug 28 01:21:00.822329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1030/2284] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:00 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:01.722287 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3e056b84-41a4-49ef-a436-18da16a2fd05 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:01.722906 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3e056b84-41a4-49ef-a436-18da16a2fd05 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:01.729865 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a4177bc0-af5f-44e7-a0bb-b047e982de2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:01.730450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e056b84-41a4-49ef-a436-18da16a2fd05 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:01.730601 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a4177bc0-af5f-44e7-a0bb-b047e982de2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:01.730693 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e056b84-41a4-49ef-a436-18da16a2fd05 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:01.731044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e056b84-41a4-49ef-a436-18da16a2fd05 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:01.731750 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e056b84-41a4-49ef-a436-18da16a2fd05 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:01.732198 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e056b84-41a4-49ef-a436-18da16a2fd05 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:01.732619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e056b84-41a4-49ef-a436-18da16a2fd05 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:01.732857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:21:01.738484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a4177bc0-af5f-44e7-a0bb-b047e982de2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:01.738965 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a4177bc0-af5f-44e7-a0bb-b047e982de2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:01.739390 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a4177bc0-af5f-44e7-a0bb-b047e982de2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:01.740097 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a4177bc0-af5f-44e7-a0bb-b047e982de2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:01.740537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a4177bc0-af5f-44e7-a0bb-b047e982de2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:01.740940 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a4177bc0-af5f-44e7-a0bb-b047e982de2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:01.935546 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3e056b84-41a4-49ef-a436-18da16a2fd05 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:01.938530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a4177bc0-af5f-44e7-a0bb-b047e982de2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:01.946488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3e056b84-41a4-49ef-a436-18da16a2fd05 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.228059 Aug 28 01:21:01.947077 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1255/2285] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:01 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:01.949487 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a4177bc0-af5f-44e7-a0bb-b047e982de2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:01.951037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a4177bc0-af5f-44e7-a0bb-b047e982de2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:01.951610 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b83c6162-6989-4445-a066-97e42059d4fd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:01.952213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b83c6162-6989-4445-a066-97e42059d4fd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:01.952355 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a4177bc0-af5f-44e7-a0bb-b047e982de2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.226492 Aug 28 01:21:01.953012 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1031/2286] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:01 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:01.960835 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b83c6162-6989-4445-a066-97e42059d4fd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:01.961378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b83c6162-6989-4445-a066-97e42059d4fd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:01.961813 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b83c6162-6989-4445-a066-97e42059d4fd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:01.962372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5ba2742d-b5e5-4d22-8907-763198e82280 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:01.962550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b83c6162-6989-4445-a066-97e42059d4fd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:01.963005 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b83c6162-6989-4445-a066-97e42059d4fd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:01.963256 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5ba2742d-b5e5-4d22-8907-763198e82280 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:01.963606 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b83c6162-6989-4445-a066-97e42059d4fd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:01.969818 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ba2742d-b5e5-4d22-8907-763198e82280 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:01.970371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ba2742d-b5e5-4d22-8907-763198e82280 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:01.970931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ba2742d-b5e5-4d22-8907-763198e82280 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:01.971713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ba2742d-b5e5-4d22-8907-763198e82280 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:01.972217 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ba2742d-b5e5-4d22-8907-763198e82280 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:01.972709 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ba2742d-b5e5-4d22-8907-763198e82280 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:02.055066 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5ba2742d-b5e5-4d22-8907-763198e82280 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:02.056555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b83c6162-6989-4445-a066-97e42059d4fd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:02.058901 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:21:02.064593 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5ba2742d-b5e5-4d22-8907-763198e82280 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.110076 Aug 28 01:21:02.065220 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1032/2287] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:01 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:02.067089 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b83c6162-6989-4445-a066-97e42059d4fd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.118610 Aug 28 01:21:02.067671 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1256/2288] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:01 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:02.970314 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-794f360b-c9a8-4fc2-aaea-95281c61e373 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:02.971127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-794f360b-c9a8-4fc2-aaea-95281c61e373 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:02.976743 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-49c832ee-2a46-4bf0-882f-b49c652199be tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:02.977377 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-49c832ee-2a46-4bf0-882f-b49c652199be tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:02.978169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-794f360b-c9a8-4fc2-aaea-95281c61e373 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:02.978662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-794f360b-c9a8-4fc2-aaea-95281c61e373 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:02.979085 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-794f360b-c9a8-4fc2-aaea-95281c61e373 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:02.979777 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-794f360b-c9a8-4fc2-aaea-95281c61e373 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:02.980221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-794f360b-c9a8-4fc2-aaea-95281c61e373 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:02.980636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-794f360b-c9a8-4fc2-aaea-95281c61e373 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:02.980951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=98 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:21:02.984664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49c832ee-2a46-4bf0-882f-b49c652199be tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:02.985182 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49c832ee-2a46-4bf0-882f-b49c652199be tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:02.985616 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49c832ee-2a46-4bf0-882f-b49c652199be tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:02.986363 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49c832ee-2a46-4bf0-882f-b49c652199be tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:02.986812 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49c832ee-2a46-4bf0-882f-b49c652199be tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:02.987233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-49c832ee-2a46-4bf0-882f-b49c652199be tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:03.425113 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-49c832ee-2a46-4bf0-882f-b49c652199be tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:03.434801 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-794f360b-c9a8-4fc2-aaea-95281c61e373 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:03.436716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-49c832ee-2a46-4bf0-882f-b49c652199be tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:03.439050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-49c832ee-2a46-4bf0-882f-b49c652199be tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:03.440339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-49c832ee-2a46-4bf0-882f-b49c652199be tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.467776 Aug 28 01:21:03.441079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1257/2289] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:02 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:03.445190 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-794f360b-c9a8-4fc2-aaea-95281c61e373 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.480109 Aug 28 01:21:03.445914 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2a5c46a0-e27b-493c-a5a9-79c4f6aa147e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:03.446304 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1033/2290] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:02 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:03.448118 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2a5c46a0-e27b-493c-a5a9-79c4f6aa147e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:03.451131 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fbded37b-d51a-494d-89ef-6091506ac270 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:03.452462 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fbded37b-d51a-494d-89ef-6091506ac270 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:03.455888 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a5c46a0-e27b-493c-a5a9-79c4f6aa147e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:03.456474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a5c46a0-e27b-493c-a5a9-79c4f6aa147e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:03.457018 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a5c46a0-e27b-493c-a5a9-79c4f6aa147e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:03.457995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a5c46a0-e27b-493c-a5a9-79c4f6aa147e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:03.458544 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a5c46a0-e27b-493c-a5a9-79c4f6aa147e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:03.459054 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2a5c46a0-e27b-493c-a5a9-79c4f6aa147e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:03.460692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fbded37b-d51a-494d-89ef-6091506ac270 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:03.461197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fbded37b-d51a-494d-89ef-6091506ac270 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:03.461631 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fbded37b-d51a-494d-89ef-6091506ac270 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:03.462328 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fbded37b-d51a-494d-89ef-6091506ac270 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:03.462778 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fbded37b-d51a-494d-89ef-6091506ac270 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:03.463194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fbded37b-d51a-494d-89ef-6091506ac270 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:03.529757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2a5c46a0-e27b-493c-a5a9-79c4f6aa147e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:03.538276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2a5c46a0-e27b-493c-a5a9-79c4f6aa147e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.095784 Aug 28 01:21:03.538843 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1258/2291] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:03 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:03.545299 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fbded37b-d51a-494d-89ef-6091506ac270 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:03.554559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fbded37b-d51a-494d-89ef-6091506ac270 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.106879 Aug 28 01:21:03.555253 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1034/2292] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:03 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:04.467863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7b779b11-cad4-4306-93a8-d351b56f8f4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:04.468462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7b779b11-cad4-4306-93a8-d351b56f8f4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:04.468613 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3cc59418-91d6-487b-bb2a-07e517cf8df6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:04.469393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3cc59418-91d6-487b-bb2a-07e517cf8df6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:04.475318 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b779b11-cad4-4306-93a8-d351b56f8f4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:04.475719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b779b11-cad4-4306-93a8-d351b56f8f4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:04.476156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b779b11-cad4-4306-93a8-d351b56f8f4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:04.476852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b779b11-cad4-4306-93a8-d351b56f8f4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:04.477003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cc59418-91d6-487b-bb2a-07e517cf8df6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:04.477327 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b779b11-cad4-4306-93a8-d351b56f8f4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:04.477453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cc59418-91d6-487b-bb2a-07e517cf8df6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:04.477729 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b779b11-cad4-4306-93a8-d351b56f8f4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:04.477853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cc59418-91d6-487b-bb2a-07e517cf8df6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:04.478507 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cc59418-91d6-487b-bb2a-07e517cf8df6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:04.478951 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cc59418-91d6-487b-bb2a-07e517cf8df6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:04.479371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cc59418-91d6-487b-bb2a-07e517cf8df6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:04.673920 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7b779b11-cad4-4306-93a8-d351b56f8f4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:04.677531 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3cc59418-91d6-487b-bb2a-07e517cf8df6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:04.683972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7b779b11-cad4-4306-93a8-d351b56f8f4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:04.685645 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7b779b11-cad4-4306-93a8-d351b56f8f4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:04.686628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3cc59418-91d6-487b-bb2a-07e517cf8df6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.223559 Aug 28 01:21:04.686790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7b779b11-cad4-4306-93a8-d351b56f8f4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.225875 Aug 28 01:21:04.687238 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1035/2293] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:04 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:04.690707 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1259/2294] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:04 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:04.692262 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-89ec4da0-7f16-469e-b897-e668f25fc47b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:04.693175 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-89ec4da0-7f16-469e-b897-e668f25fc47b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:04.698060 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-64e8822c-c878-4ac1-98db-2c3547e9b308 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:04.698940 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-64e8822c-c878-4ac1-98db-2c3547e9b308 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:04.700005 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-89ec4da0-7f16-469e-b897-e668f25fc47b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:04.700689 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-89ec4da0-7f16-469e-b897-e668f25fc47b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:04.701353 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-89ec4da0-7f16-469e-b897-e668f25fc47b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:04.702065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-89ec4da0-7f16-469e-b897-e668f25fc47b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:04.702535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-89ec4da0-7f16-469e-b897-e668f25fc47b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:04.703164 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-89ec4da0-7f16-469e-b897-e668f25fc47b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:04.706254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64e8822c-c878-4ac1-98db-2c3547e9b308 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:04.706767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64e8822c-c878-4ac1-98db-2c3547e9b308 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:04.707345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64e8822c-c878-4ac1-98db-2c3547e9b308 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:04.708021 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64e8822c-c878-4ac1-98db-2c3547e9b308 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:04.708464 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64e8822c-c878-4ac1-98db-2c3547e9b308 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:04.708883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-64e8822c-c878-4ac1-98db-2c3547e9b308 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:04.779670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-89ec4da0-7f16-469e-b897-e668f25fc47b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:04.788521 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-89ec4da0-7f16-469e-b897-e668f25fc47b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.100165 Aug 28 01:21:04.789152 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1036/2295] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:04 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:04.805361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-64e8822c-c878-4ac1-98db-2c3547e9b308 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:04.813680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-64e8822c-c878-4ac1-98db-2c3547e9b308 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.120049 Aug 28 01:21:04.814343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1260/2296] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:04 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:05.709092 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7ce45462-c104-494a-ace8-9ac56eb9eb23 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:05.709783 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7ce45462-c104-494a-ace8-9ac56eb9eb23 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:05.710313 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3ffbe3f6-a4fb-4aab-9d8d-35699f6098b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:05.710945 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3ffbe3f6-a4fb-4aab-9d8d-35699f6098b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:05.716991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ce45462-c104-494a-ace8-9ac56eb9eb23 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:05.717524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ce45462-c104-494a-ace8-9ac56eb9eb23 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:05.717696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ffbe3f6-a4fb-4aab-9d8d-35699f6098b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:05.717981 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ce45462-c104-494a-ace8-9ac56eb9eb23 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:05.718143 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ffbe3f6-a4fb-4aab-9d8d-35699f6098b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:05.718502 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ffbe3f6-a4fb-4aab-9d8d-35699f6098b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:05.718680 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ce45462-c104-494a-ace8-9ac56eb9eb23 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:05.719181 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ce45462-c104-494a-ace8-9ac56eb9eb23 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:05.719342 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ffbe3f6-a4fb-4aab-9d8d-35699f6098b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:05.719624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ce45462-c104-494a-ace8-9ac56eb9eb23 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:05.719786 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ffbe3f6-a4fb-4aab-9d8d-35699f6098b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:05.720093 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ffbe3f6-a4fb-4aab-9d8d-35699f6098b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:05.911205 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3ffbe3f6-a4fb-4aab-9d8d-35699f6098b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:05.917782 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7ce45462-c104-494a-ace8-9ac56eb9eb23 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:05.920125 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3ffbe3f6-a4fb-4aab-9d8d-35699f6098b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:05.921577 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3ffbe3f6-a4fb-4aab-9d8d-35699f6098b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:05.922756 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3ffbe3f6-a4fb-4aab-9d8d-35699f6098b1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.216456 Aug 28 01:21:05.923315 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1261/2297] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:05 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:05.926576 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7ce45462-c104-494a-ace8-9ac56eb9eb23 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.221665 Aug 28 01:21:05.927269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1037/2298] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:05 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:05.927685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3fae92d4-6220-4acf-a154-0bcbaf272671 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:05.928352 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3fae92d4-6220-4acf-a154-0bcbaf272671 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:05.934371 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3fae92d4-6220-4acf-a154-0bcbaf272671 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:05.934874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3fae92d4-6220-4acf-a154-0bcbaf272671 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:05.935686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3fae92d4-6220-4acf-a154-0bcbaf272671 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:05.936436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3fae92d4-6220-4acf-a154-0bcbaf272671 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:05.937003 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3fae92d4-6220-4acf-a154-0bcbaf272671 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:05.937295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3fae92d4-6220-4acf-a154-0bcbaf272671 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:05.943294 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e802f883-1979-49d2-8205-c169c822345c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:05.943876 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e802f883-1979-49d2-8205-c169c822345c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:05.950110 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e802f883-1979-49d2-8205-c169c822345c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:05.950591 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e802f883-1979-49d2-8205-c169c822345c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:05.951028 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e802f883-1979-49d2-8205-c169c822345c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:05.951748 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e802f883-1979-49d2-8205-c169c822345c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:05.952164 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e802f883-1979-49d2-8205-c169c822345c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:05.952607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e802f883-1979-49d2-8205-c169c822345c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:06.009627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3fae92d4-6220-4acf-a154-0bcbaf272671 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:06.018411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3fae92d4-6220-4acf-a154-0bcbaf272671 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.094065 Aug 28 01:21:06.018989 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1262/2299] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:05 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:06.029482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e802f883-1979-49d2-8205-c169c822345c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:06.040236 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e802f883-1979-49d2-8205-c169c822345c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.100857 Aug 28 01:21:06.041194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1038/2300] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:05 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:06.947259 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-23101221-a47c-4703-ba43-d0d93376c70c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:06.947986 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-979b5eb9-b3fb-4faf-b799-3120ae704623 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:06.948378 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-23101221-a47c-4703-ba43-d0d93376c70c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:06.948486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-979b5eb9-b3fb-4faf-b799-3120ae704623 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:06.954953 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23101221-a47c-4703-ba43-d0d93376c70c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:06.955162 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-979b5eb9-b3fb-4faf-b799-3120ae704623 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:06.955388 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23101221-a47c-4703-ba43-d0d93376c70c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:06.955545 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-979b5eb9-b3fb-4faf-b799-3120ae704623 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:06.955833 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23101221-a47c-4703-ba43-d0d93376c70c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:06.955998 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-979b5eb9-b3fb-4faf-b799-3120ae704623 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:06.956511 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23101221-a47c-4703-ba43-d0d93376c70c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:06.956701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-979b5eb9-b3fb-4faf-b799-3120ae704623 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:06.956955 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23101221-a47c-4703-ba43-d0d93376c70c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:06.957174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-979b5eb9-b3fb-4faf-b799-3120ae704623 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:06.957401 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23101221-a47c-4703-ba43-d0d93376c70c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:06.957525 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-979b5eb9-b3fb-4faf-b799-3120ae704623 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:07.156596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-979b5eb9-b3fb-4faf-b799-3120ae704623 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:07.161295 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-23101221-a47c-4703-ba43-d0d93376c70c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:07.166195 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-979b5eb9-b3fb-4faf-b799-3120ae704623 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:07.167610 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-979b5eb9-b3fb-4faf-b799-3120ae704623 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:07.168777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-979b5eb9-b3fb-4faf-b799-3120ae704623 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.225551 Aug 28 01:21:07.169362 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1263/2301] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:06 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:07.177308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-23101221-a47c-4703-ba43-d0d93376c70c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.226876 Aug 28 01:21:07.177308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1039/2302] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:06 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:07.177549 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e4a159fb-39ce-4d55-9e90-573ff9c2f409 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:07.177549 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e4a159fb-39ce-4d55-9e90-573ff9c2f409 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:07.179451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4f19bc13-9186-4df4-8fbe-e0dfb15a6c9f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:07.180820 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4f19bc13-9186-4df4-8fbe-e0dfb15a6c9f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:07.182538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e4a159fb-39ce-4d55-9e90-573ff9c2f409 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:07.183020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e4a159fb-39ce-4d55-9e90-573ff9c2f409 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:07.183446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e4a159fb-39ce-4d55-9e90-573ff9c2f409 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:07.184128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e4a159fb-39ce-4d55-9e90-573ff9c2f409 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:07.184636 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e4a159fb-39ce-4d55-9e90-573ff9c2f409 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:07.185287 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e4a159fb-39ce-4d55-9e90-573ff9c2f409 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:07.193173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4f19bc13-9186-4df4-8fbe-e0dfb15a6c9f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:07.193663 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4f19bc13-9186-4df4-8fbe-e0dfb15a6c9f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:07.194098 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4f19bc13-9186-4df4-8fbe-e0dfb15a6c9f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:07.194797 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4f19bc13-9186-4df4-8fbe-e0dfb15a6c9f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:07.195245 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4f19bc13-9186-4df4-8fbe-e0dfb15a6c9f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:07.195660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4f19bc13-9186-4df4-8fbe-e0dfb15a6c9f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:07.258600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e4a159fb-39ce-4d55-9e90-573ff9c2f409 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:07.267526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e4a159fb-39ce-4d55-9e90-573ff9c2f409 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.097111 Aug 28 01:21:07.268116 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1264/2303] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:07 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:07.277991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4f19bc13-9186-4df4-8fbe-e0dfb15a6c9f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:07.288355 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4f19bc13-9186-4df4-8fbe-e0dfb15a6c9f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.116000 Aug 28 01:21:07.288996 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1040/2304] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:07 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:08.192444 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-eaeec4a2-3918-4f31-b574-26ab88df8910 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:08.193115 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-70033785-b374-497a-a1fc-feed4cae51f8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:08.193485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-eaeec4a2-3918-4f31-b574-26ab88df8910 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:08.193596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-70033785-b374-497a-a1fc-feed4cae51f8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:08.199633 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70033785-b374-497a-a1fc-feed4cae51f8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:08.199834 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eaeec4a2-3918-4f31-b574-26ab88df8910 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:08.200116 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70033785-b374-497a-a1fc-feed4cae51f8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:08.200249 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eaeec4a2-3918-4f31-b574-26ab88df8910 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:08.200532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70033785-b374-497a-a1fc-feed4cae51f8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:08.200661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eaeec4a2-3918-4f31-b574-26ab88df8910 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:08.201280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70033785-b374-497a-a1fc-feed4cae51f8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:08.201405 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eaeec4a2-3918-4f31-b574-26ab88df8910 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:08.201747 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70033785-b374-497a-a1fc-feed4cae51f8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:08.201931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eaeec4a2-3918-4f31-b574-26ab88df8910 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:08.202176 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70033785-b374-497a-a1fc-feed4cae51f8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:08.202342 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eaeec4a2-3918-4f31-b574-26ab88df8910 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:08.395607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-70033785-b374-497a-a1fc-feed4cae51f8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:08.405688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-70033785-b374-497a-a1fc-feed4cae51f8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:08.406656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-eaeec4a2-3918-4f31-b574-26ab88df8910 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:08.407148 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-70033785-b374-497a-a1fc-feed4cae51f8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:08.408385 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-70033785-b374-497a-a1fc-feed4cae51f8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.220232 Aug 28 01:21:08.408988 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1041/2305] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:08 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:08.413616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d2f32840-378d-468b-9a51-c002245935b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:08.414269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d2f32840-378d-468b-9a51-c002245935b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:08.415879 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-eaeec4a2-3918-4f31-b574-26ab88df8910 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.227683 Aug 28 01:21:08.428528 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1265/2306] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:08 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:08.429599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2f32840-378d-468b-9a51-c002245935b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:08.431067 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e6bd2a61-66b6-45d5-9d14-757f8b68887d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:08.431375 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2f32840-378d-468b-9a51-c002245935b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:08.431521 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e6bd2a61-66b6-45d5-9d14-757f8b68887d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:08.431635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2f32840-378d-468b-9a51-c002245935b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:08.431635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2f32840-378d-468b-9a51-c002245935b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:08.431635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2f32840-378d-468b-9a51-c002245935b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:08.431635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2f32840-378d-468b-9a51-c002245935b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:08.441753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6bd2a61-66b6-45d5-9d14-757f8b68887d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:08.442267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6bd2a61-66b6-45d5-9d14-757f8b68887d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:08.442704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6bd2a61-66b6-45d5-9d14-757f8b68887d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:08.443416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6bd2a61-66b6-45d5-9d14-757f8b68887d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:08.443869 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6bd2a61-66b6-45d5-9d14-757f8b68887d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:08.444359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6bd2a61-66b6-45d5-9d14-757f8b68887d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:08.506925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d2f32840-378d-468b-9a51-c002245935b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:08.515808 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d2f32840-378d-468b-9a51-c002245935b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.105709 Aug 28 01:21:08.516377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1042/2307] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:08 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:08.527283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e6bd2a61-66b6-45d5-9d14-757f8b68887d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:08.536071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e6bd2a61-66b6-45d5-9d14-757f8b68887d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.117980 Aug 28 01:21:08.536662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1266/2308] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:08 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:09.437838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5ca835bf-4afb-43c5-8f85-c1c5057850f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:09.438596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5ca835bf-4afb-43c5-8f85-c1c5057850f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:09.438833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3b173a53-73d9-49fc-8342-47a36e482a7c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:09.439417 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3b173a53-73d9-49fc-8342-47a36e482a7c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:09.445749 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ca835bf-4afb-43c5-8f85-c1c5057850f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:09.446244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ca835bf-4afb-43c5-8f85-c1c5057850f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:09.446427 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3b173a53-73d9-49fc-8342-47a36e482a7c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:09.446674 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ca835bf-4afb-43c5-8f85-c1c5057850f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:09.446862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3b173a53-73d9-49fc-8342-47a36e482a7c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:09.447136 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3b173a53-73d9-49fc-8342-47a36e482a7c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:09.447353 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ca835bf-4afb-43c5-8f85-c1c5057850f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:09.447830 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ca835bf-4afb-43c5-8f85-c1c5057850f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:09.448026 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3b173a53-73d9-49fc-8342-47a36e482a7c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:09.448235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ca835bf-4afb-43c5-8f85-c1c5057850f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:09.448400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3b173a53-73d9-49fc-8342-47a36e482a7c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:09.448698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3b173a53-73d9-49fc-8342-47a36e482a7c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:09.654242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3b173a53-73d9-49fc-8342-47a36e482a7c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:09.657546 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5ca835bf-4afb-43c5-8f85-c1c5057850f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:09.662872 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3b173a53-73d9-49fc-8342-47a36e482a7c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.228508 Aug 28 01:21:09.663718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1267/2309] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:09 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:21:09.667064 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5ca835bf-4afb-43c5-8f85-c1c5057850f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:09.668017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-019c630b-d61b-4914-b380-627db4843732 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:09.668533 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5ca835bf-4afb-43c5-8f85-c1c5057850f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:09.668666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-019c630b-d61b-4914-b380-627db4843732 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:09.669747 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5ca835bf-4afb-43c5-8f85-c1c5057850f3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.236058 Aug 28 01:21:09.670283 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1043/2310] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:09 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:09.674863 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-02163549-752f-4db9-b7a3-5418d4b226e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:09.675162 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-019c630b-d61b-4914-b380-627db4843732 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:09.675656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-019c630b-d61b-4914-b380-627db4843732 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:09.676031 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-02163549-752f-4db9-b7a3-5418d4b226e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:09.676147 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-019c630b-d61b-4914-b380-627db4843732 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:09.676949 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-019c630b-d61b-4914-b380-627db4843732 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:09.677427 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-019c630b-d61b-4914-b380-627db4843732 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:09.677895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-019c630b-d61b-4914-b380-627db4843732 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:09.683653 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-02163549-752f-4db9-b7a3-5418d4b226e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:09.684420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-02163549-752f-4db9-b7a3-5418d4b226e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:09.684857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-02163549-752f-4db9-b7a3-5418d4b226e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:09.685598 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-02163549-752f-4db9-b7a3-5418d4b226e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:09.686123 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-02163549-752f-4db9-b7a3-5418d4b226e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:09.686641 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-02163549-752f-4db9-b7a3-5418d4b226e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:09.690342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:21:09.753308 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-019c630b-d61b-4914-b380-627db4843732 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:09.761930 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-019c630b-d61b-4914-b380-627db4843732 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.097425 Aug 28 01:21:09.762522 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1268/2311] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:09 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:09.773901 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-02163549-752f-4db9-b7a3-5418d4b226e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:09.782280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-02163549-752f-4db9-b7a3-5418d4b226e6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.110806 Aug 28 01:21:09.782857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1044/2312] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:09 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:10.684367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-82711e66-cfe8-49d5-badf-347f96371007 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:10.685037 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-82711e66-cfe8-49d5-badf-347f96371007 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:10.692572 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-82711e66-cfe8-49d5-badf-347f96371007 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:10.693083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-82711e66-cfe8-49d5-badf-347f96371007 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:10.693510 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-82711e66-cfe8-49d5-badf-347f96371007 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:10.694199 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-82711e66-cfe8-49d5-badf-347f96371007 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:10.694879 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-82711e66-cfe8-49d5-badf-347f96371007 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:10.695102 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d7d0ddc7-ad3f-44db-9dcb-53dae1c463c7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:10.695566 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-82711e66-cfe8-49d5-badf-347f96371007 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:10.695684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d7d0ddc7-ad3f-44db-9dcb-53dae1c463c7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:10.703060 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d7d0ddc7-ad3f-44db-9dcb-53dae1c463c7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:10.703605 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d7d0ddc7-ad3f-44db-9dcb-53dae1c463c7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:10.704100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d7d0ddc7-ad3f-44db-9dcb-53dae1c463c7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:10.704798 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d7d0ddc7-ad3f-44db-9dcb-53dae1c463c7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:10.705290 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d7d0ddc7-ad3f-44db-9dcb-53dae1c463c7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:10.705704 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d7d0ddc7-ad3f-44db-9dcb-53dae1c463c7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:10.892609 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-82711e66-cfe8-49d5-badf-347f96371007 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:10.896922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d7d0ddc7-ad3f-44db-9dcb-53dae1c463c7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:10.901115 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-82711e66-cfe8-49d5-badf-347f96371007 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.220585 Aug 28 01:21:10.901722 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1269/2313] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:10 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:10.906242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-87236a94-1459-4c82-9d30-512acc91a0ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:10.906433 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d7d0ddc7-ad3f-44db-9dcb-53dae1c463c7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:10.906835 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-87236a94-1459-4c82-9d30-512acc91a0ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:10.907825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d7d0ddc7-ad3f-44db-9dcb-53dae1c463c7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:10.908987 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d7d0ddc7-ad3f-44db-9dcb-53dae1c463c7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.218530 Aug 28 01:21:10.920442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1045/2314] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:10 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:10.920655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87236a94-1459-4c82-9d30-512acc91a0ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:10.920655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87236a94-1459-4c82-9d30-512acc91a0ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:10.920655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87236a94-1459-4c82-9d30-512acc91a0ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:10.920655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87236a94-1459-4c82-9d30-512acc91a0ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:10.921078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-84731a6f-b939-4231-b7e8-0dc03b86aa73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:10.921186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87236a94-1459-4c82-9d30-512acc91a0ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:10.921283 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-84731a6f-b939-4231-b7e8-0dc03b86aa73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:10.921369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87236a94-1459-4c82-9d30-512acc91a0ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:10.921453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-84731a6f-b939-4231-b7e8-0dc03b86aa73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:10.921548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-84731a6f-b939-4231-b7e8-0dc03b86aa73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:10.921981 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-84731a6f-b939-4231-b7e8-0dc03b86aa73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:10.922676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-84731a6f-b939-4231-b7e8-0dc03b86aa73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:10.923123 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-84731a6f-b939-4231-b7e8-0dc03b86aa73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:10.923544 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-84731a6f-b939-4231-b7e8-0dc03b86aa73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:10.984002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-87236a94-1459-4c82-9d30-512acc91a0ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:10.992361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-87236a94-1459-4c82-9d30-512acc91a0ad tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089660 Aug 28 01:21:10.992978 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1270/2315] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:10 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:11.000432 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-84731a6f-b939-4231-b7e8-0dc03b86aa73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:11.008822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-84731a6f-b939-4231-b7e8-0dc03b86aa73 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.097811 Aug 28 01:21:11.009409 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1046/2316] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:10 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:11.923242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9142da76-e0ae-4c49-ad53-b560680e56b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:11.923870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9142da76-e0ae-4c49-ad53-b560680e56b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:11.930956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9142da76-e0ae-4c49-ad53-b560680e56b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:11.931438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9142da76-e0ae-4c49-ad53-b560680e56b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:11.931721 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-98677797-891b-4155-8336-901ceb2bfcef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:11.932219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9142da76-e0ae-4c49-ad53-b560680e56b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:11.932329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-98677797-891b-4155-8336-901ceb2bfcef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:11.932503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9142da76-e0ae-4c49-ad53-b560680e56b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:11.933008 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9142da76-e0ae-4c49-ad53-b560680e56b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:11.933422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9142da76-e0ae-4c49-ad53-b560680e56b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:11.939136 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98677797-891b-4155-8336-901ceb2bfcef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:11.939620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98677797-891b-4155-8336-901ceb2bfcef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:11.940053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98677797-891b-4155-8336-901ceb2bfcef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:11.940763 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98677797-891b-4155-8336-901ceb2bfcef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:11.941279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98677797-891b-4155-8336-901ceb2bfcef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:11.942052 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-98677797-891b-4155-8336-901ceb2bfcef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:12.137043 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9142da76-e0ae-4c49-ad53-b560680e56b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:12.140094 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-98677797-891b-4155-8336-901ceb2bfcef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:12.145712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9142da76-e0ae-4c49-ad53-b560680e56b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.226824 Aug 28 01:21:12.146269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1271/2317] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:11 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:12.149178 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-98677797-891b-4155-8336-901ceb2bfcef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:12.150614 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-98677797-891b-4155-8336-901ceb2bfcef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:12.150755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8d6b3255-39b4-4887-8e07-d1876b321a5c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:12.151346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8d6b3255-39b4-4887-8e07-d1876b321a5c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:12.151785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-98677797-891b-4155-8336-901ceb2bfcef tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.223961 Aug 28 01:21:12.159980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1047/2318] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:11 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:12.159980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ba5d0c98-9795-4b3d-87e2-b117a0133fad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:12.160252 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d6b3255-39b4-4887-8e07-d1876b321a5c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:12.160374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ba5d0c98-9795-4b3d-87e2-b117a0133fad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:12.160466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d6b3255-39b4-4887-8e07-d1876b321a5c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:12.160466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d6b3255-39b4-4887-8e07-d1876b321a5c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:12.160466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d6b3255-39b4-4887-8e07-d1876b321a5c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:12.160466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d6b3255-39b4-4887-8e07-d1876b321a5c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:12.160768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d6b3255-39b4-4887-8e07-d1876b321a5c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:12.165488 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba5d0c98-9795-4b3d-87e2-b117a0133fad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:12.166345 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba5d0c98-9795-4b3d-87e2-b117a0133fad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:12.166787 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba5d0c98-9795-4b3d-87e2-b117a0133fad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:12.167497 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba5d0c98-9795-4b3d-87e2-b117a0133fad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:12.167947 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba5d0c98-9795-4b3d-87e2-b117a0133fad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:12.168367 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba5d0c98-9795-4b3d-87e2-b117a0133fad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:12.248188 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8d6b3255-39b4-4887-8e07-d1876b321a5c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:12.250461 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ba5d0c98-9795-4b3d-87e2-b117a0133fad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:12.258205 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8d6b3255-39b4-4887-8e07-d1876b321a5c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.110908 Aug 28 01:21:12.258988 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ba5d0c98-9795-4b3d-87e2-b117a0133fad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.105653 Aug 28 01:21:12.259717 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1048/2319] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:12 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:12.270304 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1272/2320] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:12 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:13.166744 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0da09948-698f-4e02-b550-a3280206cc5e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:13.167403 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0da09948-698f-4e02-b550-a3280206cc5e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:13.174534 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0da09948-698f-4e02-b550-a3280206cc5e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:13.175022 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0da09948-698f-4e02-b550-a3280206cc5e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:13.175458 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0da09948-698f-4e02-b550-a3280206cc5e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:13.175873 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3ca921b2-fc2b-45be-890b-30ba3e75551d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:13.176397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0da09948-698f-4e02-b550-a3280206cc5e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:13.176509 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3ca921b2-fc2b-45be-890b-30ba3e75551d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:13.176618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0da09948-698f-4e02-b550-a3280206cc5e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:13.176989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0da09948-698f-4e02-b550-a3280206cc5e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:13.183976 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ca921b2-fc2b-45be-890b-30ba3e75551d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:13.184462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ca921b2-fc2b-45be-890b-30ba3e75551d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:13.184899 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ca921b2-fc2b-45be-890b-30ba3e75551d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:13.185674 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ca921b2-fc2b-45be-890b-30ba3e75551d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:13.186124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ca921b2-fc2b-45be-890b-30ba3e75551d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:13.186545 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ca921b2-fc2b-45be-890b-30ba3e75551d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:13.202540 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:21:13.371003 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3ca921b2-fc2b-45be-890b-30ba3e75551d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:13.374004 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0da09948-698f-4e02-b550-a3280206cc5e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:13.379968 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3ca921b2-fc2b-45be-890b-30ba3e75551d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:13.381505 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3ca921b2-fc2b-45be-890b-30ba3e75551d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:13.382654 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3ca921b2-fc2b-45be-890b-30ba3e75551d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.210911 Aug 28 01:21:13.382819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0da09948-698f-4e02-b550-a3280206cc5e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.219999 Aug 28 01:21:13.386398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1049/2322] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:13 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:13.386590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1273/2322] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:13 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:13.387951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-11f801fa-404d-4ad2-9118-8bb330e63c01 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:13.388561 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-11f801fa-404d-4ad2-9118-8bb330e63c01 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:13.391049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2cd0b363-a34c-4c2a-b987-c1f3069a17bd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:13.391881 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2cd0b363-a34c-4c2a-b987-c1f3069a17bd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:13.394482 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-11f801fa-404d-4ad2-9118-8bb330e63c01 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:13.395188 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-11f801fa-404d-4ad2-9118-8bb330e63c01 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:13.395391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-11f801fa-404d-4ad2-9118-8bb330e63c01 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:13.396074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-11f801fa-404d-4ad2-9118-8bb330e63c01 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:13.396520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-11f801fa-404d-4ad2-9118-8bb330e63c01 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:13.396998 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-11f801fa-404d-4ad2-9118-8bb330e63c01 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:13.399291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2cd0b363-a34c-4c2a-b987-c1f3069a17bd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:13.399775 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2cd0b363-a34c-4c2a-b987-c1f3069a17bd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:13.400202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2cd0b363-a34c-4c2a-b987-c1f3069a17bd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:13.400914 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2cd0b363-a34c-4c2a-b987-c1f3069a17bd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:13.401403 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2cd0b363-a34c-4c2a-b987-c1f3069a17bd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:13.401816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2cd0b363-a34c-4c2a-b987-c1f3069a17bd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:13.465793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-11f801fa-404d-4ad2-9118-8bb330e63c01 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:13.473998 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-11f801fa-404d-4ad2-9118-8bb330e63c01 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089511 Aug 28 01:21:13.474568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1274/2323] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:13 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:13.486604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2cd0b363-a34c-4c2a-b987-c1f3069a17bd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:13.495425 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2cd0b363-a34c-4c2a-b987-c1f3069a17bd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.111100 Aug 28 01:21:13.496004 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1050/2324] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:13 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:14.404899 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-35e6627a-4961-45e5-bb93-178d7f1a943e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:14.405543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-35e6627a-4961-45e5-bb93-178d7f1a943e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:14.406789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ab1a2fcf-cfa4-4585-a062-efd46efe1c80 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:14.407415 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ab1a2fcf-cfa4-4585-a062-efd46efe1c80 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:14.413091 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-35e6627a-4961-45e5-bb93-178d7f1a943e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:14.413617 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-35e6627a-4961-45e5-bb93-178d7f1a943e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:14.414086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-35e6627a-4961-45e5-bb93-178d7f1a943e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:14.414575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab1a2fcf-cfa4-4585-a062-efd46efe1c80 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:14.414844 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-35e6627a-4961-45e5-bb93-178d7f1a943e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:14.415045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab1a2fcf-cfa4-4585-a062-efd46efe1c80 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:14.415250 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-35e6627a-4961-45e5-bb93-178d7f1a943e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:14.415487 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab1a2fcf-cfa4-4585-a062-efd46efe1c80 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:14.416028 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-35e6627a-4961-45e5-bb93-178d7f1a943e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:14.416227 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab1a2fcf-cfa4-4585-a062-efd46efe1c80 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:14.416671 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab1a2fcf-cfa4-4585-a062-efd46efe1c80 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:14.417117 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ab1a2fcf-cfa4-4585-a062-efd46efe1c80 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:14.621004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-35e6627a-4961-45e5-bb93-178d7f1a943e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:14.624039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ab1a2fcf-cfa4-4585-a062-efd46efe1c80 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:14.629781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-35e6627a-4961-45e5-bb93-178d7f1a943e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.228981 Aug 28 01:21:14.630331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1275/2325] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:14 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:14.633244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ab1a2fcf-cfa4-4585-a062-efd46efe1c80 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:14.634712 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ab1a2fcf-cfa4-4585-a062-efd46efe1c80 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:14.635336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4748aa5b-6bd5-4564-a6e3-471c29b12db1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:14.635882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ab1a2fcf-cfa4-4585-a062-efd46efe1c80 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.233296 Aug 28 01:21:14.636009 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4748aa5b-6bd5-4564-a6e3-471c29b12db1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:14.643078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1051/2326] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:14 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:14.643078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e3f73607-12ff-43be-a883-7663aa9b6bd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:14.643078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e3f73607-12ff-43be-a883-7663aa9b6bd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:14.643721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4748aa5b-6bd5-4564-a6e3-471c29b12db1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:14.643721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4748aa5b-6bd5-4564-a6e3-471c29b12db1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:14.643721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4748aa5b-6bd5-4564-a6e3-471c29b12db1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:14.643721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4748aa5b-6bd5-4564-a6e3-471c29b12db1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:14.644187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4748aa5b-6bd5-4564-a6e3-471c29b12db1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:14.644367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4748aa5b-6bd5-4564-a6e3-471c29b12db1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:14.649187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3f73607-12ff-43be-a883-7663aa9b6bd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:14.652270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3f73607-12ff-43be-a883-7663aa9b6bd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:14.652270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3f73607-12ff-43be-a883-7663aa9b6bd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:14.652270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3f73607-12ff-43be-a883-7663aa9b6bd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:14.652270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3f73607-12ff-43be-a883-7663aa9b6bd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:14.652726 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3f73607-12ff-43be-a883-7663aa9b6bd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:14.737828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4748aa5b-6bd5-4564-a6e3-471c29b12db1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:14.739931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e3f73607-12ff-43be-a883-7663aa9b6bd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:14.749866 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4748aa5b-6bd5-4564-a6e3-471c29b12db1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.118158 Aug 28 01:21:14.750636 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1276/2327] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:14 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:14.751676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e3f73607-12ff-43be-a883-7663aa9b6bd3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.114162 Aug 28 01:21:14.752439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1052/2328] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:14 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:15.651640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3a996f59-bb85-47c5-ba40-cf35afb34783 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:15.652282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3a996f59-bb85-47c5-ba40-cf35afb34783 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:15.659092 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a996f59-bb85-47c5-ba40-cf35afb34783 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:15.659303 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6df81416-98ae-4e65-ae16-516086bb1675 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:15.659826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a996f59-bb85-47c5-ba40-cf35afb34783 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:15.659945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6df81416-98ae-4e65-ae16-516086bb1675 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:15.660053 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a996f59-bb85-47c5-ba40-cf35afb34783 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:15.660724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a996f59-bb85-47c5-ba40-cf35afb34783 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:15.661209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a996f59-bb85-47c5-ba40-cf35afb34783 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:15.661737 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3a996f59-bb85-47c5-ba40-cf35afb34783 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:15.666926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6df81416-98ae-4e65-ae16-516086bb1675 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:15.667404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6df81416-98ae-4e65-ae16-516086bb1675 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:15.667824 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6df81416-98ae-4e65-ae16-516086bb1675 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:15.668528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6df81416-98ae-4e65-ae16-516086bb1675 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:15.668997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6df81416-98ae-4e65-ae16-516086bb1675 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:15.669418 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6df81416-98ae-4e65-ae16-516086bb1675 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:15.861924 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3a996f59-bb85-47c5-ba40-cf35afb34783 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:15.865816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6df81416-98ae-4e65-ae16-516086bb1675 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:15.870677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3a996f59-bb85-47c5-ba40-cf35afb34783 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.223018 Aug 28 01:21:15.871326 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1277/2329] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:15 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:15.874677 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6df81416-98ae-4e65-ae16-516086bb1675 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:15.876034 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1927b4bf-8215-4969-9608-03d9c95f9fd3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:15.876185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6df81416-98ae-4e65-ae16-516086bb1675 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:15.877007 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1927b4bf-8215-4969-9608-03d9c95f9fd3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:15.877281 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6df81416-98ae-4e65-ae16-516086bb1675 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.221920 Aug 28 01:21:15.877835 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1053/2330] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:15 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:15.882365 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-aebb54b5-611d-44bc-895d-c537bce790ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:15.882973 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-aebb54b5-611d-44bc-895d-c537bce790ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:15.884207 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1927b4bf-8215-4969-9608-03d9c95f9fd3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:15.884924 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1927b4bf-8215-4969-9608-03d9c95f9fd3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:15.885586 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1927b4bf-8215-4969-9608-03d9c95f9fd3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:15.886475 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1927b4bf-8215-4969-9608-03d9c95f9fd3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:15.887092 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1927b4bf-8215-4969-9608-03d9c95f9fd3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:15.887508 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1927b4bf-8215-4969-9608-03d9c95f9fd3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:15.889067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aebb54b5-611d-44bc-895d-c537bce790ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:15.889636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aebb54b5-611d-44bc-895d-c537bce790ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:15.890138 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aebb54b5-611d-44bc-895d-c537bce790ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:15.890888 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aebb54b5-611d-44bc-895d-c537bce790ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:15.891405 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aebb54b5-611d-44bc-895d-c537bce790ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:15.891905 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aebb54b5-611d-44bc-895d-c537bce790ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:15.957747 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1927b4bf-8215-4969-9608-03d9c95f9fd3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:15.966138 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1927b4bf-8215-4969-9608-03d9c95f9fd3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.093599 Aug 28 01:21:15.966960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1278/2331] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:15 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:15.973473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-aebb54b5-611d-44bc-895d-c537bce790ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:15.982099 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-aebb54b5-611d-44bc-895d-c537bce790ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.103295 Aug 28 01:21:15.982663 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1054/2332] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:15 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:16.894494 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7df03664-47df-4639-9cb4-999f14cfd35f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:16.895041 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7df03664-47df-4639-9cb4-999f14cfd35f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:16.899676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a060ec84-51aa-4735-8b12-966635697e28 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:16.900273 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a060ec84-51aa-4735-8b12-966635697e28 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:16.901770 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7df03664-47df-4639-9cb4-999f14cfd35f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:16.902239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7df03664-47df-4639-9cb4-999f14cfd35f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:16.902687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7df03664-47df-4639-9cb4-999f14cfd35f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:16.903417 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7df03664-47df-4639-9cb4-999f14cfd35f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:16.904009 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7df03664-47df-4639-9cb4-999f14cfd35f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:16.904445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7df03664-47df-4639-9cb4-999f14cfd35f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:16.906948 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a060ec84-51aa-4735-8b12-966635697e28 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:16.907433 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a060ec84-51aa-4735-8b12-966635697e28 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:16.908197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a060ec84-51aa-4735-8b12-966635697e28 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:16.909005 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a060ec84-51aa-4735-8b12-966635697e28 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:16.909450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a060ec84-51aa-4735-8b12-966635697e28 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:16.909894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a060ec84-51aa-4735-8b12-966635697e28 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:17.115506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7df03664-47df-4639-9cb4-999f14cfd35f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:17.121227 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a060ec84-51aa-4735-8b12-966635697e28 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:17.130197 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7df03664-47df-4639-9cb4-999f14cfd35f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.241496 Aug 28 01:21:17.130914 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1279/2333] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:16 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:17.131526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a060ec84-51aa-4735-8b12-966635697e28 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:17.133143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a060ec84-51aa-4735-8b12-966635697e28 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:17.134443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a060ec84-51aa-4735-8b12-966635697e28 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.238753 Aug 28 01:21:17.137739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-948bd9c3-642f-46e7-8c5f-1bb822e8164a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:17.138607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-948bd9c3-642f-46e7-8c5f-1bb822e8164a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:17.145235 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-948bd9c3-642f-46e7-8c5f-1bb822e8164a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:17.145956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-948bd9c3-642f-46e7-8c5f-1bb822e8164a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:17.146559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-948bd9c3-642f-46e7-8c5f-1bb822e8164a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:17.147263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-948bd9c3-642f-46e7-8c5f-1bb822e8164a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:17.147712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-948bd9c3-642f-46e7-8c5f-1bb822e8164a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:17.148133 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-948bd9c3-642f-46e7-8c5f-1bb822e8164a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:17.149386 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1055/2334] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:16 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:17.160071 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e052d1c1-d31d-4e0a-b20c-022014484497 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:17.160765 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e052d1c1-d31d-4e0a-b20c-022014484497 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:17.168775 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e052d1c1-d31d-4e0a-b20c-022014484497 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:17.169654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e052d1c1-d31d-4e0a-b20c-022014484497 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:17.170277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e052d1c1-d31d-4e0a-b20c-022014484497 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:17.171730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e052d1c1-d31d-4e0a-b20c-022014484497 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:17.172306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e052d1c1-d31d-4e0a-b20c-022014484497 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:17.172830 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e052d1c1-d31d-4e0a-b20c-022014484497 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:17.216515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-948bd9c3-642f-46e7-8c5f-1bb822e8164a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:17.224988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-948bd9c3-642f-46e7-8c5f-1bb822e8164a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.092534 Aug 28 01:21:17.225565 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1280/2335] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:17 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:17.247047 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e052d1c1-d31d-4e0a-b20c-022014484497 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:17.255778 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e052d1c1-d31d-4e0a-b20c-022014484497 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.104926 Aug 28 01:21:17.256428 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1056/2336] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:17 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:18.152815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f7268486-594b-438c-8f32-031253e48880 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:18.153529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f7268486-594b-438c-8f32-031253e48880 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:18.160050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7268486-594b-438c-8f32-031253e48880 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:18.160537 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7268486-594b-438c-8f32-031253e48880 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:18.161003 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7268486-594b-438c-8f32-031253e48880 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:18.161693 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7268486-594b-438c-8f32-031253e48880 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:18.162169 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7268486-594b-438c-8f32-031253e48880 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:18.162593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7268486-594b-438c-8f32-031253e48880 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:18.164936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d0b6c3a8-3701-4f11-bc1f-e7ddd919227f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:18.165569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d0b6c3a8-3701-4f11-bc1f-e7ddd919227f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:18.172331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0b6c3a8-3701-4f11-bc1f-e7ddd919227f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:18.172815 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0b6c3a8-3701-4f11-bc1f-e7ddd919227f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:18.173282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0b6c3a8-3701-4f11-bc1f-e7ddd919227f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:18.173982 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0b6c3a8-3701-4f11-bc1f-e7ddd919227f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:18.174427 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0b6c3a8-3701-4f11-bc1f-e7ddd919227f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:18.174840 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0b6c3a8-3701-4f11-bc1f-e7ddd919227f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:18.358780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f7268486-594b-438c-8f32-031253e48880 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:18.363438 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d0b6c3a8-3701-4f11-bc1f-e7ddd919227f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:18.368765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f7268486-594b-438c-8f32-031253e48880 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.219978 Aug 28 01:21:18.369486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1281/2337] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:18 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:18.374815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0f8bc94a-26fb-4cbc-8735-511e540a175b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:18.375563 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0f8bc94a-26fb-4cbc-8735-511e540a175b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:18.375787 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d0b6c3a8-3701-4f11-bc1f-e7ddd919227f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:18.377398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d0b6c3a8-3701-4f11-bc1f-e7ddd919227f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:18.378664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d0b6c3a8-3701-4f11-bc1f-e7ddd919227f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.217609 Aug 28 01:21:18.379585 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1057/2338] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:18 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:18.382404 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f8bc94a-26fb-4cbc-8735-511e540a175b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:18.383134 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f8bc94a-26fb-4cbc-8735-511e540a175b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:18.383800 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f8bc94a-26fb-4cbc-8735-511e540a175b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:18.384408 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e1bbf629-c00c-40fe-80f8-5ea0e3fc6d1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:18.385000 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f8bc94a-26fb-4cbc-8735-511e540a175b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:18.385125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e1bbf629-c00c-40fe-80f8-5ea0e3fc6d1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:18.385784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f8bc94a-26fb-4cbc-8735-511e540a175b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:18.386413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f8bc94a-26fb-4cbc-8735-511e540a175b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:18.392828 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1bbf629-c00c-40fe-80f8-5ea0e3fc6d1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:18.393799 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1bbf629-c00c-40fe-80f8-5ea0e3fc6d1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:18.394567 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1bbf629-c00c-40fe-80f8-5ea0e3fc6d1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:18.395847 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1bbf629-c00c-40fe-80f8-5ea0e3fc6d1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:18.396763 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1bbf629-c00c-40fe-80f8-5ea0e3fc6d1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:18.397566 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1bbf629-c00c-40fe-80f8-5ea0e3fc6d1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:18.468132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0f8bc94a-26fb-4cbc-8735-511e540a175b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:18.476997 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0f8bc94a-26fb-4cbc-8735-511e540a175b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.106225 Aug 28 01:21:18.477846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1282/2339] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:18 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:18.483260 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e1bbf629-c00c-40fe-80f8-5ea0e3fc6d1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:18.492029 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e1bbf629-c00c-40fe-80f8-5ea0e3fc6d1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.111441 Aug 28 01:21:18.492597 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1058/2340] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:18 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:19.400083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cfe16eae-4f27-4571-b3e5-f7e142bbf261 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:19.400707 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cfe16eae-4f27-4571-b3e5-f7e142bbf261 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:19.407519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cfe16eae-4f27-4571-b3e5-f7e142bbf261 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:19.408004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cfe16eae-4f27-4571-b3e5-f7e142bbf261 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:19.408438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cfe16eae-4f27-4571-b3e5-f7e142bbf261 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:19.409174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cfe16eae-4f27-4571-b3e5-f7e142bbf261 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:19.409668 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cfe16eae-4f27-4571-b3e5-f7e142bbf261 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:19.410096 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cfe16eae-4f27-4571-b3e5-f7e142bbf261 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:19.415899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-60ab53a2-16fe-4336-a39a-db2a7e9e5bf9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:19.416582 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-60ab53a2-16fe-4336-a39a-db2a7e9e5bf9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:19.423529 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-60ab53a2-16fe-4336-a39a-db2a7e9e5bf9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:19.424003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-60ab53a2-16fe-4336-a39a-db2a7e9e5bf9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:19.424431 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-60ab53a2-16fe-4336-a39a-db2a7e9e5bf9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:19.425158 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-60ab53a2-16fe-4336-a39a-db2a7e9e5bf9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:19.425598 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-60ab53a2-16fe-4336-a39a-db2a7e9e5bf9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:19.426012 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-60ab53a2-16fe-4336-a39a-db2a7e9e5bf9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:19.606234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-cfe16eae-4f27-4571-b3e5-f7e142bbf261 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:19.616724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cfe16eae-4f27-4571-b3e5-f7e142bbf261 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.220483 Aug 28 01:21:19.617328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1283/2341] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:19 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:19.622038 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2136284f-0bfa-4457-a027-dd2b06ad5f7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:19.622649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2136284f-0bfa-4457-a027-dd2b06ad5f7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:19.627404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-60ab53a2-16fe-4336-a39a-db2a7e9e5bf9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:19.628922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2136284f-0bfa-4457-a027-dd2b06ad5f7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:19.629444 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2136284f-0bfa-4457-a027-dd2b06ad5f7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:19.629875 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2136284f-0bfa-4457-a027-dd2b06ad5f7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:19.630569 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2136284f-0bfa-4457-a027-dd2b06ad5f7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:19.631016 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2136284f-0bfa-4457-a027-dd2b06ad5f7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:19.631443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2136284f-0bfa-4457-a027-dd2b06ad5f7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:19.639044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-60ab53a2-16fe-4336-a39a-db2a7e9e5bf9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:19.640692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-60ab53a2-16fe-4336-a39a-db2a7e9e5bf9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:19.642326 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-60ab53a2-16fe-4336-a39a-db2a7e9e5bf9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.230211 Aug 28 01:21:19.643433 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1059/2342] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:19 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:19.654597 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-90d79404-d460-4bc5-b1e6-b6652eeb8308 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:19.654597 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-90d79404-d460-4bc5-b1e6-b6652eeb8308 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:19.662451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90d79404-d460-4bc5-b1e6-b6652eeb8308 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:19.662954 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90d79404-d460-4bc5-b1e6-b6652eeb8308 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:19.663391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90d79404-d460-4bc5-b1e6-b6652eeb8308 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:19.664098 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90d79404-d460-4bc5-b1e6-b6652eeb8308 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:19.664552 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90d79404-d460-4bc5-b1e6-b6652eeb8308 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:19.664995 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90d79404-d460-4bc5-b1e6-b6652eeb8308 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:19.696949 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2136284f-0bfa-4457-a027-dd2b06ad5f7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:19.705790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2136284f-0bfa-4457-a027-dd2b06ad5f7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.087166 Aug 28 01:21:19.706463 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1284/2343] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:19 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:19.728436 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-90d79404-d460-4bc5-b1e6-b6652eeb8308 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:19.730717 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:21:19.737846 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-90d79404-d460-4bc5-b1e6-b6652eeb8308 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.092994 Aug 28 01:21:19.738413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1060/2344] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:19 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:20.640144 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-eb9a7cd2-c8d2-41cb-b04f-cc8f48ac3571 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:20.640788 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-eb9a7cd2-c8d2-41cb-b04f-cc8f48ac3571 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:20.650389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb9a7cd2-c8d2-41cb-b04f-cc8f48ac3571 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:20.650928 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb9a7cd2-c8d2-41cb-b04f-cc8f48ac3571 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:20.651499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb9a7cd2-c8d2-41cb-b04f-cc8f48ac3571 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:20.652494 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb9a7cd2-c8d2-41cb-b04f-cc8f48ac3571 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:20.652995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb9a7cd2-c8d2-41cb-b04f-cc8f48ac3571 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:20.653468 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb9a7cd2-c8d2-41cb-b04f-cc8f48ac3571 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:20.669418 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-da9720e8-fe37-44f2-87e2-f6989dd3406c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:20.670126 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-da9720e8-fe37-44f2-87e2-f6989dd3406c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:20.677288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da9720e8-fe37-44f2-87e2-f6989dd3406c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:20.677779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da9720e8-fe37-44f2-87e2-f6989dd3406c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:20.678205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da9720e8-fe37-44f2-87e2-f6989dd3406c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:20.678906 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da9720e8-fe37-44f2-87e2-f6989dd3406c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:20.679383 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da9720e8-fe37-44f2-87e2-f6989dd3406c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:20.679825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da9720e8-fe37-44f2-87e2-f6989dd3406c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:20.848290 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-eb9a7cd2-c8d2-41cb-b04f-cc8f48ac3571 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:20.857289 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-eb9a7cd2-c8d2-41cb-b04f-cc8f48ac3571 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.222949 Aug 28 01:21:20.869208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1285/2345] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:20 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:20.869455 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-da9720e8-fe37-44f2-87e2-f6989dd3406c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:20.869587 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-85c19991-31df-4d63-a888-b19d05956ec5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:20.869587 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-85c19991-31df-4d63-a888-b19d05956ec5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:20.869587 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85c19991-31df-4d63-a888-b19d05956ec5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:20.869890 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-da9720e8-fe37-44f2-87e2-f6989dd3406c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:20.870000 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85c19991-31df-4d63-a888-b19d05956ec5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:20.870000 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85c19991-31df-4d63-a888-b19d05956ec5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:20.870375 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-da9720e8-fe37-44f2-87e2-f6989dd3406c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:20.870524 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85c19991-31df-4d63-a888-b19d05956ec5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:20.870837 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85c19991-31df-4d63-a888-b19d05956ec5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:20.871251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85c19991-31df-4d63-a888-b19d05956ec5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:20.872228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-da9720e8-fe37-44f2-87e2-f6989dd3406c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.207087 Aug 28 01:21:20.872984 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1061/2346] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:20 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:20.877633 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-eb3a1aaa-d352-418a-815a-48b4c0cfb843 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:20.878277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-eb3a1aaa-d352-418a-815a-48b4c0cfb843 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:20.884646 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb3a1aaa-d352-418a-815a-48b4c0cfb843 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:20.885164 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb3a1aaa-d352-418a-815a-48b4c0cfb843 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:20.885588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb3a1aaa-d352-418a-815a-48b4c0cfb843 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:20.886441 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb3a1aaa-d352-418a-815a-48b4c0cfb843 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:20.886714 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb3a1aaa-d352-418a-815a-48b4c0cfb843 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:20.887142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb3a1aaa-d352-418a-815a-48b4c0cfb843 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:20.963921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-85c19991-31df-4d63-a888-b19d05956ec5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:20.965553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-eb3a1aaa-d352-418a-815a-48b4c0cfb843 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:20.974476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-85c19991-31df-4d63-a888-b19d05956ec5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.115580 Aug 28 01:21:20.979811 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1286/2347] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:20 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:20.980025 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-eb3a1aaa-d352-418a-815a-48b4c0cfb843 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.102317 Aug 28 01:21:20.980025 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1062/2348] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:20 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:21.880541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9de9def2-37fc-4c71-8473-3df79c142f31 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:21.881266 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9de9def2-37fc-4c71-8473-3df79c142f31 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:21.888182 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9de9def2-37fc-4c71-8473-3df79c142f31 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:21.888678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9de9def2-37fc-4c71-8473-3df79c142f31 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:21.889209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9de9def2-37fc-4c71-8473-3df79c142f31 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:21.889940 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9de9def2-37fc-4c71-8473-3df79c142f31 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:21.890468 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9de9def2-37fc-4c71-8473-3df79c142f31 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:21.890907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9de9def2-37fc-4c71-8473-3df79c142f31 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:21.895757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9c62a091-d977-4183-a57f-1910510cbb1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:21.896336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9c62a091-d977-4183-a57f-1910510cbb1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:21.903515 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9c62a091-d977-4183-a57f-1910510cbb1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:21.903995 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9c62a091-d977-4183-a57f-1910510cbb1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:21.904417 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9c62a091-d977-4183-a57f-1910510cbb1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:21.905137 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9c62a091-d977-4183-a57f-1910510cbb1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:21.905576 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9c62a091-d977-4183-a57f-1910510cbb1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:21.905993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9c62a091-d977-4183-a57f-1910510cbb1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:22.090212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9de9def2-37fc-4c71-8473-3df79c142f31 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:22.097173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9c62a091-d977-4183-a57f-1910510cbb1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:22.099079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9de9def2-37fc-4c71-8473-3df79c142f31 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.222399 Aug 28 01:21:22.099742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1287/2349] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:21 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:22.109224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d5d55060-59d7-4d33-b693-4be185767de6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:22.109224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d5d55060-59d7-4d33-b693-4be185767de6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:22.109459 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9c62a091-d977-4183-a57f-1910510cbb1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:22.109459 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9c62a091-d977-4183-a57f-1910510cbb1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:22.109459 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9c62a091-d977-4183-a57f-1910510cbb1b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.217385 Aug 28 01:21:22.110084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1063/2350] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:21 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:22.114716 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0f4992dd-46cc-46c5-b104-43e268ae2e91 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:22.115314 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0f4992dd-46cc-46c5-b104-43e268ae2e91 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:22.119964 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5d55060-59d7-4d33-b693-4be185767de6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:22.120768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5d55060-59d7-4d33-b693-4be185767de6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:22.120881 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5d55060-59d7-4d33-b693-4be185767de6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:22.121690 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5d55060-59d7-4d33-b693-4be185767de6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:22.122201 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5d55060-59d7-4d33-b693-4be185767de6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:22.122880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5d55060-59d7-4d33-b693-4be185767de6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:22.123393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f4992dd-46cc-46c5-b104-43e268ae2e91 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:22.123872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f4992dd-46cc-46c5-b104-43e268ae2e91 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:22.124297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f4992dd-46cc-46c5-b104-43e268ae2e91 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:22.124997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f4992dd-46cc-46c5-b104-43e268ae2e91 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:22.125434 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f4992dd-46cc-46c5-b104-43e268ae2e91 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:22.125983 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0f4992dd-46cc-46c5-b104-43e268ae2e91 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:22.206229 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d5d55060-59d7-4d33-b693-4be185767de6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:22.207991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0f4992dd-46cc-46c5-b104-43e268ae2e91 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:22.216817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d5d55060-59d7-4d33-b693-4be185767de6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.115680 Aug 28 01:21:22.218048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1288/2351] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:22 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:22.218598 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0f4992dd-46cc-46c5-b104-43e268ae2e91 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.107332 Aug 28 01:21:22.219130 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1064/2352] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:22 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:23.129012 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-65e19359-669e-4755-ba67-b1e679889412 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:23.129688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-65e19359-669e-4755-ba67-b1e679889412 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:23.135993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5b6f7760-cd5d-4445-bc53-8374ea3cae1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:23.136620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5b6f7760-cd5d-4445-bc53-8374ea3cae1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:23.136803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65e19359-669e-4755-ba67-b1e679889412 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:23.137238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65e19359-669e-4755-ba67-b1e679889412 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:23.137667 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65e19359-669e-4755-ba67-b1e679889412 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:23.138361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65e19359-669e-4755-ba67-b1e679889412 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:23.138799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65e19359-669e-4755-ba67-b1e679889412 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:23.139221 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65e19359-669e-4755-ba67-b1e679889412 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:23.142855 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b6f7760-cd5d-4445-bc53-8374ea3cae1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:23.143347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b6f7760-cd5d-4445-bc53-8374ea3cae1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:23.143770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b6f7760-cd5d-4445-bc53-8374ea3cae1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:23.144491 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b6f7760-cd5d-4445-bc53-8374ea3cae1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:23.144937 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b6f7760-cd5d-4445-bc53-8374ea3cae1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:23.145385 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b6f7760-cd5d-4445-bc53-8374ea3cae1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:23.331669 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-65e19359-669e-4755-ba67-b1e679889412 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:23.334803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5b6f7760-cd5d-4445-bc53-8374ea3cae1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:23.340627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-65e19359-669e-4755-ba67-b1e679889412 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.215589 Aug 28 01:21:23.353265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1289/2353] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:23 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:23.353499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5b6f7760-cd5d-4445-bc53-8374ea3cae1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:23.353499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5b6f7760-cd5d-4445-bc53-8374ea3cae1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:23.353721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c3c035e1-5cd8-425b-b815-06b22fa2f633 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:23.353810 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5b6f7760-cd5d-4445-bc53-8374ea3cae1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.214788 Aug 28 01:21:23.353896 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c3c035e1-5cd8-425b-b815-06b22fa2f633 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:23.353998 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1065/2354] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:23 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:23.353998 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b326b898-98e8-4bea-8a34-f58c793ab277 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:23.353998 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b326b898-98e8-4bea-8a34-f58c793ab277 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:23.354239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3c035e1-5cd8-425b-b815-06b22fa2f633 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:23.354239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3c035e1-5cd8-425b-b815-06b22fa2f633 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:23.354239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3c035e1-5cd8-425b-b815-06b22fa2f633 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:23.354927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3c035e1-5cd8-425b-b815-06b22fa2f633 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:23.355019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3c035e1-5cd8-425b-b815-06b22fa2f633 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:23.355541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3c035e1-5cd8-425b-b815-06b22fa2f633 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:23.357861 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b326b898-98e8-4bea-8a34-f58c793ab277 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:23.358336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b326b898-98e8-4bea-8a34-f58c793ab277 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:23.358523 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:21:23.358773 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b326b898-98e8-4bea-8a34-f58c793ab277 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:23.359998 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b326b898-98e8-4bea-8a34-f58c793ab277 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:23.360463 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b326b898-98e8-4bea-8a34-f58c793ab277 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:23.361002 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b326b898-98e8-4bea-8a34-f58c793ab277 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:23.445017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c3c035e1-5cd8-425b-b815-06b22fa2f633 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:23.447192 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b326b898-98e8-4bea-8a34-f58c793ab277 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:23.458196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c3c035e1-5cd8-425b-b815-06b22fa2f633 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.115261 Aug 28 01:21:23.459003 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1290/2355] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:23 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:23.460107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b326b898-98e8-4bea-8a34-f58c793ab277 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.111757 Aug 28 01:21:23.460710 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1066/2356] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:23 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:24.364123 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a980a2e5-13b8-41ab-9751-e87d9af2f1d9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:24.364753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a980a2e5-13b8-41ab-9751-e87d9af2f1d9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:24.369565 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7c3b4716-3f45-44a3-9742-d2c70fdd684c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:24.370201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7c3b4716-3f45-44a3-9742-d2c70fdd684c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:24.371621 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a980a2e5-13b8-41ab-9751-e87d9af2f1d9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:24.372102 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a980a2e5-13b8-41ab-9751-e87d9af2f1d9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:24.372525 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a980a2e5-13b8-41ab-9751-e87d9af2f1d9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:24.373311 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a980a2e5-13b8-41ab-9751-e87d9af2f1d9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:24.373699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a980a2e5-13b8-41ab-9751-e87d9af2f1d9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:24.374085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a980a2e5-13b8-41ab-9751-e87d9af2f1d9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:24.376675 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7c3b4716-3f45-44a3-9742-d2c70fdd684c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:24.377189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7c3b4716-3f45-44a3-9742-d2c70fdd684c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:24.377621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7c3b4716-3f45-44a3-9742-d2c70fdd684c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:24.378305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7c3b4716-3f45-44a3-9742-d2c70fdd684c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:24.378767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7c3b4716-3f45-44a3-9742-d2c70fdd684c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:24.379158 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7c3b4716-3f45-44a3-9742-d2c70fdd684c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:24.575448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a980a2e5-13b8-41ab-9751-e87d9af2f1d9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:24.579558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7c3b4716-3f45-44a3-9742-d2c70fdd684c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:24.588304 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a980a2e5-13b8-41ab-9751-e87d9af2f1d9 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.227771 Aug 28 01:21:24.589343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1291/2357] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:24 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:24.591208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7c3b4716-3f45-44a3-9742-d2c70fdd684c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:24.592903 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7c3b4716-3f45-44a3-9742-d2c70fdd684c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:24.594006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b7dece6e-de57-42df-8b1e-c5505de42d99 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:24.594165 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7c3b4716-3f45-44a3-9742-d2c70fdd684c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.228521 Aug 28 01:21:24.594583 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1067/2358] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:24 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:24.594725 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b7dece6e-de57-42df-8b1e-c5505de42d99 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:24.599212 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a7e36dc7-d15b-4070-8627-65880801dcc5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:24.600075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a7e36dc7-d15b-4070-8627-65880801dcc5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:24.600928 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7dece6e-de57-42df-8b1e-c5505de42d99 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:24.601506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7dece6e-de57-42df-8b1e-c5505de42d99 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:24.602011 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7dece6e-de57-42df-8b1e-c5505de42d99 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:24.602771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7dece6e-de57-42df-8b1e-c5505de42d99 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:24.603286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7dece6e-de57-42df-8b1e-c5505de42d99 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:24.603715 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b7dece6e-de57-42df-8b1e-c5505de42d99 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:24.608276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7e36dc7-d15b-4070-8627-65880801dcc5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:24.608757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7e36dc7-d15b-4070-8627-65880801dcc5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:24.609185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7e36dc7-d15b-4070-8627-65880801dcc5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:24.609870 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7e36dc7-d15b-4070-8627-65880801dcc5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:24.610312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7e36dc7-d15b-4070-8627-65880801dcc5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:24.610723 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7e36dc7-d15b-4070-8627-65880801dcc5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:24.687787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b7dece6e-de57-42df-8b1e-c5505de42d99 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:24.689653 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a7e36dc7-d15b-4070-8627-65880801dcc5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:24.697098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b7dece6e-de57-42df-8b1e-c5505de42d99 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.106696 Aug 28 01:21:24.697685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1292/2359] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:24 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:24.699414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a7e36dc7-d15b-4070-8627-65880801dcc5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.103108 Aug 28 01:21:24.700259 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1068/2360] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:24 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:25.611511 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-88e276b9-6bfd-41dd-856d-8b00b8dda510 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:25.612164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-88e276b9-6bfd-41dd-856d-8b00b8dda510 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:25.618324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c137104d-5874-433b-855e-61f6c31ab9a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:25.618886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c137104d-5874-433b-855e-61f6c31ab9a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:25.619370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88e276b9-6bfd-41dd-856d-8b00b8dda510 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:25.619826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88e276b9-6bfd-41dd-856d-8b00b8dda510 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:25.620250 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88e276b9-6bfd-41dd-856d-8b00b8dda510 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:25.621059 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88e276b9-6bfd-41dd-856d-8b00b8dda510 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:25.621529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88e276b9-6bfd-41dd-856d-8b00b8dda510 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:25.622026 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88e276b9-6bfd-41dd-856d-8b00b8dda510 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:25.625259 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c137104d-5874-433b-855e-61f6c31ab9a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:25.625743 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c137104d-5874-433b-855e-61f6c31ab9a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:25.627453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c137104d-5874-433b-855e-61f6c31ab9a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:25.627453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c137104d-5874-433b-855e-61f6c31ab9a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:25.627453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c137104d-5874-433b-855e-61f6c31ab9a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:25.627774 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c137104d-5874-433b-855e-61f6c31ab9a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:25.817954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-88e276b9-6bfd-41dd-856d-8b00b8dda510 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:25.821112 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c137104d-5874-433b-855e-61f6c31ab9a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:25.826886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-88e276b9-6bfd-41dd-856d-8b00b8dda510 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.219447 Aug 28 01:21:25.827495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1293/2361] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:25 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:25.830075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c137104d-5874-433b-855e-61f6c31ab9a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:25.831757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c137104d-5874-433b-855e-61f6c31ab9a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:25.832096 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-06fe743e-e1ae-4474-a605-e2e0483c2355 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:25.832732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-06fe743e-e1ae-4474-a605-e2e0483c2355 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:25.833000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c137104d-5874-433b-855e-61f6c31ab9a6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.218751 Aug 28 01:21:25.842509 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1069/2362] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:25 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:25.842509 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9272417d-187c-4f9a-abc4-ad4176334f00 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:25.842509 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9272417d-187c-4f9a-abc4-ad4176334f00 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:25.842954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-06fe743e-e1ae-4474-a605-e2e0483c2355 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:25.842954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-06fe743e-e1ae-4474-a605-e2e0483c2355 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:25.842954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-06fe743e-e1ae-4474-a605-e2e0483c2355 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:25.842954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-06fe743e-e1ae-4474-a605-e2e0483c2355 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:25.843296 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-06fe743e-e1ae-4474-a605-e2e0483c2355 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:25.843296 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-06fe743e-e1ae-4474-a605-e2e0483c2355 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:25.844677 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9272417d-187c-4f9a-abc4-ad4176334f00 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:25.845186 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9272417d-187c-4f9a-abc4-ad4176334f00 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:25.845621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9272417d-187c-4f9a-abc4-ad4176334f00 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:25.846325 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9272417d-187c-4f9a-abc4-ad4176334f00 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:25.846801 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9272417d-187c-4f9a-abc4-ad4176334f00 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:25.847196 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9272417d-187c-4f9a-abc4-ad4176334f00 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:25.912669 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-06fe743e-e1ae-4474-a605-e2e0483c2355 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:25.921638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-06fe743e-e1ae-4474-a605-e2e0483c2355 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.093170 Aug 28 01:21:25.922420 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1294/2363] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:25 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:25.928304 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9272417d-187c-4f9a-abc4-ad4176334f00 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:25.937745 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9272417d-187c-4f9a-abc4-ad4176334f00 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.103117 Aug 28 01:21:25.938599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1070/2364] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:25 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:26.851398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a245dc7f-8c91-44d3-a3f3-234811b958db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:26.851769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a245dc7f-8c91-44d3-a3f3-234811b958db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:26.856680 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-117121ec-1d2b-426c-b8af-4e45f550fa72 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:26.857379 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-117121ec-1d2b-426c-b8af-4e45f550fa72 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:26.858455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a245dc7f-8c91-44d3-a3f3-234811b958db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:26.858952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a245dc7f-8c91-44d3-a3f3-234811b958db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:26.859390 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a245dc7f-8c91-44d3-a3f3-234811b958db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:26.860102 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a245dc7f-8c91-44d3-a3f3-234811b958db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:26.860541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a245dc7f-8c91-44d3-a3f3-234811b958db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:26.860990 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a245dc7f-8c91-44d3-a3f3-234811b958db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:26.864195 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-117121ec-1d2b-426c-b8af-4e45f550fa72 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:26.864728 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-117121ec-1d2b-426c-b8af-4e45f550fa72 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:26.865115 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-117121ec-1d2b-426c-b8af-4e45f550fa72 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:26.865834 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-117121ec-1d2b-426c-b8af-4e45f550fa72 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:26.866279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-117121ec-1d2b-426c-b8af-4e45f550fa72 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:26.866847 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-117121ec-1d2b-426c-b8af-4e45f550fa72 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:27.061209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a245dc7f-8c91-44d3-a3f3-234811b958db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:27.064383 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-117121ec-1d2b-426c-b8af-4e45f550fa72 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:27.070542 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a245dc7f-8c91-44d3-a3f3-234811b958db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.225093 Aug 28 01:21:27.074891 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1295/2365] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:26 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:27.075111 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-117121ec-1d2b-426c-b8af-4e45f550fa72 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:27.075587 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-117121ec-1d2b-426c-b8af-4e45f550fa72 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:27.076101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cee38fd4-7d2e-4dc5-af44-f0f9df0bcc8d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:27.076757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cee38fd4-7d2e-4dc5-af44-f0f9df0bcc8d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:27.076921 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-117121ec-1d2b-426c-b8af-4e45f550fa72 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.224292 Aug 28 01:21:27.077794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1071/2366] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:26 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:27.083722 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cee38fd4-7d2e-4dc5-af44-f0f9df0bcc8d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:27.084554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f2ff22cb-3475-4dfe-af36-77be11b46cf5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:27.084695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cee38fd4-7d2e-4dc5-af44-f0f9df0bcc8d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:27.085333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f2ff22cb-3475-4dfe-af36-77be11b46cf5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:27.085464 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cee38fd4-7d2e-4dc5-af44-f0f9df0bcc8d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:27.086302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cee38fd4-7d2e-4dc5-af44-f0f9df0bcc8d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:27.086748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cee38fd4-7d2e-4dc5-af44-f0f9df0bcc8d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:27.087162 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cee38fd4-7d2e-4dc5-af44-f0f9df0bcc8d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:27.091441 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2ff22cb-3475-4dfe-af36-77be11b46cf5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:27.091956 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2ff22cb-3475-4dfe-af36-77be11b46cf5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:27.092392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2ff22cb-3475-4dfe-af36-77be11b46cf5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:27.093112 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2ff22cb-3475-4dfe-af36-77be11b46cf5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:27.093562 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2ff22cb-3475-4dfe-af36-77be11b46cf5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:27.093977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f2ff22cb-3475-4dfe-af36-77be11b46cf5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:27.158261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-cee38fd4-7d2e-4dc5-af44-f0f9df0bcc8d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:27.166634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cee38fd4-7d2e-4dc5-af44-f0f9df0bcc8d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.094404 Aug 28 01:21:27.167198 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1296/2367] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:27 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:27.177616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f2ff22cb-3475-4dfe-af36-77be11b46cf5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:27.186463 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f2ff22cb-3475-4dfe-af36-77be11b46cf5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.107672 Aug 28 01:21:27.187044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1072/2368] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:27 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:28.093002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c7830bd9-8a6f-4a69-9cb3-4b778984df25 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:28.093655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c7830bd9-8a6f-4a69-9cb3-4b778984df25 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:28.100648 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7830bd9-8a6f-4a69-9cb3-4b778984df25 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:28.101156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7830bd9-8a6f-4a69-9cb3-4b778984df25 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:28.101421 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-18f8bc4a-7fcf-4ca4-ade9-82ec25f86cd9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:28.101897 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7830bd9-8a6f-4a69-9cb3-4b778984df25 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:28.102014 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-18f8bc4a-7fcf-4ca4-ade9-82ec25f86cd9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:28.102259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7830bd9-8a6f-4a69-9cb3-4b778984df25 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:28.102708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7830bd9-8a6f-4a69-9cb3-4b778984df25 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:28.103124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c7830bd9-8a6f-4a69-9cb3-4b778984df25 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:28.108804 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-18f8bc4a-7fcf-4ca4-ade9-82ec25f86cd9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:28.109772 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-18f8bc4a-7fcf-4ca4-ade9-82ec25f86cd9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:28.110662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-18f8bc4a-7fcf-4ca4-ade9-82ec25f86cd9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:28.111131 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-18f8bc4a-7fcf-4ca4-ade9-82ec25f86cd9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:28.111578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-18f8bc4a-7fcf-4ca4-ade9-82ec25f86cd9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:28.111980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-18f8bc4a-7fcf-4ca4-ade9-82ec25f86cd9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:28.292433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c7830bd9-8a6f-4a69-9cb3-4b778984df25 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:28.298068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-18f8bc4a-7fcf-4ca4-ade9-82ec25f86cd9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:28.304297 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c7830bd9-8a6f-4a69-9cb3-4b778984df25 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.214880 Aug 28 01:21:28.305119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1297/2369] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:28 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:28.308656 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-18f8bc4a-7fcf-4ca4-ade9-82ec25f86cd9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:28.309632 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7c997e0b-058b-49bc-a2fe-2982181d7398 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:28.310133 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-18f8bc4a-7fcf-4ca4-ade9-82ec25f86cd9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:28.310577 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7c997e0b-058b-49bc-a2fe-2982181d7398 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:28.311300 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-18f8bc4a-7fcf-4ca4-ade9-82ec25f86cd9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.214180 Aug 28 01:21:28.312004 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1073/2370] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:28 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:28.316925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ead8f4c2-c06e-4fea-be3b-0f6b9b4e8059 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:28.317774 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ead8f4c2-c06e-4fea-be3b-0f6b9b4e8059 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:28.324205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ead8f4c2-c06e-4fea-be3b-0f6b9b4e8059 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:28.324712 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ead8f4c2-c06e-4fea-be3b-0f6b9b4e8059 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:28.325169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ead8f4c2-c06e-4fea-be3b-0f6b9b4e8059 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:28.325761 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c997e0b-058b-49bc-a2fe-2982181d7398 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:28.325915 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ead8f4c2-c06e-4fea-be3b-0f6b9b4e8059 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:28.326297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ead8f4c2-c06e-4fea-be3b-0f6b9b4e8059 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:28.326663 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c997e0b-058b-49bc-a2fe-2982181d7398 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:28.326811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ead8f4c2-c06e-4fea-be3b-0f6b9b4e8059 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:28.327484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c997e0b-058b-49bc-a2fe-2982181d7398 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:28.328578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c997e0b-058b-49bc-a2fe-2982181d7398 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:28.329306 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c997e0b-058b-49bc-a2fe-2982181d7398 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:28.329968 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c997e0b-058b-49bc-a2fe-2982181d7398 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:28.399853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ead8f4c2-c06e-4fea-be3b-0f6b9b4e8059 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:28.408676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ead8f4c2-c06e-4fea-be3b-0f6b9b4e8059 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.095809 Aug 28 01:21:28.409333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1074/2371] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:28 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:28.417156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7c997e0b-058b-49bc-a2fe-2982181d7398 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:28.431073 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7c997e0b-058b-49bc-a2fe-2982181d7398 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.124925 Aug 28 01:21:28.431657 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1298/2372] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:28 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:29.325776 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a2ec8acc-9ab3-4c16-8b80-7ee59c5b7547 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:29.326359 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a2ec8acc-9ab3-4c16-8b80-7ee59c5b7547 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:29.333368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a2ec8acc-9ab3-4c16-8b80-7ee59c5b7547 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:29.333853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a2ec8acc-9ab3-4c16-8b80-7ee59c5b7547 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:29.334285 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a2ec8acc-9ab3-4c16-8b80-7ee59c5b7547 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:29.334974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a2ec8acc-9ab3-4c16-8b80-7ee59c5b7547 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:29.335128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b073fc6a-3f54-4dbc-a07d-da4b1fcdf84a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:29.335554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a2ec8acc-9ab3-4c16-8b80-7ee59c5b7547 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:29.335668 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b073fc6a-3f54-4dbc-a07d-da4b1fcdf84a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:29.335832 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a2ec8acc-9ab3-4c16-8b80-7ee59c5b7547 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:29.342391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b073fc6a-3f54-4dbc-a07d-da4b1fcdf84a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:29.342881 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b073fc6a-3f54-4dbc-a07d-da4b1fcdf84a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:29.343292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b073fc6a-3f54-4dbc-a07d-da4b1fcdf84a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:29.344012 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b073fc6a-3f54-4dbc-a07d-da4b1fcdf84a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:29.344451 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b073fc6a-3f54-4dbc-a07d-da4b1fcdf84a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:29.344848 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b073fc6a-3f54-4dbc-a07d-da4b1fcdf84a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:29.534380 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a2ec8acc-9ab3-4c16-8b80-7ee59c5b7547 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:29.537293 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b073fc6a-3f54-4dbc-a07d-da4b1fcdf84a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:29.543481 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a2ec8acc-9ab3-4c16-8b80-7ee59c5b7547 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.221666 Aug 28 01:21:29.544052 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1075/2373] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:29 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:29.546700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b073fc6a-3f54-4dbc-a07d-da4b1fcdf84a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:29.548147 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b073fc6a-3f54-4dbc-a07d-da4b1fcdf84a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:29.548540 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fe969943-cfd7-4ed0-b1da-fb4be6a27267 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:29.549211 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fe969943-cfd7-4ed0-b1da-fb4be6a27267 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:29.549404 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b073fc6a-3f54-4dbc-a07d-da4b1fcdf84a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.218541 Aug 28 01:21:29.550092 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1299/2374] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:29 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:29.554417 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-670fc24e-b7a0-4027-b07b-4ab8b6524cf0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:29.555035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-670fc24e-b7a0-4027-b07b-4ab8b6524cf0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:29.555335 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fe969943-cfd7-4ed0-b1da-fb4be6a27267 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:29.555814 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fe969943-cfd7-4ed0-b1da-fb4be6a27267 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:29.556242 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fe969943-cfd7-4ed0-b1da-fb4be6a27267 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:29.556925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fe969943-cfd7-4ed0-b1da-fb4be6a27267 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:29.557392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fe969943-cfd7-4ed0-b1da-fb4be6a27267 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:29.557809 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fe969943-cfd7-4ed0-b1da-fb4be6a27267 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:29.561047 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-670fc24e-b7a0-4027-b07b-4ab8b6524cf0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:29.562883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-670fc24e-b7a0-4027-b07b-4ab8b6524cf0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:29.562883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-670fc24e-b7a0-4027-b07b-4ab8b6524cf0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:29.562883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-670fc24e-b7a0-4027-b07b-4ab8b6524cf0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:29.563195 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-670fc24e-b7a0-4027-b07b-4ab8b6524cf0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:29.563493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-670fc24e-b7a0-4027-b07b-4ab8b6524cf0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:29.626726 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fe969943-cfd7-4ed0-b1da-fb4be6a27267 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:29.635692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fe969943-cfd7-4ed0-b1da-fb4be6a27267 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.090631 Aug 28 01:21:29.636255 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1076/2375] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:29 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:29.642235 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-670fc24e-b7a0-4027-b07b-4ab8b6524cf0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:29.650537 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-670fc24e-b7a0-4027-b07b-4ab8b6524cf0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.099669 Aug 28 01:21:29.651134 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1300/2376] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:29 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:30.575084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-eb5399bd-2051-44bd-b241-c58a167c9fb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:30.575686 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-eb5399bd-2051-44bd-b241-c58a167c9fb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:30.579563 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9b9b8f98-c976-4488-9f4b-11e921373b0e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:30.580206 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9b9b8f98-c976-4488-9f4b-11e921373b0e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:30.582423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb5399bd-2051-44bd-b241-c58a167c9fb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:30.582900 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb5399bd-2051-44bd-b241-c58a167c9fb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:30.583345 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb5399bd-2051-44bd-b241-c58a167c9fb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:30.584037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb5399bd-2051-44bd-b241-c58a167c9fb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:30.584484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb5399bd-2051-44bd-b241-c58a167c9fb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:30.584902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb5399bd-2051-44bd-b241-c58a167c9fb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:30.586725 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b9b8f98-c976-4488-9f4b-11e921373b0e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:30.587200 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b9b8f98-c976-4488-9f4b-11e921373b0e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:30.587645 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b9b8f98-c976-4488-9f4b-11e921373b0e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:30.588325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b9b8f98-c976-4488-9f4b-11e921373b0e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:30.588762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b9b8f98-c976-4488-9f4b-11e921373b0e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:30.589200 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b9b8f98-c976-4488-9f4b-11e921373b0e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:30.601264 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:21:30.778268 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-eb5399bd-2051-44bd-b241-c58a167c9fb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:30.783454 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9b9b8f98-c976-4488-9f4b-11e921373b0e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:30.790329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-eb5399bd-2051-44bd-b241-c58a167c9fb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:30.792262 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-eb5399bd-2051-44bd-b241-c58a167c9fb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:30.793972 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-eb5399bd-2051-44bd-b241-c58a167c9fb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.223043 Aug 28 01:21:30.794689 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1077/2377] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:30 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:30.795051 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9b9b8f98-c976-4488-9f4b-11e921373b0e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.219207 Aug 28 01:21:30.805172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1301/2378] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:30 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:30.805382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e94c5412-440b-429f-9b5b-4b82dca03398 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:30.805382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e94c5412-440b-429f-9b5b-4b82dca03398 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:30.805600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9692af5b-ddc2-480b-907d-2ee79401beba tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:30.805600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9692af5b-ddc2-480b-907d-2ee79401beba tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:30.805775 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e94c5412-440b-429f-9b5b-4b82dca03398 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:30.806085 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e94c5412-440b-429f-9b5b-4b82dca03398 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:30.806404 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9692af5b-ddc2-480b-907d-2ee79401beba tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:30.806530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e94c5412-440b-429f-9b5b-4b82dca03398 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:30.810285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9692af5b-ddc2-480b-907d-2ee79401beba tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:30.810421 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e94c5412-440b-429f-9b5b-4b82dca03398 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:30.810516 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9692af5b-ddc2-480b-907d-2ee79401beba tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:30.810630 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e94c5412-440b-429f-9b5b-4b82dca03398 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:30.810730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9692af5b-ddc2-480b-907d-2ee79401beba tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:30.810832 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e94c5412-440b-429f-9b5b-4b82dca03398 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:30.810933 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9692af5b-ddc2-480b-907d-2ee79401beba tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:30.810933 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9692af5b-ddc2-480b-907d-2ee79401beba tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:30.892532 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9692af5b-ddc2-480b-907d-2ee79401beba tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:30.894527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e94c5412-440b-429f-9b5b-4b82dca03398 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:30.901004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9692af5b-ddc2-480b-907d-2ee79401beba tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.104383 Aug 28 01:21:30.901578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1302/2379] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:30 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:30.902750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e94c5412-440b-429f-9b5b-4b82dca03398 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.107050 Aug 28 01:21:30.903282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1078/2380] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:30 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:31.822584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8a77fea4-49dc-471e-b50c-9d484cfb1a7b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:31.823187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-baf2c308-886f-4a03-a087-228dd74f4adc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:31.823592 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8a77fea4-49dc-471e-b50c-9d484cfb1a7b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:31.823701 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-baf2c308-886f-4a03-a087-228dd74f4adc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:31.825888 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=96 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:21:31.830426 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8a77fea4-49dc-471e-b50c-9d484cfb1a7b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:31.830909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8a77fea4-49dc-471e-b50c-9d484cfb1a7b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:31.831048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-baf2c308-886f-4a03-a087-228dd74f4adc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:31.831334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8a77fea4-49dc-471e-b50c-9d484cfb1a7b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:31.831448 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-baf2c308-886f-4a03-a087-228dd74f4adc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:31.831822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-baf2c308-886f-4a03-a087-228dd74f4adc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:31.832029 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8a77fea4-49dc-471e-b50c-9d484cfb1a7b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:31.832447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8a77fea4-49dc-471e-b50c-9d484cfb1a7b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:31.832572 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-baf2c308-886f-4a03-a087-228dd74f4adc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:31.832865 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8a77fea4-49dc-471e-b50c-9d484cfb1a7b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:31.833003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-baf2c308-886f-4a03-a087-228dd74f4adc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:31.833374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-baf2c308-886f-4a03-a087-228dd74f4adc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:32.021181 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-baf2c308-886f-4a03-a087-228dd74f4adc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:32.028514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8a77fea4-49dc-471e-b50c-9d484cfb1a7b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:32.037289 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8a77fea4-49dc-471e-b50c-9d484cfb1a7b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.221260 Aug 28 01:21:32.037923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1303/2381] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:31 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:32.038068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-baf2c308-886f-4a03-a087-228dd74f4adc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:32.039441 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-baf2c308-886f-4a03-a087-228dd74f4adc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:32.040577 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-baf2c308-886f-4a03-a087-228dd74f4adc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.224198 Aug 28 01:21:32.041304 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1079/2382] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:31 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:32.045499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-de71f967-a125-447e-a476-104439368e28 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:32.046119 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-de71f967-a125-447e-a476-104439368e28 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:32.052883 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de71f967-a125-447e-a476-104439368e28 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:32.053083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-87fe4b00-b8c9-43bf-807a-9a01679d1d36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:32.053399 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de71f967-a125-447e-a476-104439368e28 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:32.053639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-87fe4b00-b8c9-43bf-807a-9a01679d1d36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:32.053826 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de71f967-a125-447e-a476-104439368e28 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:32.054503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de71f967-a125-447e-a476-104439368e28 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:32.054946 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de71f967-a125-447e-a476-104439368e28 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:32.055349 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de71f967-a125-447e-a476-104439368e28 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:32.059744 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87fe4b00-b8c9-43bf-807a-9a01679d1d36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:32.060196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87fe4b00-b8c9-43bf-807a-9a01679d1d36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:32.060631 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87fe4b00-b8c9-43bf-807a-9a01679d1d36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:32.061394 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87fe4b00-b8c9-43bf-807a-9a01679d1d36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:32.061845 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87fe4b00-b8c9-43bf-807a-9a01679d1d36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:32.062303 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87fe4b00-b8c9-43bf-807a-9a01679d1d36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:32.131367 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-de71f967-a125-447e-a476-104439368e28 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:32.139865 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-de71f967-a125-447e-a476-104439368e28 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.097807 Aug 28 01:21:32.140532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1080/2383] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:32 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:32.147202 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-87fe4b00-b8c9-43bf-807a-9a01679d1d36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:32.155527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-87fe4b00-b8c9-43bf-807a-9a01679d1d36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.116521 Aug 28 01:21:32.156097 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1304/2384] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:32 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:33.060348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5c7b3aac-68d8-4d46-aead-6dfa95b9c075 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:33.061065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5c7b3aac-68d8-4d46-aead-6dfa95b9c075 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:33.063843 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0472b676-4292-48e1-9764-1bd84d5f1726 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:33.064862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0472b676-4292-48e1-9764-1bd84d5f1726 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:33.066841 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=96 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:21:33.068649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5c7b3aac-68d8-4d46-aead-6dfa95b9c075 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:33.069233 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5c7b3aac-68d8-4d46-aead-6dfa95b9c075 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:33.069600 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5c7b3aac-68d8-4d46-aead-6dfa95b9c075 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:33.070293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5c7b3aac-68d8-4d46-aead-6dfa95b9c075 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:33.070760 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5c7b3aac-68d8-4d46-aead-6dfa95b9c075 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:33.071189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5c7b3aac-68d8-4d46-aead-6dfa95b9c075 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:33.072311 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0472b676-4292-48e1-9764-1bd84d5f1726 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:33.072799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0472b676-4292-48e1-9764-1bd84d5f1726 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:33.073255 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0472b676-4292-48e1-9764-1bd84d5f1726 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:33.073960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0472b676-4292-48e1-9764-1bd84d5f1726 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:33.074399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0472b676-4292-48e1-9764-1bd84d5f1726 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:33.074820 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0472b676-4292-48e1-9764-1bd84d5f1726 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:33.266343 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5c7b3aac-68d8-4d46-aead-6dfa95b9c075 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:33.271024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0472b676-4292-48e1-9764-1bd84d5f1726 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:33.277212 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5c7b3aac-68d8-4d46-aead-6dfa95b9c075 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.221538 Aug 28 01:21:33.278031 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1081/2385] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:33 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:33.282376 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ae237cc1-f312-475b-bb21-0f14c83fc111 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:33.282984 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ae237cc1-f312-475b-bb21-0f14c83fc111 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:33.290163 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae237cc1-f312-475b-bb21-0f14c83fc111 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:33.290723 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae237cc1-f312-475b-bb21-0f14c83fc111 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:33.290886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0472b676-4292-48e1-9764-1bd84d5f1726 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:33.291149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae237cc1-f312-475b-bb21-0f14c83fc111 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:33.291883 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae237cc1-f312-475b-bb21-0f14c83fc111 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:33.292327 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae237cc1-f312-475b-bb21-0f14c83fc111 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:33.292709 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae237cc1-f312-475b-bb21-0f14c83fc111 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:33.292857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0472b676-4292-48e1-9764-1bd84d5f1726 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:33.294272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0472b676-4292-48e1-9764-1bd84d5f1726 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.235044 Aug 28 01:21:33.304601 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1305/2386] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:33 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:33.309576 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-88a86dc4-c6d6-4812-a687-8558a5bd6401 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:33.310461 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-88a86dc4-c6d6-4812-a687-8558a5bd6401 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:33.317002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88a86dc4-c6d6-4812-a687-8558a5bd6401 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:33.317737 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88a86dc4-c6d6-4812-a687-8558a5bd6401 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:33.318176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88a86dc4-c6d6-4812-a687-8558a5bd6401 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:33.318877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88a86dc4-c6d6-4812-a687-8558a5bd6401 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:33.319330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88a86dc4-c6d6-4812-a687-8558a5bd6401 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:33.319755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88a86dc4-c6d6-4812-a687-8558a5bd6401 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:33.357979 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ae237cc1-f312-475b-bb21-0f14c83fc111 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:33.366738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ae237cc1-f312-475b-bb21-0f14c83fc111 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.087882 Aug 28 01:21:33.367323 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1082/2387] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:33 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:33.383495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-88a86dc4-c6d6-4812-a687-8558a5bd6401 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:33.385931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:21:33.393266 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-88a86dc4-c6d6-4812-a687-8558a5bd6401 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.087566 Aug 28 01:21:33.393817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1306/2388] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:33 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:34.299974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2bdc8913-ba8a-4c6f-a4a9-386f5d62aa48 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:34.300626 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2bdc8913-ba8a-4c6f-a4a9-386f5d62aa48 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:34.307745 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2bdc8913-ba8a-4c6f-a4a9-386f5d62aa48 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:34.308228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2bdc8913-ba8a-4c6f-a4a9-386f5d62aa48 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:34.308652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2bdc8913-ba8a-4c6f-a4a9-386f5d62aa48 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:34.309376 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2bdc8913-ba8a-4c6f-a4a9-386f5d62aa48 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:34.309822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2bdc8913-ba8a-4c6f-a4a9-386f5d62aa48 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:34.310222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2bdc8913-ba8a-4c6f-a4a9-386f5d62aa48 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:34.318223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c0fa9e8b-0ea7-4022-a35f-16d79f84d3d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:34.318953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c0fa9e8b-0ea7-4022-a35f-16d79f84d3d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:34.325887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0fa9e8b-0ea7-4022-a35f-16d79f84d3d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:34.326455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0fa9e8b-0ea7-4022-a35f-16d79f84d3d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:34.326882 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0fa9e8b-0ea7-4022-a35f-16d79f84d3d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:34.327595 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0fa9e8b-0ea7-4022-a35f-16d79f84d3d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:34.328034 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0fa9e8b-0ea7-4022-a35f-16d79f84d3d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:34.328475 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0fa9e8b-0ea7-4022-a35f-16d79f84d3d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:34.740912 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2bdc8913-ba8a-4c6f-a4a9-386f5d62aa48 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:34.749661 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2bdc8913-ba8a-4c6f-a4a9-386f5d62aa48 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.453622 Aug 28 01:21:34.750079 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1083/2389] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:34 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:34.752449 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c0fa9e8b-0ea7-4022-a35f-16d79f84d3d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:34.754537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-696efe4c-d4c1-42c3-acc0-192904ab0ed1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:34.755150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-696efe4c-d4c1-42c3-acc0-192904ab0ed1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:34.761367 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-696efe4c-d4c1-42c3-acc0-192904ab0ed1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:34.761758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c0fa9e8b-0ea7-4022-a35f-16d79f84d3d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:34.761897 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-696efe4c-d4c1-42c3-acc0-192904ab0ed1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:34.762264 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-696efe4c-d4c1-42c3-acc0-192904ab0ed1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:34.762961 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-696efe4c-d4c1-42c3-acc0-192904ab0ed1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:34.763219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c0fa9e8b-0ea7-4022-a35f-16d79f84d3d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:34.763394 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-696efe4c-d4c1-42c3-acc0-192904ab0ed1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:34.763812 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-696efe4c-d4c1-42c3-acc0-192904ab0ed1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:34.764433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c0fa9e8b-0ea7-4022-a35f-16d79f84d3d2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.450664 Aug 28 01:21:34.765011 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1307/2390] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:34 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:34.769367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-40ba4ad3-aff5-4503-8495-1d22bd70695e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:34.769961 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-40ba4ad3-aff5-4503-8495-1d22bd70695e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:34.775712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-40ba4ad3-aff5-4503-8495-1d22bd70695e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:34.776241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-40ba4ad3-aff5-4503-8495-1d22bd70695e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:34.776696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-40ba4ad3-aff5-4503-8495-1d22bd70695e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:34.777413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-40ba4ad3-aff5-4503-8495-1d22bd70695e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:34.777861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-40ba4ad3-aff5-4503-8495-1d22bd70695e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:34.778277 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-40ba4ad3-aff5-4503-8495-1d22bd70695e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:34.835199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-696efe4c-d4c1-42c3-acc0-192904ab0ed1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:34.843656 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-696efe4c-d4c1-42c3-acc0-192904ab0ed1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.092608 Aug 28 01:21:34.844203 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1084/2391] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:34 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:34.855275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-40ba4ad3-aff5-4503-8495-1d22bd70695e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:34.863889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-40ba4ad3-aff5-4503-8495-1d22bd70695e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.097937 Aug 28 01:21:34.864491 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1308/2392] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:34 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:35.774157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8dba33cb-daa3-4f81-92f5-19aa28a41cad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:35.775021 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8dba33cb-daa3-4f81-92f5-19aa28a41cad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:35.783545 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8dba33cb-daa3-4f81-92f5-19aa28a41cad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:35.784232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8dba33cb-daa3-4f81-92f5-19aa28a41cad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:35.784440 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8dba33cb-daa3-4f81-92f5-19aa28a41cad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:35.785195 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8dba33cb-daa3-4f81-92f5-19aa28a41cad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:35.785645 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8dba33cb-daa3-4f81-92f5-19aa28a41cad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:35.786066 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8dba33cb-daa3-4f81-92f5-19aa28a41cad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:35.788321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-381e75ea-22f4-42cf-a562-c1f21211a8d3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:35.788938 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-381e75ea-22f4-42cf-a562-c1f21211a8d3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:35.795300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-381e75ea-22f4-42cf-a562-c1f21211a8d3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:35.795786 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-381e75ea-22f4-42cf-a562-c1f21211a8d3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:35.796226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-381e75ea-22f4-42cf-a562-c1f21211a8d3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:35.796926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-381e75ea-22f4-42cf-a562-c1f21211a8d3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:35.797418 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-381e75ea-22f4-42cf-a562-c1f21211a8d3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:35.797844 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-381e75ea-22f4-42cf-a562-c1f21211a8d3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:35.982073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8dba33cb-daa3-4f81-92f5-19aa28a41cad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:35.986987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-381e75ea-22f4-42cf-a562-c1f21211a8d3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:35.993741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8dba33cb-daa3-4f81-92f5-19aa28a41cad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.225352 Aug 28 01:21:35.994315 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1085/2393] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:35 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:35.998230 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-381e75ea-22f4-42cf-a562-c1f21211a8d3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:35.998779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ea1a1b3e-29f5-43e0-9c81-967b085f7511 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:35.999378 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ea1a1b3e-29f5-43e0-9c81-967b085f7511 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:35.999918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-381e75ea-22f4-42cf-a562-c1f21211a8d3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:36.002255 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-381e75ea-22f4-42cf-a562-c1f21211a8d3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.217224 Aug 28 01:21:36.002687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1309/2394] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:35 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:36.007265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-52524c6a-99dc-4ac9-8b4e-0e0997b4cb06 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:36.008594 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-52524c6a-99dc-4ac9-8b4e-0e0997b4cb06 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:36.010603 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea1a1b3e-29f5-43e0-9c81-967b085f7511 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:36.011121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea1a1b3e-29f5-43e0-9c81-967b085f7511 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:36.011534 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea1a1b3e-29f5-43e0-9c81-967b085f7511 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:36.012230 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea1a1b3e-29f5-43e0-9c81-967b085f7511 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:36.012678 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea1a1b3e-29f5-43e0-9c81-967b085f7511 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:36.013134 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea1a1b3e-29f5-43e0-9c81-967b085f7511 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:36.014505 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52524c6a-99dc-4ac9-8b4e-0e0997b4cb06 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:36.014937 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52524c6a-99dc-4ac9-8b4e-0e0997b4cb06 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:36.015385 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52524c6a-99dc-4ac9-8b4e-0e0997b4cb06 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:36.016226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52524c6a-99dc-4ac9-8b4e-0e0997b4cb06 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:36.016683 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52524c6a-99dc-4ac9-8b4e-0e0997b4cb06 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:36.018157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52524c6a-99dc-4ac9-8b4e-0e0997b4cb06 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:36.098233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-52524c6a-99dc-4ac9-8b4e-0e0997b4cb06 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:36.101000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ea1a1b3e-29f5-43e0-9c81-967b085f7511 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:36.107006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-52524c6a-99dc-4ac9-8b4e-0e0997b4cb06 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.103214 Aug 28 01:21:36.107634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1310/2395] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:36 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:36.113998 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ea1a1b3e-29f5-43e0-9c81-967b085f7511 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.118594 Aug 28 01:21:36.122586 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1086/2396] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:35 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:37.015344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cae4ca5d-7d4a-44cd-a427-c97dc43a9593 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:37.016095 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cae4ca5d-7d4a-44cd-a427-c97dc43a9593 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:37.023723 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cae4ca5d-7d4a-44cd-a427-c97dc43a9593 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:37.024247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7526584b-d4f0-4080-bbf0-9a794dfedd4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:37.024842 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cae4ca5d-7d4a-44cd-a427-c97dc43a9593 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:37.024982 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7526584b-d4f0-4080-bbf0-9a794dfedd4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:37.025097 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cae4ca5d-7d4a-44cd-a427-c97dc43a9593 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:37.025800 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cae4ca5d-7d4a-44cd-a427-c97dc43a9593 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:37.026256 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cae4ca5d-7d4a-44cd-a427-c97dc43a9593 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:37.026826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cae4ca5d-7d4a-44cd-a427-c97dc43a9593 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:37.032181 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7526584b-d4f0-4080-bbf0-9a794dfedd4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:37.032669 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7526584b-d4f0-4080-bbf0-9a794dfedd4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:37.033129 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7526584b-d4f0-4080-bbf0-9a794dfedd4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:37.034024 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7526584b-d4f0-4080-bbf0-9a794dfedd4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:37.034681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7526584b-d4f0-4080-bbf0-9a794dfedd4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:37.035100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7526584b-d4f0-4080-bbf0-9a794dfedd4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:37.221090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-cae4ca5d-7d4a-44cd-a427-c97dc43a9593 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:37.224559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7526584b-d4f0-4080-bbf0-9a794dfedd4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:37.230041 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cae4ca5d-7d4a-44cd-a427-c97dc43a9593 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.218622 Aug 28 01:21:37.230743 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1311/2397] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:37 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:37.234207 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7526584b-d4f0-4080-bbf0-9a794dfedd4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:37.235046 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-65a4f3b5-0c27-4bbc-ba58-df71da3f6b55 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:37.235639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-65a4f3b5-0c27-4bbc-ba58-df71da3f6b55 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:37.235864 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7526584b-d4f0-4080-bbf0-9a794dfedd4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:37.237175 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7526584b-d4f0-4080-bbf0-9a794dfedd4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.216679 Aug 28 01:21:37.237837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1087/2398] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:37 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:37.241945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1dbef10e-377f-4d69-8f85-3e7bda830b13 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:37.242547 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1dbef10e-377f-4d69-8f85-3e7bda830b13 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:37.248528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1dbef10e-377f-4d69-8f85-3e7bda830b13 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:37.249034 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1dbef10e-377f-4d69-8f85-3e7bda830b13 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:37.249593 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1dbef10e-377f-4d69-8f85-3e7bda830b13 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:37.250330 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1dbef10e-377f-4d69-8f85-3e7bda830b13 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:37.250774 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1dbef10e-377f-4d69-8f85-3e7bda830b13 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:37.251185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1dbef10e-377f-4d69-8f85-3e7bda830b13 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:37.253068 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65a4f3b5-0c27-4bbc-ba58-df71da3f6b55 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:37.253560 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65a4f3b5-0c27-4bbc-ba58-df71da3f6b55 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:37.253995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65a4f3b5-0c27-4bbc-ba58-df71da3f6b55 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:37.254724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65a4f3b5-0c27-4bbc-ba58-df71da3f6b55 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:37.255179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65a4f3b5-0c27-4bbc-ba58-df71da3f6b55 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:37.255604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65a4f3b5-0c27-4bbc-ba58-df71da3f6b55 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:37.337971 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1dbef10e-377f-4d69-8f85-3e7bda830b13 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:37.339873 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-65a4f3b5-0c27-4bbc-ba58-df71da3f6b55 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:37.346842 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1dbef10e-377f-4d69-8f85-3e7bda830b13 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.108203 Aug 28 01:21:37.347420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1088/2399] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:37 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:37.348422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-65a4f3b5-0c27-4bbc-ba58-df71da3f6b55 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.116818 Aug 28 01:21:37.349072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1312/2400] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:37 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:38.254129 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-adc83390-91d5-43fc-a431-713c6db51e9e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:38.254834 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-adc83390-91d5-43fc-a431-713c6db51e9e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:38.259748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-93221ffd-3081-4a38-af82-299db9790b3f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:38.260364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-93221ffd-3081-4a38-af82-299db9790b3f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:38.261962 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-adc83390-91d5-43fc-a431-713c6db51e9e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:38.262471 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-adc83390-91d5-43fc-a431-713c6db51e9e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:38.262896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-adc83390-91d5-43fc-a431-713c6db51e9e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:38.263578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-adc83390-91d5-43fc-a431-713c6db51e9e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:38.264022 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-adc83390-91d5-43fc-a431-713c6db51e9e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:38.264437 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-adc83390-91d5-43fc-a431-713c6db51e9e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:38.266890 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93221ffd-3081-4a38-af82-299db9790b3f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:38.267402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93221ffd-3081-4a38-af82-299db9790b3f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:38.267826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93221ffd-3081-4a38-af82-299db9790b3f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:38.268514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93221ffd-3081-4a38-af82-299db9790b3f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:38.268975 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93221ffd-3081-4a38-af82-299db9790b3f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:38.269411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93221ffd-3081-4a38-af82-299db9790b3f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:38.465538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-93221ffd-3081-4a38-af82-299db9790b3f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:38.467986 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-adc83390-91d5-43fc-a431-713c6db51e9e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:38.474568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-93221ffd-3081-4a38-af82-299db9790b3f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:38.476006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-93221ffd-3081-4a38-af82-299db9790b3f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:38.476926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-adc83390-91d5-43fc-a431-713c6db51e9e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.228283 Aug 28 01:21:38.477196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-93221ffd-3081-4a38-af82-299db9790b3f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.221439 Aug 28 01:21:38.477530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1089/2401] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:38 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:38.489015 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1313/2402] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:38 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:38.489240 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7009dd8f-b3da-4ec4-a42f-da03ac61cb55 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:38.489340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e13ae5bb-962e-4ac7-8037-128959450bd1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:38.489457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7009dd8f-b3da-4ec4-a42f-da03ac61cb55 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:38.489553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e13ae5bb-962e-4ac7-8037-128959450bd1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:38.489640 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7009dd8f-b3da-4ec4-a42f-da03ac61cb55 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:38.489731 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7009dd8f-b3da-4ec4-a42f-da03ac61cb55 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:38.490121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7009dd8f-b3da-4ec4-a42f-da03ac61cb55 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:38.490250 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e13ae5bb-962e-4ac7-8037-128959450bd1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:38.490668 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e13ae5bb-962e-4ac7-8037-128959450bd1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:38.490819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7009dd8f-b3da-4ec4-a42f-da03ac61cb55 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:38.491101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e13ae5bb-962e-4ac7-8037-128959450bd1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:38.491238 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7009dd8f-b3da-4ec4-a42f-da03ac61cb55 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:38.491629 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7009dd8f-b3da-4ec4-a42f-da03ac61cb55 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:38.491792 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e13ae5bb-962e-4ac7-8037-128959450bd1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:38.492238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e13ae5bb-962e-4ac7-8037-128959450bd1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:38.492659 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e13ae5bb-962e-4ac7-8037-128959450bd1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:38.575987 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7009dd8f-b3da-4ec4-a42f-da03ac61cb55 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:38.577607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e13ae5bb-962e-4ac7-8037-128959450bd1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:38.584733 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7009dd8f-b3da-4ec4-a42f-da03ac61cb55 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.106153 Aug 28 01:21:38.585330 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1090/2403] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:38 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:38.585901 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e13ae5bb-962e-4ac7-8037-128959450bd1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.106621 Aug 28 01:21:38.586464 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1314/2404] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:38 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:39.499796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a8e86b7c-6596-4d68-96a4-75eb69acc523 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:39.500419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a8e86b7c-6596-4d68-96a4-75eb69acc523 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:39.501976 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-91917dec-5050-4a79-b18a-b67cd973fa66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:39.502562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-91917dec-5050-4a79-b18a-b67cd973fa66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:39.507343 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a8e86b7c-6596-4d68-96a4-75eb69acc523 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:39.507847 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a8e86b7c-6596-4d68-96a4-75eb69acc523 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:39.508266 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a8e86b7c-6596-4d68-96a4-75eb69acc523 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:39.508944 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a8e86b7c-6596-4d68-96a4-75eb69acc523 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:39.509137 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91917dec-5050-4a79-b18a-b67cd973fa66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:39.509425 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a8e86b7c-6596-4d68-96a4-75eb69acc523 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:39.509611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91917dec-5050-4a79-b18a-b67cd973fa66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:39.509844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a8e86b7c-6596-4d68-96a4-75eb69acc523 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:39.510036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91917dec-5050-4a79-b18a-b67cd973fa66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:39.510716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91917dec-5050-4a79-b18a-b67cd973fa66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:39.511252 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91917dec-5050-4a79-b18a-b67cd973fa66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:39.511694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91917dec-5050-4a79-b18a-b67cd973fa66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:39.704032 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a8e86b7c-6596-4d68-96a4-75eb69acc523 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:39.708524 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-91917dec-5050-4a79-b18a-b67cd973fa66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:39.715527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a8e86b7c-6596-4d68-96a4-75eb69acc523 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.219961 Aug 28 01:21:39.716098 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1091/2405] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:39 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:39.719864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-91917dec-5050-4a79-b18a-b67cd973fa66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:39.720515 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-38fbf705-82ef-42d7-a0a7-812853ffac81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:39.721116 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-38fbf705-82ef-42d7-a0a7-812853ffac81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:39.721626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-91917dec-5050-4a79-b18a-b67cd973fa66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:39.722887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-91917dec-5050-4a79-b18a-b67cd973fa66 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.224680 Aug 28 01:21:39.729045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1315/2406] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:39 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:39.729277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38fbf705-82ef-42d7-a0a7-812853ffac81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:39.729277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38fbf705-82ef-42d7-a0a7-812853ffac81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:39.729528 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5c5d7867-f397-4e11-be8d-c60af387f92b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:39.729655 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38fbf705-82ef-42d7-a0a7-812853ffac81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:39.729759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5c5d7867-f397-4e11-be8d-c60af387f92b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:39.729869 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38fbf705-82ef-42d7-a0a7-812853ffac81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:39.729869 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38fbf705-82ef-42d7-a0a7-812853ffac81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:39.729869 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38fbf705-82ef-42d7-a0a7-812853ffac81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:39.735487 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5c5d7867-f397-4e11-be8d-c60af387f92b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:39.736262 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5c5d7867-f397-4e11-be8d-c60af387f92b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:39.736752 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5c5d7867-f397-4e11-be8d-c60af387f92b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:39.737545 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5c5d7867-f397-4e11-be8d-c60af387f92b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:39.738058 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5c5d7867-f397-4e11-be8d-c60af387f92b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:39.738539 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5c5d7867-f397-4e11-be8d-c60af387f92b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:39.811180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-38fbf705-82ef-42d7-a0a7-812853ffac81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:39.814045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5c5d7867-f397-4e11-be8d-c60af387f92b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:39.824303 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-38fbf705-82ef-42d7-a0a7-812853ffac81 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.106844 Aug 28 01:21:39.825109 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1092/2407] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:39 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:39.826590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5c5d7867-f397-4e11-be8d-c60af387f92b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.101755 Aug 28 01:21:39.827484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1316/2408] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:39 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:40.739964 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-81a58d22-605a-4165-a689-0cc1cca79d37 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:40.742365 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-81a58d22-605a-4165-a689-0cc1cca79d37 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:40.747876 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-81a58d22-605a-4165-a689-0cc1cca79d37 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:40.748092 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e38cc862-7516-4f57-8bec-3a69791bce5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:40.748557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-81a58d22-605a-4165-a689-0cc1cca79d37 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:40.748680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e38cc862-7516-4f57-8bec-3a69791bce5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:40.748779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-81a58d22-605a-4165-a689-0cc1cca79d37 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:40.749502 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-81a58d22-605a-4165-a689-0cc1cca79d37 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:40.749935 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-81a58d22-605a-4165-a689-0cc1cca79d37 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:40.750392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-81a58d22-605a-4165-a689-0cc1cca79d37 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:40.755380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e38cc862-7516-4f57-8bec-3a69791bce5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:40.755855 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e38cc862-7516-4f57-8bec-3a69791bce5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:40.756279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e38cc862-7516-4f57-8bec-3a69791bce5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:40.756984 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e38cc862-7516-4f57-8bec-3a69791bce5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:40.757446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e38cc862-7516-4f57-8bec-3a69791bce5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:40.757859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e38cc862-7516-4f57-8bec-3a69791bce5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:40.946492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-81a58d22-605a-4165-a689-0cc1cca79d37 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:40.950166 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e38cc862-7516-4f57-8bec-3a69791bce5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:40.955536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-81a58d22-605a-4165-a689-0cc1cca79d37 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.221757 Aug 28 01:21:40.956174 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1093/2409] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:40 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:40.959564 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e38cc862-7516-4f57-8bec-3a69791bce5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:40.960706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-90af0b30-7711-4e14-9c03-7cfd7698ddcb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:40.961100 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e38cc862-7516-4f57-8bec-3a69791bce5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:40.961304 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-90af0b30-7711-4e14-9c03-7cfd7698ddcb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:40.962253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e38cc862-7516-4f57-8bec-3a69791bce5f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.218240 Aug 28 01:21:40.962756 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1317/2410] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:40 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:40.967555 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90af0b30-7711-4e14-9c03-7cfd7698ddcb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:40.968040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90af0b30-7711-4e14-9c03-7cfd7698ddcb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:40.968463 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90af0b30-7711-4e14-9c03-7cfd7698ddcb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:40.969180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90af0b30-7711-4e14-9c03-7cfd7698ddcb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:40.969619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90af0b30-7711-4e14-9c03-7cfd7698ddcb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:40.970028 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90af0b30-7711-4e14-9c03-7cfd7698ddcb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:40.972597 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:21:40.976158 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2fb51321-020f-4053-8fbb-7b93b230a29a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:40.976789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2fb51321-020f-4053-8fbb-7b93b230a29a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:40.982784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2fb51321-020f-4053-8fbb-7b93b230a29a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:40.983270 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2fb51321-020f-4053-8fbb-7b93b230a29a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:40.983710 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2fb51321-020f-4053-8fbb-7b93b230a29a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:40.984386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2fb51321-020f-4053-8fbb-7b93b230a29a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:40.984836 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2fb51321-020f-4053-8fbb-7b93b230a29a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:40.985298 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2fb51321-020f-4053-8fbb-7b93b230a29a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:41.038852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-90af0b30-7711-4e14-9c03-7cfd7698ddcb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:41.047215 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-90af0b30-7711-4e14-9c03-7cfd7698ddcb tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.090045 Aug 28 01:21:41.048003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1094/2411] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:40 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:41.065607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2fb51321-020f-4053-8fbb-7b93b230a29a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:41.074387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2fb51321-020f-4053-8fbb-7b93b230a29a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.110468 Aug 28 01:21:41.074954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1318/2412] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:40 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:41.980070 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-04337565-d59c-49e9-a6e7-7ac6636f12b1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:41.980863 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-04337565-d59c-49e9-a6e7-7ac6636f12b1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:41.988453 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0e715c90-aa36-44de-935e-981a43cebe8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:41.989526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0e715c90-aa36-44de-935e-981a43cebe8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:41.989949 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04337565-d59c-49e9-a6e7-7ac6636f12b1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:41.990628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04337565-d59c-49e9-a6e7-7ac6636f12b1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:41.991189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04337565-d59c-49e9-a6e7-7ac6636f12b1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:41.992115 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04337565-d59c-49e9-a6e7-7ac6636f12b1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:41.992723 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04337565-d59c-49e9-a6e7-7ac6636f12b1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:41.993305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04337565-d59c-49e9-a6e7-7ac6636f12b1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:41.997476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e715c90-aa36-44de-935e-981a43cebe8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:41.997955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e715c90-aa36-44de-935e-981a43cebe8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:41.998388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e715c90-aa36-44de-935e-981a43cebe8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:41.999103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e715c90-aa36-44de-935e-981a43cebe8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:41.999691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e715c90-aa36-44de-935e-981a43cebe8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:42.000116 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e715c90-aa36-44de-935e-981a43cebe8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:42.192798 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-04337565-d59c-49e9-a6e7-7ac6636f12b1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:42.195877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0e715c90-aa36-44de-935e-981a43cebe8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:42.201971 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-04337565-d59c-49e9-a6e7-7ac6636f12b1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.227157 Aug 28 01:21:42.202550 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1095/2413] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:41 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:42.205234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0e715c90-aa36-44de-935e-981a43cebe8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:42.206654 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0e715c90-aa36-44de-935e-981a43cebe8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:42.206895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7555c251-8aea-4416-b8b7-befa33475473 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:42.207459 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7555c251-8aea-4416-b8b7-befa33475473 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:42.207813 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0e715c90-aa36-44de-935e-981a43cebe8f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.224656 Aug 28 01:21:42.208537 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1319/2414] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:41 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 01:21:42.213933 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7555c251-8aea-4416-b8b7-befa33475473 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:42.215468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7555c251-8aea-4416-b8b7-befa33475473 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:42.216160 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7555c251-8aea-4416-b8b7-befa33475473 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:42.217147 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7555c251-8aea-4416-b8b7-befa33475473 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:42.217728 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7555c251-8aea-4416-b8b7-befa33475473 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:42.218601 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7555c251-8aea-4416-b8b7-befa33475473 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:42.284700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7555c251-8aea-4416-b8b7-befa33475473 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:42.293128 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7555c251-8aea-4416-b8b7-befa33475473 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.089638 Aug 28 01:21:42.293693 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1096/2415] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:42 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:42.349121 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-079b948e-7db9-4211-9784-8d81d9d76bb8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:42.349726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-079b948e-7db9-4211-9784-8d81d9d76bb8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:42.355812 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-079b948e-7db9-4211-9784-8d81d9d76bb8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:42.356296 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-079b948e-7db9-4211-9784-8d81d9d76bb8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:42.356741 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-079b948e-7db9-4211-9784-8d81d9d76bb8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:42.357450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-079b948e-7db9-4211-9784-8d81d9d76bb8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:42.357891 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-079b948e-7db9-4211-9784-8d81d9d76bb8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:42.358301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-079b948e-7db9-4211-9784-8d81d9d76bb8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:42.432441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-079b948e-7db9-4211-9784-8d81d9d76bb8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:42.443265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-079b948e-7db9-4211-9784-8d81d9d76bb8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.233991 Aug 28 01:21:42.443819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1320/2416] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:42 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:43.225458 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fb5018f3-3f5b-4e40-b7d3-f85d263cfd66 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:43.226129 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fb5018f3-3f5b-4e40-b7d3-f85d263cfd66 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:43.233097 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fb5018f3-3f5b-4e40-b7d3-f85d263cfd66 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:43.233588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fb5018f3-3f5b-4e40-b7d3-f85d263cfd66 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:43.234028 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fb5018f3-3f5b-4e40-b7d3-f85d263cfd66 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:43.234699 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fb5018f3-3f5b-4e40-b7d3-f85d263cfd66 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:43.235138 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fb5018f3-3f5b-4e40-b7d3-f85d263cfd66 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:43.235552 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fb5018f3-3f5b-4e40-b7d3-f85d263cfd66 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:43.371311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fb5018f3-3f5b-4e40-b7d3-f85d263cfd66 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:43.379776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8ea4f16a-39ac-49f2-89fb-32c1191fb852 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:43.380776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8ea4f16a-39ac-49f2-89fb-32c1191fb852 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:43.383097 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fb5018f3-3f5b-4e40-b7d3-f85d263cfd66 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.161253 Aug 28 01:21:43.384104 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1097/2417] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:43 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:43.390579 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8ea4f16a-39ac-49f2-89fb-32c1191fb852 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:43.391346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8ea4f16a-39ac-49f2-89fb-32c1191fb852 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:43.391999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-620ed086-30e9-42de-ae88-3ea7eb786988 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:43.392138 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8ea4f16a-39ac-49f2-89fb-32c1191fb852 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:43.392999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-620ed086-30e9-42de-ae88-3ea7eb786988 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:43.393145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8ea4f16a-39ac-49f2-89fb-32c1191fb852 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:43.393814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8ea4f16a-39ac-49f2-89fb-32c1191fb852 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:43.394477 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8ea4f16a-39ac-49f2-89fb-32c1191fb852 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:43.402187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-620ed086-30e9-42de-ae88-3ea7eb786988 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:43.402956 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-620ed086-30e9-42de-ae88-3ea7eb786988 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:43.403648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-620ed086-30e9-42de-ae88-3ea7eb786988 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:43.404762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-620ed086-30e9-42de-ae88-3ea7eb786988 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:43.405519 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-620ed086-30e9-42de-ae88-3ea7eb786988 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:43.406196 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-620ed086-30e9-42de-ae88-3ea7eb786988 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:43.502864 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-620ed086-30e9-42de-ae88-3ea7eb786988 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:43.511869 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-620ed086-30e9-42de-ae88-3ea7eb786988 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.126310 Aug 28 01:21:43.512438 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1098/2418] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:43 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:43.516899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5b6da7b3-6e02-4785-bcc7-1aaf73ce5c01 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:43.517505 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5b6da7b3-6e02-4785-bcc7-1aaf73ce5c01 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:43.525920 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b6da7b3-6e02-4785-bcc7-1aaf73ce5c01 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:43.526538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b6da7b3-6e02-4785-bcc7-1aaf73ce5c01 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:43.526953 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b6da7b3-6e02-4785-bcc7-1aaf73ce5c01 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:43.527652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b6da7b3-6e02-4785-bcc7-1aaf73ce5c01 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:43.528127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b6da7b3-6e02-4785-bcc7-1aaf73ce5c01 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:43.528574 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5b6da7b3-6e02-4785-bcc7-1aaf73ce5c01 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:43.580030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8ea4f16a-39ac-49f2-89fb-32c1191fb852 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:43.589464 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8ea4f16a-39ac-49f2-89fb-32c1191fb852 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:43.590994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8ea4f16a-39ac-49f2-89fb-32c1191fb852 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:43.592242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8ea4f16a-39ac-49f2-89fb-32c1191fb852 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.362179 Aug 28 01:21:43.593165 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1321/2419] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:43 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:43.595938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5b6da7b3-6e02-4785-bcc7-1aaf73ce5c01 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:43.604588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5b6da7b3-6e02-4785-bcc7-1aaf73ce5c01 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.091210 Aug 28 01:21:43.607665 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1099/2420] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:43 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:44.406795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bd84958a-fef0-47c7-bccc-a604c353f96a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:44.407426 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bd84958a-fef0-47c7-bccc-a604c353f96a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:44.417107 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bd84958a-fef0-47c7-bccc-a604c353f96a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:44.417626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bd84958a-fef0-47c7-bccc-a604c353f96a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:44.418057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bd84958a-fef0-47c7-bccc-a604c353f96a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:44.418748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bd84958a-fef0-47c7-bccc-a604c353f96a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:44.419189 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bd84958a-fef0-47c7-bccc-a604c353f96a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:44.419603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bd84958a-fef0-47c7-bccc-a604c353f96a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:44.535708 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-09041833-a242-4993-b34a-19de63ab1c32 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:44.536310 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-09041833-a242-4993-b34a-19de63ab1c32 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:44.543899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-09041833-a242-4993-b34a-19de63ab1c32 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:44.544380 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-09041833-a242-4993-b34a-19de63ab1c32 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:44.544833 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-09041833-a242-4993-b34a-19de63ab1c32 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:44.545535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-09041833-a242-4993-b34a-19de63ab1c32 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:44.545982 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-09041833-a242-4993-b34a-19de63ab1c32 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:44.546385 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-09041833-a242-4993-b34a-19de63ab1c32 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:44.569458 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bd84958a-fef0-47c7-bccc-a604c353f96a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:44.577961 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bd84958a-fef0-47c7-bccc-a604c353f96a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.176642 Aug 28 01:21:44.578642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1322/2421] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:44 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:44.611666 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-09041833-a242-4993-b34a-19de63ab1c32 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:44.615422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5e9e21ae-f759-4faa-a244-faf39f3cd955 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:44.616021 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5e9e21ae-f759-4faa-a244-faf39f3cd955 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:44.620403 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-09041833-a242-4993-b34a-19de63ab1c32 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.088491 Aug 28 01:21:44.621090 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1100/2422] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:44 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:44.622184 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e9e21ae-f759-4faa-a244-faf39f3cd955 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:44.622654 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e9e21ae-f759-4faa-a244-faf39f3cd955 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:44.623081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e9e21ae-f759-4faa-a244-faf39f3cd955 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:44.623759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e9e21ae-f759-4faa-a244-faf39f3cd955 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:44.624204 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e9e21ae-f759-4faa-a244-faf39f3cd955 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:44.624610 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e9e21ae-f759-4faa-a244-faf39f3cd955 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:44.636806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d48f92a5-c780-4d0b-9edf-f329e4133599 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:44.637439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d48f92a5-c780-4d0b-9edf-f329e4133599 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:44.640325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:21:44.644536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d48f92a5-c780-4d0b-9edf-f329e4133599 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:44.645051 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d48f92a5-c780-4d0b-9edf-f329e4133599 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:44.645483 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d48f92a5-c780-4d0b-9edf-f329e4133599 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:44.646186 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d48f92a5-c780-4d0b-9edf-f329e4133599 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:44.646646 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d48f92a5-c780-4d0b-9edf-f329e4133599 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:44.647066 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d48f92a5-c780-4d0b-9edf-f329e4133599 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:44.720690 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d48f92a5-c780-4d0b-9edf-f329e4133599 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:44.729713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d48f92a5-c780-4d0b-9edf-f329e4133599 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.103698 Aug 28 01:21:44.732438 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1101/2423] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:44 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:44.785468 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5e9e21ae-f759-4faa-a244-faf39f3cd955 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:44.798736 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5e9e21ae-f759-4faa-a244-faf39f3cd955 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:44.801180 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5e9e21ae-f759-4faa-a244-faf39f3cd955 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:44.803035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5e9e21ae-f759-4faa-a244-faf39f3cd955 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.191156 Aug 28 01:21:44.803934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1323/2424] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:44 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 01:21:45.603137 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8e07dd09-77d5-4578-884b-7309c04d6c9d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:45.604127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8e07dd09-77d5-4578-884b-7309c04d6c9d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:45.614867 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e07dd09-77d5-4578-884b-7309c04d6c9d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:45.615659 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e07dd09-77d5-4578-884b-7309c04d6c9d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:45.616367 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e07dd09-77d5-4578-884b-7309c04d6c9d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:45.617547 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e07dd09-77d5-4578-884b-7309c04d6c9d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:45.618293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e07dd09-77d5-4578-884b-7309c04d6c9d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:45.618989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e07dd09-77d5-4578-884b-7309c04d6c9d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:45.642278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-055f70cf-7a1d-405e-ab38-0dbe1a9b9e12 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:45.642960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-055f70cf-7a1d-405e-ab38-0dbe1a9b9e12 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:45.650569 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-055f70cf-7a1d-405e-ab38-0dbe1a9b9e12 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:45.651027 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-055f70cf-7a1d-405e-ab38-0dbe1a9b9e12 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:45.651456 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-055f70cf-7a1d-405e-ab38-0dbe1a9b9e12 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:45.652147 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-055f70cf-7a1d-405e-ab38-0dbe1a9b9e12 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:45.652588 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-055f70cf-7a1d-405e-ab38-0dbe1a9b9e12 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:45.653098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-055f70cf-7a1d-405e-ab38-0dbe1a9b9e12 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:45.722834 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-055f70cf-7a1d-405e-ab38-0dbe1a9b9e12 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:45.731962 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-055f70cf-7a1d-405e-ab38-0dbe1a9b9e12 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.093450 Aug 28 01:21:45.732798 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1324/2425] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:45 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:45.751724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4e36afed-dc02-4058-9656-fdf8c5422a83 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:45.752326 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4e36afed-dc02-4058-9656-fdf8c5422a83 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:45.758639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4e36afed-dc02-4058-9656-fdf8c5422a83 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:45.759118 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4e36afed-dc02-4058-9656-fdf8c5422a83 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:45.759549 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4e36afed-dc02-4058-9656-fdf8c5422a83 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:45.760217 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4e36afed-dc02-4058-9656-fdf8c5422a83 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:45.760817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4e36afed-dc02-4058-9656-fdf8c5422a83 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:45.761268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4e36afed-dc02-4058-9656-fdf8c5422a83 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:45.781642 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8e07dd09-77d5-4578-884b-7309c04d6c9d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:45.789970 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8e07dd09-77d5-4578-884b-7309c04d6c9d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.193520 Aug 28 01:21:45.790562 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1102/2426] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:45 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:45.823869 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4e36afed-dc02-4058-9656-fdf8c5422a83 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:45.832793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4e36afed-dc02-4058-9656-fdf8c5422a83 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.084833 Aug 28 01:21:45.835425 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-51af246f-fd30-4360-bb26-dc887d1c8cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:45.836251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-51af246f-fd30-4360-bb26-dc887d1c8cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:45.842253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1325/2427] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:45 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:45.842876 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-51af246f-fd30-4360-bb26-dc887d1c8cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:45.843562 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-51af246f-fd30-4360-bb26-dc887d1c8cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:45.844016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-51af246f-fd30-4360-bb26-dc887d1c8cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:45.844715 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-51af246f-fd30-4360-bb26-dc887d1c8cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:45.845273 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-51af246f-fd30-4360-bb26-dc887d1c8cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:45.845697 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-51af246f-fd30-4360-bb26-dc887d1c8cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:45.986309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-51af246f-fd30-4360-bb26-dc887d1c8cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:45.995528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-51af246f-fd30-4360-bb26-dc887d1c8cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:45.996977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-51af246f-fd30-4360-bb26-dc887d1c8cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:45.998155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-51af246f-fd30-4360-bb26-dc887d1c8cf1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.166726 Aug 28 01:21:45.998720 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1103/2428] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:45 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:46.752278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a9b4d784-0144-48cb-981f-fb01548f7305 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:46.752854 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a9b4d784-0144-48cb-981f-fb01548f7305 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:46.762033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9b4d784-0144-48cb-981f-fb01548f7305 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:46.762959 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9b4d784-0144-48cb-981f-fb01548f7305 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:46.763748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9b4d784-0144-48cb-981f-fb01548f7305 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:46.764999 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9b4d784-0144-48cb-981f-fb01548f7305 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:46.765892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9b4d784-0144-48cb-981f-fb01548f7305 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:46.766743 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a9b4d784-0144-48cb-981f-fb01548f7305 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:46.820011 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7af34307-8c33-4675-8f2d-275cb93508b1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:46.820011 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7af34307-8c33-4675-8f2d-275cb93508b1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:46.826946 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7af34307-8c33-4675-8f2d-275cb93508b1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:46.827431 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7af34307-8c33-4675-8f2d-275cb93508b1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:46.827857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7af34307-8c33-4675-8f2d-275cb93508b1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:46.828544 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7af34307-8c33-4675-8f2d-275cb93508b1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:46.828989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7af34307-8c33-4675-8f2d-275cb93508b1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:46.829430 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7af34307-8c33-4675-8f2d-275cb93508b1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:46.841888 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a9b4d784-0144-48cb-981f-fb01548f7305 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:46.851301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a9b4d784-0144-48cb-981f-fb01548f7305 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.102983 Aug 28 01:21:46.856266 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1326/2429] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:46 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:46.856665 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bc1561fd-328a-492e-8c9b-2f4efdd19928 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:46.857282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bc1561fd-328a-492e-8c9b-2f4efdd19928 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:46.863648 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc1561fd-328a-492e-8c9b-2f4efdd19928 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:46.864206 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc1561fd-328a-492e-8c9b-2f4efdd19928 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:46.864709 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc1561fd-328a-492e-8c9b-2f4efdd19928 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:46.865579 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc1561fd-328a-492e-8c9b-2f4efdd19928 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:46.866130 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc1561fd-328a-492e-8c9b-2f4efdd19928 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:46.866523 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc1561fd-328a-492e-8c9b-2f4efdd19928 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:46.945771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bc1561fd-328a-492e-8c9b-2f4efdd19928 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:46.954552 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bc1561fd-328a-492e-8c9b-2f4efdd19928 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.101632 Aug 28 01:21:46.955426 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1327/2430] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:46 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:46.981302 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7af34307-8c33-4675-8f2d-275cb93508b1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:46.989998 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7af34307-8c33-4675-8f2d-275cb93508b1 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.181251 Aug 28 01:21:46.990553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1104/2431] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:46 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:47.021547 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a21747b0-d1d5-43ec-8fa6-9a85c1eda9ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:47.022065 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a21747b0-d1d5-43ec-8fa6-9a85c1eda9ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:47.028319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a21747b0-d1d5-43ec-8fa6-9a85c1eda9ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:47.028805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a21747b0-d1d5-43ec-8fa6-9a85c1eda9ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:47.029260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a21747b0-d1d5-43ec-8fa6-9a85c1eda9ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:47.029916 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a21747b0-d1d5-43ec-8fa6-9a85c1eda9ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:47.030354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a21747b0-d1d5-43ec-8fa6-9a85c1eda9ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:47.030767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a21747b0-d1d5-43ec-8fa6-9a85c1eda9ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:47.168108 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a21747b0-d1d5-43ec-8fa6-9a85c1eda9ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:47.181956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a21747b0-d1d5-43ec-8fa6-9a85c1eda9ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:47.184161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a21747b0-d1d5-43ec-8fa6-9a85c1eda9ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:47.185655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a21747b0-d1d5-43ec-8fa6-9a85c1eda9ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.168140 Aug 28 01:21:47.186241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1328/2432] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:47 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:47.872772 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a0ca4268-e98a-4ea5-af13-84a42cc4151d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:47.873380 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a0ca4268-e98a-4ea5-af13-84a42cc4151d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:47.880177 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a0ca4268-e98a-4ea5-af13-84a42cc4151d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:47.880656 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a0ca4268-e98a-4ea5-af13-84a42cc4151d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:47.881121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a0ca4268-e98a-4ea5-af13-84a42cc4151d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:47.881802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a0ca4268-e98a-4ea5-af13-84a42cc4151d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:47.882245 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a0ca4268-e98a-4ea5-af13-84a42cc4151d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:47.882655 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a0ca4268-e98a-4ea5-af13-84a42cc4151d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:47.951305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a0ca4268-e98a-4ea5-af13-84a42cc4151d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:47.959729 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a0ca4268-e98a-4ea5-af13-84a42cc4151d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.091403 Aug 28 01:21:47.960291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1105/2433] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:47 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:47.975903 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-81623a74-5409-4627-9a11-c5364eca6014 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:47.976477 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-81623a74-5409-4627-9a11-c5364eca6014 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:47.983217 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81623a74-5409-4627-9a11-c5364eca6014 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:47.983700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81623a74-5409-4627-9a11-c5364eca6014 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:47.984133 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81623a74-5409-4627-9a11-c5364eca6014 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:47.984825 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81623a74-5409-4627-9a11-c5364eca6014 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:47.985303 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81623a74-5409-4627-9a11-c5364eca6014 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:47.985720 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81623a74-5409-4627-9a11-c5364eca6014 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:48.011683 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-df48a7af-5148-4796-94fa-e8c2a375aff8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:48.012284 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-df48a7af-5148-4796-94fa-e8c2a375aff8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:48.018449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-df48a7af-5148-4796-94fa-e8c2a375aff8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:48.018928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-df48a7af-5148-4796-94fa-e8c2a375aff8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:48.019350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-df48a7af-5148-4796-94fa-e8c2a375aff8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:48.020040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-df48a7af-5148-4796-94fa-e8c2a375aff8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:48.020480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-df48a7af-5148-4796-94fa-e8c2a375aff8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:48.020893 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-df48a7af-5148-4796-94fa-e8c2a375aff8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:48.048844 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-81623a74-5409-4627-9a11-c5364eca6014 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:48.057357 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-81623a74-5409-4627-9a11-c5364eca6014 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.085229 Aug 28 01:21:48.057923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1329/2434] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:47 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:48.155058 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-df48a7af-5148-4796-94fa-e8c2a375aff8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:48.164348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-df48a7af-5148-4796-94fa-e8c2a375aff8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.156490 Aug 28 01:21:48.164926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1106/2435] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:48 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:48.212353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-87a55e93-d372-4d3c-9185-73d49ad0e921 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:48.213150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-87a55e93-d372-4d3c-9185-73d49ad0e921 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:48.220735 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87a55e93-d372-4d3c-9185-73d49ad0e921 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:48.221261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87a55e93-d372-4d3c-9185-73d49ad0e921 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:48.221714 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87a55e93-d372-4d3c-9185-73d49ad0e921 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:48.222407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87a55e93-d372-4d3c-9185-73d49ad0e921 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:48.222853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87a55e93-d372-4d3c-9185-73d49ad0e921 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:48.223334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-87a55e93-d372-4d3c-9185-73d49ad0e921 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:48.360582 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-87a55e93-d372-4d3c-9185-73d49ad0e921 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:48.374513 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-87a55e93-d372-4d3c-9185-73d49ad0e921 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:48.377376 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-87a55e93-d372-4d3c-9185-73d49ad0e921 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:48.379643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-87a55e93-d372-4d3c-9185-73d49ad0e921 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.173581 Aug 28 01:21:48.380429 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1330/2436] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:48 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:48.981406 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-819942d4-fd5f-4097-a483-c435723478ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:48.982066 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-819942d4-fd5f-4097-a483-c435723478ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:48.988866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-819942d4-fd5f-4097-a483-c435723478ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:48.989380 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-819942d4-fd5f-4097-a483-c435723478ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:48.989806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-819942d4-fd5f-4097-a483-c435723478ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:48.990491 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-819942d4-fd5f-4097-a483-c435723478ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:48.990936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-819942d4-fd5f-4097-a483-c435723478ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:48.991346 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-819942d4-fd5f-4097-a483-c435723478ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:49.053611 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-819942d4-fd5f-4097-a483-c435723478ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:49.062175 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-819942d4-fd5f-4097-a483-c435723478ad tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.085212 Aug 28 01:21:49.062733 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1107/2437] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:48 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:49.079347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-26b2bd91-b4db-4d91-bb50-1a67f5b5cec7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:49.079961 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-26b2bd91-b4db-4d91-bb50-1a67f5b5cec7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:49.086555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26b2bd91-b4db-4d91-bb50-1a67f5b5cec7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:49.087031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26b2bd91-b4db-4d91-bb50-1a67f5b5cec7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:49.087466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26b2bd91-b4db-4d91-bb50-1a67f5b5cec7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:49.088154 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26b2bd91-b4db-4d91-bb50-1a67f5b5cec7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:49.088601 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26b2bd91-b4db-4d91-bb50-1a67f5b5cec7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:49.089046 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26b2bd91-b4db-4d91-bb50-1a67f5b5cec7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:49.154503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-26b2bd91-b4db-4d91-bb50-1a67f5b5cec7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:49.166119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-26b2bd91-b4db-4d91-bb50-1a67f5b5cec7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.090687 Aug 28 01:21:49.166907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1331/2438] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:49 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:21:49.185833 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a8fab483-931c-4581-9401-a1d423fa7b92 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:49.186435 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a8fab483-931c-4581-9401-a1d423fa7b92 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:49.194008 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a8fab483-931c-4581-9401-a1d423fa7b92 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:49.194499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a8fab483-931c-4581-9401-a1d423fa7b92 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:49.194928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a8fab483-931c-4581-9401-a1d423fa7b92 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:49.195624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a8fab483-931c-4581-9401-a1d423fa7b92 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:49.196069 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a8fab483-931c-4581-9401-a1d423fa7b92 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:49.196468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a8fab483-931c-4581-9401-a1d423fa7b92 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:49.332787 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a8fab483-931c-4581-9401-a1d423fa7b92 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:49.341714 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a8fab483-931c-4581-9401-a1d423fa7b92 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.159881 Aug 28 01:21:49.342473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1108/2439] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:49 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:49.406635 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ad19c7c6-e438-445b-a561-32b99bfaaa76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:49.407291 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ad19c7c6-e438-445b-a561-32b99bfaaa76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:49.413985 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ad19c7c6-e438-445b-a561-32b99bfaaa76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:49.414465 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ad19c7c6-e438-445b-a561-32b99bfaaa76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:49.414895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ad19c7c6-e438-445b-a561-32b99bfaaa76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:49.415581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ad19c7c6-e438-445b-a561-32b99bfaaa76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:49.416583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ad19c7c6-e438-445b-a561-32b99bfaaa76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:49.416583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ad19c7c6-e438-445b-a561-32b99bfaaa76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:49.567079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ad19c7c6-e438-445b-a561-32b99bfaaa76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:49.578931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ad19c7c6-e438-445b-a561-32b99bfaaa76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:49.580352 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ad19c7c6-e438-445b-a561-32b99bfaaa76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:49.581547 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ad19c7c6-e438-445b-a561-32b99bfaaa76 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.182057 Aug 28 01:21:49.582102 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1332/2440] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:49 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:50.083532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c91d5952-dd6d-41fc-9119-04307e40a188 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:50.084173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c91d5952-dd6d-41fc-9119-04307e40a188 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:50.091594 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c91d5952-dd6d-41fc-9119-04307e40a188 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:50.092078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c91d5952-dd6d-41fc-9119-04307e40a188 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:50.092508 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c91d5952-dd6d-41fc-9119-04307e40a188 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:50.093233 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c91d5952-dd6d-41fc-9119-04307e40a188 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:50.093671 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c91d5952-dd6d-41fc-9119-04307e40a188 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:50.094115 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c91d5952-dd6d-41fc-9119-04307e40a188 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:50.160822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c91d5952-dd6d-41fc-9119-04307e40a188 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:50.169264 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c91d5952-dd6d-41fc-9119-04307e40a188 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.089650 Aug 28 01:21:50.169829 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1109/2441] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:50 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:50.187786 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-58c3c4d8-d3a2-401e-b142-cb54b00265c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:50.188366 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-58c3c4d8-d3a2-401e-b142-cb54b00265c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:50.195316 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-58c3c4d8-d3a2-401e-b142-cb54b00265c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:50.195789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-58c3c4d8-d3a2-401e-b142-cb54b00265c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:50.196215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-58c3c4d8-d3a2-401e-b142-cb54b00265c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:50.196885 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-58c3c4d8-d3a2-401e-b142-cb54b00265c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:50.197368 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-58c3c4d8-d3a2-401e-b142-cb54b00265c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:50.197783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-58c3c4d8-d3a2-401e-b142-cb54b00265c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:50.266959 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-58c3c4d8-d3a2-401e-b142-cb54b00265c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:50.279135 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-58c3c4d8-d3a2-401e-b142-cb54b00265c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.095270 Aug 28 01:21:50.279708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1333/2442] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:50 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:50.363384 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a40329de-5147-4d52-bc6f-7942c3279a68 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:50.363901 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a40329de-5147-4d52-bc6f-7942c3279a68 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:50.370684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a40329de-5147-4d52-bc6f-7942c3279a68 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:50.371172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a40329de-5147-4d52-bc6f-7942c3279a68 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:50.371596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a40329de-5147-4d52-bc6f-7942c3279a68 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:50.372398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a40329de-5147-4d52-bc6f-7942c3279a68 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:50.372849 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a40329de-5147-4d52-bc6f-7942c3279a68 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:50.373304 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a40329de-5147-4d52-bc6f-7942c3279a68 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:50.510606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a40329de-5147-4d52-bc6f-7942c3279a68 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:50.512893 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:21:50.520032 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a40329de-5147-4d52-bc6f-7942c3279a68 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.160796 Aug 28 01:21:50.520611 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1110/2443] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:50 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:50.615397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b27571bf-683e-4330-99b5-190bdc4b1877 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:50.616170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b27571bf-683e-4330-99b5-190bdc4b1877 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:50.625244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b27571bf-683e-4330-99b5-190bdc4b1877 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:50.625894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b27571bf-683e-4330-99b5-190bdc4b1877 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:50.626481 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b27571bf-683e-4330-99b5-190bdc4b1877 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:50.627421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b27571bf-683e-4330-99b5-190bdc4b1877 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:50.628045 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b27571bf-683e-4330-99b5-190bdc4b1877 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:50.628634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b27571bf-683e-4330-99b5-190bdc4b1877 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:50.768535 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b27571bf-683e-4330-99b5-190bdc4b1877 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:50.780584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b27571bf-683e-4330-99b5-190bdc4b1877 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:50.782044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b27571bf-683e-4330-99b5-190bdc4b1877 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:50.783177 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b27571bf-683e-4330-99b5-190bdc4b1877 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.173302 Aug 28 01:21:50.783746 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1334/2444] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:50 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:51.194325 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-36eeb159-4f80-43b9-97ae-2bf40f10de4b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:51.195234 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-36eeb159-4f80-43b9-97ae-2bf40f10de4b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:51.204140 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36eeb159-4f80-43b9-97ae-2bf40f10de4b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:51.204634 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36eeb159-4f80-43b9-97ae-2bf40f10de4b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:51.205107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36eeb159-4f80-43b9-97ae-2bf40f10de4b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:51.205811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36eeb159-4f80-43b9-97ae-2bf40f10de4b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:51.206255 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36eeb159-4f80-43b9-97ae-2bf40f10de4b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:51.206678 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36eeb159-4f80-43b9-97ae-2bf40f10de4b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:51.274824 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-36eeb159-4f80-43b9-97ae-2bf40f10de4b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:51.283549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-36eeb159-4f80-43b9-97ae-2bf40f10de4b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.095952 Aug 28 01:21:51.284134 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1111/2445] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:51 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:51.299837 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-383af14f-c8c9-495c-835d-0d0aab347fb0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:51.300439 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-383af14f-c8c9-495c-835d-0d0aab347fb0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:51.307105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-383af14f-c8c9-495c-835d-0d0aab347fb0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:51.307588 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-383af14f-c8c9-495c-835d-0d0aab347fb0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:51.308006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-383af14f-c8c9-495c-835d-0d0aab347fb0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:51.308679 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-383af14f-c8c9-495c-835d-0d0aab347fb0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:51.309166 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-383af14f-c8c9-495c-835d-0d0aab347fb0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:51.309574 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-383af14f-c8c9-495c-835d-0d0aab347fb0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:51.372882 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-383af14f-c8c9-495c-835d-0d0aab347fb0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:51.381887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-383af14f-c8c9-495c-835d-0d0aab347fb0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.085883 Aug 28 01:21:51.382496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1335/2446] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:51 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:51.541788 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c9053f7c-b332-4dd2-af6e-c705bc2e65e0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:51.542301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c9053f7c-b332-4dd2-af6e-c705bc2e65e0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:51.548770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9053f7c-b332-4dd2-af6e-c705bc2e65e0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:51.549284 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9053f7c-b332-4dd2-af6e-c705bc2e65e0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:51.549711 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9053f7c-b332-4dd2-af6e-c705bc2e65e0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:51.550388 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9053f7c-b332-4dd2-af6e-c705bc2e65e0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:51.550828 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9053f7c-b332-4dd2-af6e-c705bc2e65e0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:51.551243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9053f7c-b332-4dd2-af6e-c705bc2e65e0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:51.697986 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c9053f7c-b332-4dd2-af6e-c705bc2e65e0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:51.710086 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c9053f7c-b332-4dd2-af6e-c705bc2e65e0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.172069 Aug 28 01:21:51.710931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1112/2447] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:51 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:51.818586 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6dd2454b-e897-47ac-aeaa-8c180c9f3cb8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:51.819228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6dd2454b-e897-47ac-aeaa-8c180c9f3cb8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:51.826782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6dd2454b-e897-47ac-aeaa-8c180c9f3cb8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:51.827259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6dd2454b-e897-47ac-aeaa-8c180c9f3cb8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:51.827690 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6dd2454b-e897-47ac-aeaa-8c180c9f3cb8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:51.828373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6dd2454b-e897-47ac-aeaa-8c180c9f3cb8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:51.828821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6dd2454b-e897-47ac-aeaa-8c180c9f3cb8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:51.829273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6dd2454b-e897-47ac-aeaa-8c180c9f3cb8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:51.967260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6dd2454b-e897-47ac-aeaa-8c180c9f3cb8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:51.976139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6dd2454b-e897-47ac-aeaa-8c180c9f3cb8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:51.977596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6dd2454b-e897-47ac-aeaa-8c180c9f3cb8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:51.978771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6dd2454b-e897-47ac-aeaa-8c180c9f3cb8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.165634 Aug 28 01:21:51.979335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1336/2448] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:51 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:52.306135 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c47d2249-44e3-4d0a-8667-2f879b813217 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:52.307088 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c47d2249-44e3-4d0a-8667-2f879b813217 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:52.315366 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c47d2249-44e3-4d0a-8667-2f879b813217 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:52.315856 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c47d2249-44e3-4d0a-8667-2f879b813217 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:52.316624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c47d2249-44e3-4d0a-8667-2f879b813217 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:52.317483 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c47d2249-44e3-4d0a-8667-2f879b813217 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:52.318178 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c47d2249-44e3-4d0a-8667-2f879b813217 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:52.318823 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c47d2249-44e3-4d0a-8667-2f879b813217 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:52.389942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c47d2249-44e3-4d0a-8667-2f879b813217 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:52.398617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c47d2249-44e3-4d0a-8667-2f879b813217 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.097726 Aug 28 01:21:52.399187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1113/2449] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:52 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:52.403549 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1c72e72e-9fa3-4ced-a78d-3ca14fc33262 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:52.404072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1c72e72e-9fa3-4ced-a78d-3ca14fc33262 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:52.411248 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1c72e72e-9fa3-4ced-a78d-3ca14fc33262 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:52.411731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1c72e72e-9fa3-4ced-a78d-3ca14fc33262 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:52.412157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1c72e72e-9fa3-4ced-a78d-3ca14fc33262 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:52.412832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1c72e72e-9fa3-4ced-a78d-3ca14fc33262 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:52.413306 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1c72e72e-9fa3-4ced-a78d-3ca14fc33262 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:52.413715 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1c72e72e-9fa3-4ced-a78d-3ca14fc33262 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:52.476994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1c72e72e-9fa3-4ced-a78d-3ca14fc33262 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:52.485344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1c72e72e-9fa3-4ced-a78d-3ca14fc33262 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.085861 Aug 28 01:21:52.485922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1337/2450] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:52 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:52.732915 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c4008fe1-f504-4d10-877e-6c63a4f3c3bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:52.733421 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c4008fe1-f504-4d10-877e-6c63a4f3c3bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:52.740100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c4008fe1-f504-4d10-877e-6c63a4f3c3bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:52.740584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c4008fe1-f504-4d10-877e-6c63a4f3c3bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:52.741043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c4008fe1-f504-4d10-877e-6c63a4f3c3bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:52.741728 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c4008fe1-f504-4d10-877e-6c63a4f3c3bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:52.742171 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c4008fe1-f504-4d10-877e-6c63a4f3c3bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:52.742616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c4008fe1-f504-4d10-877e-6c63a4f3c3bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:52.900885 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c4008fe1-f504-4d10-877e-6c63a4f3c3bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:52.909945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c4008fe1-f504-4d10-877e-6c63a4f3c3bc tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.181362 Aug 28 01:21:52.910533 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1114/2451] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:52 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:53.002877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0dffe0b8-0e36-40f8-b48e-db24769d3646 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:53.003423 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0dffe0b8-0e36-40f8-b48e-db24769d3646 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:53.010091 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0dffe0b8-0e36-40f8-b48e-db24769d3646 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:53.010583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0dffe0b8-0e36-40f8-b48e-db24769d3646 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:53.011013 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0dffe0b8-0e36-40f8-b48e-db24769d3646 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:53.011723 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0dffe0b8-0e36-40f8-b48e-db24769d3646 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:53.012172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0dffe0b8-0e36-40f8-b48e-db24769d3646 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:53.012646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0dffe0b8-0e36-40f8-b48e-db24769d3646 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:53.150771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0dffe0b8-0e36-40f8-b48e-db24769d3646 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:53.160182 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0dffe0b8-0e36-40f8-b48e-db24769d3646 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:53.161656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0dffe0b8-0e36-40f8-b48e-db24769d3646 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:53.162814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0dffe0b8-0e36-40f8-b48e-db24769d3646 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.164093 Aug 28 01:21:53.163362 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1338/2452] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:52 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:53.420485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-550f6baf-198c-461f-aa77-377ff510fb92 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:53.421155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-550f6baf-198c-461f-aa77-377ff510fb92 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:53.427942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-550f6baf-198c-461f-aa77-377ff510fb92 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:53.428497 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-550f6baf-198c-461f-aa77-377ff510fb92 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:53.428954 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-550f6baf-198c-461f-aa77-377ff510fb92 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:53.429660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-550f6baf-198c-461f-aa77-377ff510fb92 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:53.430138 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-550f6baf-198c-461f-aa77-377ff510fb92 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:53.430534 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-550f6baf-198c-461f-aa77-377ff510fb92 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:53.496077 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-550f6baf-198c-461f-aa77-377ff510fb92 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:53.505121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-550f6baf-198c-461f-aa77-377ff510fb92 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.088946 Aug 28 01:21:53.505700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1115/2453] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:53 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:53.511995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f75e3d13-cd0c-4d4e-b8fd-e61555674d21 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:53.512842 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f75e3d13-cd0c-4d4e-b8fd-e61555674d21 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:53.520715 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f75e3d13-cd0c-4d4e-b8fd-e61555674d21 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:53.521245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f75e3d13-cd0c-4d4e-b8fd-e61555674d21 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:53.521687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f75e3d13-cd0c-4d4e-b8fd-e61555674d21 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:53.522402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f75e3d13-cd0c-4d4e-b8fd-e61555674d21 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:53.522852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f75e3d13-cd0c-4d4e-b8fd-e61555674d21 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:53.523274 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f75e3d13-cd0c-4d4e-b8fd-e61555674d21 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:53.591882 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f75e3d13-cd0c-4d4e-b8fd-e61555674d21 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:53.600408 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f75e3d13-cd0c-4d4e-b8fd-e61555674d21 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.092880 Aug 28 01:21:53.601020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1339/2454] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:53 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:53.933050 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7dfddadd-6a60-43bf-be7a-a5dfe1d2b80a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:53.933800 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7dfddadd-6a60-43bf-be7a-a5dfe1d2b80a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:53.941931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7dfddadd-6a60-43bf-be7a-a5dfe1d2b80a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:53.942410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7dfddadd-6a60-43bf-be7a-a5dfe1d2b80a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:53.942863 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7dfddadd-6a60-43bf-be7a-a5dfe1d2b80a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:53.943556 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7dfddadd-6a60-43bf-be7a-a5dfe1d2b80a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:53.943994 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7dfddadd-6a60-43bf-be7a-a5dfe1d2b80a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:53.944411 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7dfddadd-6a60-43bf-be7a-a5dfe1d2b80a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:54.079000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7dfddadd-6a60-43bf-be7a-a5dfe1d2b80a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:54.090238 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7dfddadd-6a60-43bf-be7a-a5dfe1d2b80a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.161690 Aug 28 01:21:54.090871 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1116/2455] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:53 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:54.187119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9fd663dc-f568-4eee-8f93-ea60956f2141 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:54.187627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9fd663dc-f568-4eee-8f93-ea60956f2141 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:54.195468 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9fd663dc-f568-4eee-8f93-ea60956f2141 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:54.196228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9fd663dc-f568-4eee-8f93-ea60956f2141 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:54.196723 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9fd663dc-f568-4eee-8f93-ea60956f2141 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:54.197606 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9fd663dc-f568-4eee-8f93-ea60956f2141 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:54.198109 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9fd663dc-f568-4eee-8f93-ea60956f2141 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:54.198588 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9fd663dc-f568-4eee-8f93-ea60956f2141 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:54.201441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:21:54.347894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9fd663dc-f568-4eee-8f93-ea60956f2141 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:54.359541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9fd663dc-f568-4eee-8f93-ea60956f2141 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:54.361096 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9fd663dc-f568-4eee-8f93-ea60956f2141 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:54.362385 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9fd663dc-f568-4eee-8f93-ea60956f2141 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.179576 Aug 28 01:21:54.362919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1340/2456] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:54 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:54.529457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8753a63e-98dc-4816-98d1-d92e7e73eb08 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:54.530053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8753a63e-98dc-4816-98d1-d92e7e73eb08 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:54.536938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8753a63e-98dc-4816-98d1-d92e7e73eb08 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:54.537442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8753a63e-98dc-4816-98d1-d92e7e73eb08 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:54.537866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8753a63e-98dc-4816-98d1-d92e7e73eb08 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:54.538546 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8753a63e-98dc-4816-98d1-d92e7e73eb08 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:54.538986 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8753a63e-98dc-4816-98d1-d92e7e73eb08 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:54.539404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8753a63e-98dc-4816-98d1-d92e7e73eb08 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:54.607776 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8753a63e-98dc-4816-98d1-d92e7e73eb08 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:54.616888 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8753a63e-98dc-4816-98d1-d92e7e73eb08 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.092898 Aug 28 01:21:54.617535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1117/2457] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:54 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:54.621714 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2338baf4-b32f-4411-8746-0bdd00464688 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:54.622484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2338baf4-b32f-4411-8746-0bdd00464688 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:54.629321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2338baf4-b32f-4411-8746-0bdd00464688 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:54.629807 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2338baf4-b32f-4411-8746-0bdd00464688 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:54.630240 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2338baf4-b32f-4411-8746-0bdd00464688 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:54.630934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2338baf4-b32f-4411-8746-0bdd00464688 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:54.631382 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2338baf4-b32f-4411-8746-0bdd00464688 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:54.631805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2338baf4-b32f-4411-8746-0bdd00464688 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:54.702830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2338baf4-b32f-4411-8746-0bdd00464688 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:54.711250 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2338baf4-b32f-4411-8746-0bdd00464688 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.093886 Aug 28 01:21:54.711821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1341/2458] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:54 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:55.117481 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-76dd7dce-f439-4388-8884-53ea104e163f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:55.118119 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-76dd7dce-f439-4388-8884-53ea104e163f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:55.125276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-76dd7dce-f439-4388-8884-53ea104e163f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:55.126057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-76dd7dce-f439-4388-8884-53ea104e163f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:55.126772 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-76dd7dce-f439-4388-8884-53ea104e163f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:55.127928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-76dd7dce-f439-4388-8884-53ea104e163f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:55.128622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-76dd7dce-f439-4388-8884-53ea104e163f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:55.129281 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-76dd7dce-f439-4388-8884-53ea104e163f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:55.263610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-76dd7dce-f439-4388-8884-53ea104e163f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:55.272843 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-76dd7dce-f439-4388-8884-53ea104e163f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.160117 Aug 28 01:21:55.273533 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1118/2459] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:55 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:55.386352 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-66c55259-66b3-49a8-91f9-d077bc4f357d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:55.386956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-66c55259-66b3-49a8-91f9-d077bc4f357d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:55.393406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66c55259-66b3-49a8-91f9-d077bc4f357d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:55.393883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66c55259-66b3-49a8-91f9-d077bc4f357d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:55.394312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66c55259-66b3-49a8-91f9-d077bc4f357d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:55.394995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66c55259-66b3-49a8-91f9-d077bc4f357d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:55.395442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66c55259-66b3-49a8-91f9-d077bc4f357d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:55.395855 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66c55259-66b3-49a8-91f9-d077bc4f357d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:55.548652 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-66c55259-66b3-49a8-91f9-d077bc4f357d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:55.558560 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-66c55259-66b3-49a8-91f9-d077bc4f357d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:55.560087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-66c55259-66b3-49a8-91f9-d077bc4f357d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:55.561300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-66c55259-66b3-49a8-91f9-d077bc4f357d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.178900 Aug 28 01:21:55.561870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1342/2460] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:55 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:55.638021 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-de83ec0f-548a-4d8e-b4a5-ef328189d122 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:55.638625 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-de83ec0f-548a-4d8e-b4a5-ef328189d122 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:55.645402 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de83ec0f-548a-4d8e-b4a5-ef328189d122 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:55.645887 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de83ec0f-548a-4d8e-b4a5-ef328189d122 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:55.646296 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de83ec0f-548a-4d8e-b4a5-ef328189d122 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:55.647000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de83ec0f-548a-4d8e-b4a5-ef328189d122 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:55.647435 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de83ec0f-548a-4d8e-b4a5-ef328189d122 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:55.647884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-de83ec0f-548a-4d8e-b4a5-ef328189d122 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:55.710730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-de83ec0f-548a-4d8e-b4a5-ef328189d122 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:55.719340 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-de83ec0f-548a-4d8e-b4a5-ef328189d122 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.085824 Aug 28 01:21:55.719908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1119/2461] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:55 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:55.732355 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b5161dbe-2c37-416a-8f2e-eb209a3f2177 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:55.732926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b5161dbe-2c37-416a-8f2e-eb209a3f2177 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:55.740284 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5161dbe-2c37-416a-8f2e-eb209a3f2177 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:55.740762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5161dbe-2c37-416a-8f2e-eb209a3f2177 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:55.741226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5161dbe-2c37-416a-8f2e-eb209a3f2177 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:55.741908 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5161dbe-2c37-416a-8f2e-eb209a3f2177 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:55.742351 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5161dbe-2c37-416a-8f2e-eb209a3f2177 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:55.742754 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5161dbe-2c37-416a-8f2e-eb209a3f2177 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:55.811850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b5161dbe-2c37-416a-8f2e-eb209a3f2177 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:55.820752 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b5161dbe-2c37-416a-8f2e-eb209a3f2177 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.092431 Aug 28 01:21:55.821392 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1343/2462] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:55 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:56.295631 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9cdd1934-1ce1-4512-9ccf-66ae3346ce97 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:56.296144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9cdd1934-1ce1-4512-9ccf-66ae3346ce97 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:56.303202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9cdd1934-1ce1-4512-9ccf-66ae3346ce97 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:56.303687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9cdd1934-1ce1-4512-9ccf-66ae3346ce97 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:56.304125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9cdd1934-1ce1-4512-9ccf-66ae3346ce97 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:56.304821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9cdd1934-1ce1-4512-9ccf-66ae3346ce97 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:56.305314 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9cdd1934-1ce1-4512-9ccf-66ae3346ce97 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:56.305733 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9cdd1934-1ce1-4512-9ccf-66ae3346ce97 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:56.446766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9cdd1934-1ce1-4512-9ccf-66ae3346ce97 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:56.457105 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9cdd1934-1ce1-4512-9ccf-66ae3346ce97 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.165502 Aug 28 01:21:56.457667 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1120/2463] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:56 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:56.588775 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2f5709c8-5b53-46fb-8bc4-a6ade94dda7d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:56.589586 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2f5709c8-5b53-46fb-8bc4-a6ade94dda7d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:56.597602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f5709c8-5b53-46fb-8bc4-a6ade94dda7d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:56.598055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f5709c8-5b53-46fb-8bc4-a6ade94dda7d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:56.598500 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f5709c8-5b53-46fb-8bc4-a6ade94dda7d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:56.599181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f5709c8-5b53-46fb-8bc4-a6ade94dda7d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:56.599620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f5709c8-5b53-46fb-8bc4-a6ade94dda7d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:56.600054 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f5709c8-5b53-46fb-8bc4-a6ade94dda7d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:56.740036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-eb5c8c43-4476-4303-9abb-4aaba45d3c85 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:56.740637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-eb5c8c43-4476-4303-9abb-4aaba45d3c85 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:56.743369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2f5709c8-5b53-46fb-8bc4-a6ade94dda7d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:56.747205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb5c8c43-4476-4303-9abb-4aaba45d3c85 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:56.747684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb5c8c43-4476-4303-9abb-4aaba45d3c85 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:56.748111 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb5c8c43-4476-4303-9abb-4aaba45d3c85 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:56.748936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb5c8c43-4476-4303-9abb-4aaba45d3c85 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:56.749486 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb5c8c43-4476-4303-9abb-4aaba45d3c85 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:56.749994 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-eb5c8c43-4476-4303-9abb-4aaba45d3c85 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:56.752702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2f5709c8-5b53-46fb-8bc4-a6ade94dda7d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:56.754195 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2f5709c8-5b53-46fb-8bc4-a6ade94dda7d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:56.755356 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2f5709c8-5b53-46fb-8bc4-a6ade94dda7d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.173090 Aug 28 01:21:56.755944 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1344/2464] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:56 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:56.824178 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-eb5c8c43-4476-4303-9abb-4aaba45d3c85 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:56.836161 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-eb5c8c43-4476-4303-9abb-4aaba45d3c85 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.100032 Aug 28 01:21:56.836707 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1121/2465] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:56 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:56.844607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5392b35f-a9af-49bd-b553-f313c7c2e3da tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:56.845206 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5392b35f-a9af-49bd-b553-f313c7c2e3da tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:56.851541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5392b35f-a9af-49bd-b553-f313c7c2e3da tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:56.852027 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5392b35f-a9af-49bd-b553-f313c7c2e3da tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:56.852444 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5392b35f-a9af-49bd-b553-f313c7c2e3da tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:56.853163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5392b35f-a9af-49bd-b553-f313c7c2e3da tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:56.853606 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5392b35f-a9af-49bd-b553-f313c7c2e3da tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:56.854016 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5392b35f-a9af-49bd-b553-f313c7c2e3da tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:56.920467 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5392b35f-a9af-49bd-b553-f313c7c2e3da tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:56.933386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5392b35f-a9af-49bd-b553-f313c7c2e3da tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.094181 Aug 28 01:21:56.934144 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1345/2466] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:56 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:57.485090 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-20b26b3f-d50c-41b4-8f33-6b148727423d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:57.485697 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-20b26b3f-d50c-41b4-8f33-6b148727423d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:57.492027 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20b26b3f-d50c-41b4-8f33-6b148727423d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:57.492520 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20b26b3f-d50c-41b4-8f33-6b148727423d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:57.492947 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20b26b3f-d50c-41b4-8f33-6b148727423d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:57.493666 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20b26b3f-d50c-41b4-8f33-6b148727423d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:57.494111 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20b26b3f-d50c-41b4-8f33-6b148727423d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:57.494525 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20b26b3f-d50c-41b4-8f33-6b148727423d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:57.627147 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-20b26b3f-d50c-41b4-8f33-6b148727423d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:57.635939 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-20b26b3f-d50c-41b4-8f33-6b148727423d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.155282 Aug 28 01:21:57.636508 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1122/2467] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:57 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:57.778920 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e6f6cde6-fb31-4eca-ad17-11fc2c96264a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:57.779530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e6f6cde6-fb31-4eca-ad17-11fc2c96264a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:57.786645 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6f6cde6-fb31-4eca-ad17-11fc2c96264a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:57.787139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6f6cde6-fb31-4eca-ad17-11fc2c96264a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:57.787598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6f6cde6-fb31-4eca-ad17-11fc2c96264a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:57.788301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6f6cde6-fb31-4eca-ad17-11fc2c96264a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:57.788744 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6f6cde6-fb31-4eca-ad17-11fc2c96264a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:57.789233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6f6cde6-fb31-4eca-ad17-11fc2c96264a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:57.860492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-895ccc0b-dc48-447c-8116-1a56f35a22fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:57.861298 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-895ccc0b-dc48-447c-8116-1a56f35a22fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:57.867842 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-895ccc0b-dc48-447c-8116-1a56f35a22fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:57.868442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-895ccc0b-dc48-447c-8116-1a56f35a22fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:57.868937 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-895ccc0b-dc48-447c-8116-1a56f35a22fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:57.869703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-895ccc0b-dc48-447c-8116-1a56f35a22fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:57.870159 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-895ccc0b-dc48-447c-8116-1a56f35a22fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:57.870578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-895ccc0b-dc48-447c-8116-1a56f35a22fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:57.942669 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-895ccc0b-dc48-447c-8116-1a56f35a22fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:57.951398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-895ccc0b-dc48-447c-8116-1a56f35a22fe tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.094968 Aug 28 01:21:57.951985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1123/2468] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:57 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:57.961214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-357abf58-9b08-480d-a5be-fd51a8215f6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:57.961923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-357abf58-9b08-480d-a5be-fd51a8215f6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:57.966636 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e6f6cde6-fb31-4eca-ad17-11fc2c96264a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:57.968556 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-357abf58-9b08-480d-a5be-fd51a8215f6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:57.969098 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-357abf58-9b08-480d-a5be-fd51a8215f6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:57.969560 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-357abf58-9b08-480d-a5be-fd51a8215f6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:57.970264 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-357abf58-9b08-480d-a5be-fd51a8215f6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:57.970713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-357abf58-9b08-480d-a5be-fd51a8215f6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:57.971139 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-357abf58-9b08-480d-a5be-fd51a8215f6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:57.976519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e6f6cde6-fb31-4eca-ad17-11fc2c96264a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:57.977996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e6f6cde6-fb31-4eca-ad17-11fc2c96264a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:57.979169 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e6f6cde6-fb31-4eca-ad17-11fc2c96264a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.204368 Aug 28 01:21:57.979734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1346/2469] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:57 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:58.034960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-357abf58-9b08-480d-a5be-fd51a8215f6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:58.043771 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-357abf58-9b08-480d-a5be-fd51a8215f6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.086491 Aug 28 01:21:58.044360 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1124/2470] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:57 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:58.657510 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-96940fd3-6c34-4aa5-b204-f68daa6d1792 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:58.658127 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-96940fd3-6c34-4aa5-b204-f68daa6d1792 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:58.664328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-96940fd3-6c34-4aa5-b204-f68daa6d1792 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:58.664819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-96940fd3-6c34-4aa5-b204-f68daa6d1792 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:58.665269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-96940fd3-6c34-4aa5-b204-f68daa6d1792 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:58.665946 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-96940fd3-6c34-4aa5-b204-f68daa6d1792 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:58.666399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-96940fd3-6c34-4aa5-b204-f68daa6d1792 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:58.666809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-96940fd3-6c34-4aa5-b204-f68daa6d1792 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:58.819433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-96940fd3-6c34-4aa5-b204-f68daa6d1792 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:58.828071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-96940fd3-6c34-4aa5-b204-f68daa6d1792 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.174458 Aug 28 01:21:58.828641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1347/2471] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:58 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:58.972736 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d289ed8d-23a4-4ec0-adae-9e1bf8a03c57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:58.973373 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d289ed8d-23a4-4ec0-adae-9e1bf8a03c57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:58.981106 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d289ed8d-23a4-4ec0-adae-9e1bf8a03c57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:58.981808 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d289ed8d-23a4-4ec0-adae-9e1bf8a03c57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:58.982254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d289ed8d-23a4-4ec0-adae-9e1bf8a03c57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:58.982977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d289ed8d-23a4-4ec0-adae-9e1bf8a03c57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:58.983391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d289ed8d-23a4-4ec0-adae-9e1bf8a03c57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:58.983819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d289ed8d-23a4-4ec0-adae-9e1bf8a03c57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:59.004089 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-94a0439e-5dab-40d5-b109-002a1e2cac8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:59.005133 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-94a0439e-5dab-40d5-b109-002a1e2cac8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:59.013985 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-94a0439e-5dab-40d5-b109-002a1e2cac8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:59.014630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-94a0439e-5dab-40d5-b109-002a1e2cac8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:59.015321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-94a0439e-5dab-40d5-b109-002a1e2cac8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:59.016512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-94a0439e-5dab-40d5-b109-002a1e2cac8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:59.017354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-94a0439e-5dab-40d5-b109-002a1e2cac8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:59.018093 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-94a0439e-5dab-40d5-b109-002a1e2cac8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:59.046994 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d289ed8d-23a4-4ec0-adae-9e1bf8a03c57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:59.055373 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d289ed8d-23a4-4ec0-adae-9e1bf8a03c57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.086997 Aug 28 01:21:59.055937 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1125/2472] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:58 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:59.064574 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bb38bcd5-465d-48f3-8704-8fd60edf645b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:59.065207 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bb38bcd5-465d-48f3-8704-8fd60edf645b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:59.071616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb38bcd5-465d-48f3-8704-8fd60edf645b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:59.072091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb38bcd5-465d-48f3-8704-8fd60edf645b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:59.072515 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb38bcd5-465d-48f3-8704-8fd60edf645b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:59.073208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb38bcd5-465d-48f3-8704-8fd60edf645b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:59.073670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb38bcd5-465d-48f3-8704-8fd60edf645b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:59.074083 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb38bcd5-465d-48f3-8704-8fd60edf645b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:59.145326 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bb38bcd5-465d-48f3-8704-8fd60edf645b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:59.153835 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bb38bcd5-465d-48f3-8704-8fd60edf645b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.093076 Aug 28 01:21:59.154473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1126/2473] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:59 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:21:59.175784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-94a0439e-5dab-40d5-b109-002a1e2cac8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:59.187689 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-94a0439e-5dab-40d5-b109-002a1e2cac8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:59.189422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-94a0439e-5dab-40d5-b109-002a1e2cac8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:21:59.190553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-94a0439e-5dab-40d5-b109-002a1e2cac8e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.191508 Aug 28 01:21:59.191467 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1348/2474] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:21:58 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:21:59.850644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8861ebbc-1ef4-4157-99cc-d309d3fb540f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:21:59.851250 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8861ebbc-1ef4-4157-99cc-d309d3fb540f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:21:59.857757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8861ebbc-1ef4-4157-99cc-d309d3fb540f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:59.858239 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8861ebbc-1ef4-4157-99cc-d309d3fb540f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:59.858662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8861ebbc-1ef4-4157-99cc-d309d3fb540f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:59.859341 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8861ebbc-1ef4-4157-99cc-d309d3fb540f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:21:59.859785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8861ebbc-1ef4-4157-99cc-d309d3fb540f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:21:59.860198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8861ebbc-1ef4-4157-99cc-d309d3fb540f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:21:59.875108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:21:59.995599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8861ebbc-1ef4-4157-99cc-d309d3fb540f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:00.004695 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8861ebbc-1ef4-4157-99cc-d309d3fb540f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.158459 Aug 28 01:22:00.005325 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1127/2475] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:21:59 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:00.086118 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b8ed4878-7681-4d6d-ab90-6a0b0cb7751d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:00.086709 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b8ed4878-7681-4d6d-ab90-6a0b0cb7751d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:00.096592 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8ed4878-7681-4d6d-ab90-6a0b0cb7751d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:00.097437 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8ed4878-7681-4d6d-ab90-6a0b0cb7751d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:00.097877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8ed4878-7681-4d6d-ab90-6a0b0cb7751d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:00.099015 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8ed4878-7681-4d6d-ab90-6a0b0cb7751d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:00.099791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8ed4878-7681-4d6d-ab90-6a0b0cb7751d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:00.100230 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8ed4878-7681-4d6d-ab90-6a0b0cb7751d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:00.175446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3386a49a-c987-4b7d-b7f0-f4c37bd7f54a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:00.176055 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3386a49a-c987-4b7d-b7f0-f4c37bd7f54a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:00.182991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3386a49a-c987-4b7d-b7f0-f4c37bd7f54a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:00.183688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3386a49a-c987-4b7d-b7f0-f4c37bd7f54a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:00.184206 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b8ed4878-7681-4d6d-ab90-6a0b0cb7751d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:00.184390 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3386a49a-c987-4b7d-b7f0-f4c37bd7f54a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:00.185325 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3386a49a-c987-4b7d-b7f0-f4c37bd7f54a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:00.185912 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3386a49a-c987-4b7d-b7f0-f4c37bd7f54a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:00.186486 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3386a49a-c987-4b7d-b7f0-f4c37bd7f54a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:00.194406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b8ed4878-7681-4d6d-ab90-6a0b0cb7751d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.114278 Aug 28 01:22:00.195013 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1349/2476] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:00 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:00.214587 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3c07692a-34f1-4316-a70a-e2c5c0fd5d77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:00.215276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3c07692a-34f1-4316-a70a-e2c5c0fd5d77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:00.221349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3c07692a-34f1-4316-a70a-e2c5c0fd5d77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:00.221986 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3c07692a-34f1-4316-a70a-e2c5c0fd5d77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:00.222674 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3c07692a-34f1-4316-a70a-e2c5c0fd5d77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:00.223449 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3c07692a-34f1-4316-a70a-e2c5c0fd5d77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:00.224210 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3c07692a-34f1-4316-a70a-e2c5c0fd5d77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:00.224754 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3c07692a-34f1-4316-a70a-e2c5c0fd5d77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:00.257604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3386a49a-c987-4b7d-b7f0-f4c37bd7f54a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:00.266355 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3386a49a-c987-4b7d-b7f0-f4c37bd7f54a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.094820 Aug 28 01:22:00.267149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1128/2477] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:00 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:00.359445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3c07692a-34f1-4316-a70a-e2c5c0fd5d77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:00.372685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3c07692a-34f1-4316-a70a-e2c5c0fd5d77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:00.374932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3c07692a-34f1-4316-a70a-e2c5c0fd5d77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:00.376754 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3c07692a-34f1-4316-a70a-e2c5c0fd5d77 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.165788 Aug 28 01:22:00.377632 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1350/2478] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:00 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:01.026589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-caa57493-13c1-4aef-afb3-2d4b9a357ea5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:01.027249 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-caa57493-13c1-4aef-afb3-2d4b9a357ea5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:01.029644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:22:01.035377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-caa57493-13c1-4aef-afb3-2d4b9a357ea5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:01.035866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-caa57493-13c1-4aef-afb3-2d4b9a357ea5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:01.036296 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-caa57493-13c1-4aef-afb3-2d4b9a357ea5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:01.037031 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-caa57493-13c1-4aef-afb3-2d4b9a357ea5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:01.037481 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-caa57493-13c1-4aef-afb3-2d4b9a357ea5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:01.037900 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-caa57493-13c1-4aef-afb3-2d4b9a357ea5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:01.177703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-caa57493-13c1-4aef-afb3-2d4b9a357ea5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:01.186605 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-caa57493-13c1-4aef-afb3-2d4b9a357ea5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.163898 Aug 28 01:22:01.187196 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1129/2479] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:01 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:01.223238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b68b1aea-873c-457c-bb38-1c5ebf142bc8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:01.224353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b68b1aea-873c-457c-bb38-1c5ebf142bc8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:01.235600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b68b1aea-873c-457c-bb38-1c5ebf142bc8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:01.236471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b68b1aea-873c-457c-bb38-1c5ebf142bc8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:01.237263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b68b1aea-873c-457c-bb38-1c5ebf142bc8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:01.238469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b68b1aea-873c-457c-bb38-1c5ebf142bc8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:01.239272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b68b1aea-873c-457c-bb38-1c5ebf142bc8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:01.240048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b68b1aea-873c-457c-bb38-1c5ebf142bc8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:01.288380 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-469c4c4d-97ef-42e0-827e-86c990714ff7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:01.289091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-469c4c4d-97ef-42e0-827e-86c990714ff7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:01.295943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-469c4c4d-97ef-42e0-827e-86c990714ff7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:01.296528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-469c4c4d-97ef-42e0-827e-86c990714ff7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:01.297075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-469c4c4d-97ef-42e0-827e-86c990714ff7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:01.297878 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-469c4c4d-97ef-42e0-827e-86c990714ff7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:01.298414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-469c4c4d-97ef-42e0-827e-86c990714ff7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:01.298939 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-469c4c4d-97ef-42e0-827e-86c990714ff7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:01.313582 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b68b1aea-873c-457c-bb38-1c5ebf142bc8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:01.322959 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b68b1aea-873c-457c-bb38-1c5ebf142bc8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.106971 Aug 28 01:22:01.323769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1351/2480] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:01 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:01.365147 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-469c4c4d-97ef-42e0-827e-86c990714ff7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:01.373662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-469c4c4d-97ef-42e0-827e-86c990714ff7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089464 Aug 28 01:22:01.374226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1130/2481] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:01 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:01.400555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4019f49c-3f62-432d-80c9-30017f473167 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:01.401273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4019f49c-3f62-432d-80c9-30017f473167 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:01.407412 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4019f49c-3f62-432d-80c9-30017f473167 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:01.407896 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4019f49c-3f62-432d-80c9-30017f473167 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:01.408354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4019f49c-3f62-432d-80c9-30017f473167 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:01.409153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4019f49c-3f62-432d-80c9-30017f473167 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:01.409603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4019f49c-3f62-432d-80c9-30017f473167 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:01.410033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4019f49c-3f62-432d-80c9-30017f473167 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:01.551127 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4019f49c-3f62-432d-80c9-30017f473167 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:01.563686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4019f49c-3f62-432d-80c9-30017f473167 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:01.565918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4019f49c-3f62-432d-80c9-30017f473167 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:01.567492 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4019f49c-3f62-432d-80c9-30017f473167 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.170988 Aug 28 01:22:01.568066 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1352/2482] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:01 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:02.209656 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7144322c-8935-4547-a064-6d9387ff07f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:02.210245 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7144322c-8935-4547-a064-6d9387ff07f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:02.217007 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7144322c-8935-4547-a064-6d9387ff07f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:02.217506 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7144322c-8935-4547-a064-6d9387ff07f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:02.218080 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7144322c-8935-4547-a064-6d9387ff07f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:02.218680 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7144322c-8935-4547-a064-6d9387ff07f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:02.219124 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7144322c-8935-4547-a064-6d9387ff07f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:02.219533 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7144322c-8935-4547-a064-6d9387ff07f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:02.349230 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-88ea596c-703a-4842-bb73-cb51c600ffce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:02.350169 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-88ea596c-703a-4842-bb73-cb51c600ffce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:02.357062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88ea596c-703a-4842-bb73-cb51c600ffce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:02.357595 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88ea596c-703a-4842-bb73-cb51c600ffce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:02.358055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88ea596c-703a-4842-bb73-cb51c600ffce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:02.358342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7144322c-8935-4547-a064-6d9387ff07f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:02.358747 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88ea596c-703a-4842-bb73-cb51c600ffce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:02.359216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88ea596c-703a-4842-bb73-cb51c600ffce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:02.359640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88ea596c-703a-4842-bb73-cb51c600ffce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:02.373725 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7144322c-8935-4547-a064-6d9387ff07f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.167948 Aug 28 01:22:02.374754 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1131/2483] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:02 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:02.395065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d6036683-3e7c-4e7d-9f90-b299a182cf3a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:02.395680 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d6036683-3e7c-4e7d-9f90-b299a182cf3a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:02.402306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6036683-3e7c-4e7d-9f90-b299a182cf3a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:02.402802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6036683-3e7c-4e7d-9f90-b299a182cf3a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:02.403239 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6036683-3e7c-4e7d-9f90-b299a182cf3a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:02.403925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6036683-3e7c-4e7d-9f90-b299a182cf3a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:02.404372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6036683-3e7c-4e7d-9f90-b299a182cf3a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:02.404818 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6036683-3e7c-4e7d-9f90-b299a182cf3a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:02.427846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-88ea596c-703a-4842-bb73-cb51c600ffce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:02.440027 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-88ea596c-703a-4842-bb73-cb51c600ffce tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.094676 Aug 28 01:22:02.440666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1353/2484] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:02 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:02.471615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d6036683-3e7c-4e7d-9f90-b299a182cf3a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:02.480308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d6036683-3e7c-4e7d-9f90-b299a182cf3a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089241 Aug 28 01:22:02.480884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1132/2485] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:02 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:22:02.594460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-956ed59f-9266-440e-9c30-fe2a37407127 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:02.595185 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-956ed59f-9266-440e-9c30-fe2a37407127 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:02.601992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-956ed59f-9266-440e-9c30-fe2a37407127 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:02.602543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-956ed59f-9266-440e-9c30-fe2a37407127 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:02.603006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-956ed59f-9266-440e-9c30-fe2a37407127 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:02.603812 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-956ed59f-9266-440e-9c30-fe2a37407127 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:02.604338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-956ed59f-9266-440e-9c30-fe2a37407127 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:02.604753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-956ed59f-9266-440e-9c30-fe2a37407127 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:02.750065 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-956ed59f-9266-440e-9c30-fe2a37407127 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:02.759407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-956ed59f-9266-440e-9c30-fe2a37407127 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:02.760855 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-956ed59f-9266-440e-9c30-fe2a37407127 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:02.762059 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-956ed59f-9266-440e-9c30-fe2a37407127 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.171977 Aug 28 01:22:02.762621 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1354/2486] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:02 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:03.407206 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ada17c85-23bf-4704-aab3-9be2489aff84 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:03.407775 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ada17c85-23bf-4704-aab3-9be2489aff84 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:03.414875 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ada17c85-23bf-4704-aab3-9be2489aff84 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:03.415361 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ada17c85-23bf-4704-aab3-9be2489aff84 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:03.415784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ada17c85-23bf-4704-aab3-9be2489aff84 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:03.416468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ada17c85-23bf-4704-aab3-9be2489aff84 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:03.416913 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ada17c85-23bf-4704-aab3-9be2489aff84 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:03.417356 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ada17c85-23bf-4704-aab3-9be2489aff84 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:03.471896 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-055267ba-22ce-48d0-90b4-dcc4a467fe2c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:03.472468 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-055267ba-22ce-48d0-90b4-dcc4a467fe2c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:03.474873 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:22:03.480598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-055267ba-22ce-48d0-90b4-dcc4a467fe2c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:03.481097 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-055267ba-22ce-48d0-90b4-dcc4a467fe2c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:03.481531 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-055267ba-22ce-48d0-90b4-dcc4a467fe2c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:03.482237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-055267ba-22ce-48d0-90b4-dcc4a467fe2c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:03.482673 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-055267ba-22ce-48d0-90b4-dcc4a467fe2c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:03.483093 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-055267ba-22ce-48d0-90b4-dcc4a467fe2c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:03.560393 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-055267ba-22ce-48d0-90b4-dcc4a467fe2c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:03.570337 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-055267ba-22ce-48d0-90b4-dcc4a467fe2c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.102310 Aug 28 01:22:03.570927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1355/2487] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:03 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:03.575483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-725dbc02-5014-4381-8a2d-699b0935ff39 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:03.576522 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-725dbc02-5014-4381-8a2d-699b0935ff39 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:03.582737 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-725dbc02-5014-4381-8a2d-699b0935ff39 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:03.583233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-725dbc02-5014-4381-8a2d-699b0935ff39 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:03.583648 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-725dbc02-5014-4381-8a2d-699b0935ff39 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:03.584343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-725dbc02-5014-4381-8a2d-699b0935ff39 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:03.584787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-725dbc02-5014-4381-8a2d-699b0935ff39 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:03.585231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-725dbc02-5014-4381-8a2d-699b0935ff39 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:03.595684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ada17c85-23bf-4704-aab3-9be2489aff84 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:03.604636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ada17c85-23bf-4704-aab3-9be2489aff84 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.201238 Aug 28 01:22:03.605499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1133/2488] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:03 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:03.648068 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-725dbc02-5014-4381-8a2d-699b0935ff39 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:03.650354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:22:03.657926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-725dbc02-5014-4381-8a2d-699b0935ff39 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.085933 Aug 28 01:22:03.658460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1356/2489] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:03 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:03.785401 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8627dbbe-b016-45ff-8a22-d1094b6a8923 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:03.785976 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8627dbbe-b016-45ff-8a22-d1094b6a8923 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:03.792483 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8627dbbe-b016-45ff-8a22-d1094b6a8923 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:03.792998 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8627dbbe-b016-45ff-8a22-d1094b6a8923 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:03.793505 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8627dbbe-b016-45ff-8a22-d1094b6a8923 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:03.794303 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8627dbbe-b016-45ff-8a22-d1094b6a8923 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:03.794749 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8627dbbe-b016-45ff-8a22-d1094b6a8923 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:03.795222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8627dbbe-b016-45ff-8a22-d1094b6a8923 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:03.927898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8627dbbe-b016-45ff-8a22-d1094b6a8923 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:03.937444 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8627dbbe-b016-45ff-8a22-d1094b6a8923 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:03.938945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8627dbbe-b016-45ff-8a22-d1094b6a8923 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:03.940164 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8627dbbe-b016-45ff-8a22-d1094b6a8923 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.159033 Aug 28 01:22:03.940794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1134/2490] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:03 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:04.591449 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3eff9299-a524-4699-bf92-b36a615a48cd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:04.592136 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3eff9299-a524-4699-bf92-b36a615a48cd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:04.599224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3eff9299-a524-4699-bf92-b36a615a48cd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:04.599702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3eff9299-a524-4699-bf92-b36a615a48cd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:04.600130 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3eff9299-a524-4699-bf92-b36a615a48cd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:04.600850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3eff9299-a524-4699-bf92-b36a615a48cd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:04.601320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3eff9299-a524-4699-bf92-b36a615a48cd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:04.601742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3eff9299-a524-4699-bf92-b36a615a48cd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:04.626018 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-203160bd-f6a1-4a45-a5ee-c8d3e3717815 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:04.626729 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-203160bd-f6a1-4a45-a5ee-c8d3e3717815 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:04.634443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-203160bd-f6a1-4a45-a5ee-c8d3e3717815 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:04.635059 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-203160bd-f6a1-4a45-a5ee-c8d3e3717815 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:04.635613 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-203160bd-f6a1-4a45-a5ee-c8d3e3717815 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:04.636553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-203160bd-f6a1-4a45-a5ee-c8d3e3717815 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:04.637242 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-203160bd-f6a1-4a45-a5ee-c8d3e3717815 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:04.637918 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-203160bd-f6a1-4a45-a5ee-c8d3e3717815 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:04.667321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3eff9299-a524-4699-bf92-b36a615a48cd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:04.676169 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3eff9299-a524-4699-bf92-b36a615a48cd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.088625 Aug 28 01:22:04.676784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1357/2491] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:04 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:04.681702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2c88b424-7b0c-4693-b8e1-0e4e6ca4b8c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:04.682306 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2c88b424-7b0c-4693-b8e1-0e4e6ca4b8c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:04.688512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c88b424-7b0c-4693-b8e1-0e4e6ca4b8c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:04.689020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c88b424-7b0c-4693-b8e1-0e4e6ca4b8c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:04.689451 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c88b424-7b0c-4693-b8e1-0e4e6ca4b8c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:04.690379 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c88b424-7b0c-4693-b8e1-0e4e6ca4b8c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:04.690832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c88b424-7b0c-4693-b8e1-0e4e6ca4b8c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:04.691633 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2c88b424-7b0c-4693-b8e1-0e4e6ca4b8c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:04.762216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2c88b424-7b0c-4693-b8e1-0e4e6ca4b8c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:04.770992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2c88b424-7b0c-4693-b8e1-0e4e6ca4b8c5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.093055 Aug 28 01:22:04.771803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1358/2492] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:04 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:04.792809 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-203160bd-f6a1-4a45-a5ee-c8d3e3717815 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:04.803718 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-203160bd-f6a1-4a45-a5ee-c8d3e3717815 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.181620 Aug 28 01:22:04.804290 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1135/2493] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:04 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:04.967542 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5b1b5548-0095-4a1a-a512-a8d36be5fef5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:04.968170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5b1b5548-0095-4a1a-a512-a8d36be5fef5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:04.975953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b1b5548-0095-4a1a-a512-a8d36be5fef5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:04.976433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b1b5548-0095-4a1a-a512-a8d36be5fef5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:04.976862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b1b5548-0095-4a1a-a512-a8d36be5fef5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:04.977592 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b1b5548-0095-4a1a-a512-a8d36be5fef5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:04.978044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b1b5548-0095-4a1a-a512-a8d36be5fef5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:04.978474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b1b5548-0095-4a1a-a512-a8d36be5fef5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:05.121578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5b1b5548-0095-4a1a-a512-a8d36be5fef5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:05.132297 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5b1b5548-0095-4a1a-a512-a8d36be5fef5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:05.133782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5b1b5548-0095-4a1a-a512-a8d36be5fef5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:05.134935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5b1b5548-0095-4a1a-a512-a8d36be5fef5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.173562 Aug 28 01:22:05.135498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1359/2494] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:04 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:05.697458 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b35e5bb8-a5e5-4f96-8447-c57f43c818af tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:05.698101 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b35e5bb8-a5e5-4f96-8447-c57f43c818af tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:05.705505 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b35e5bb8-a5e5-4f96-8447-c57f43c818af tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:05.706040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b35e5bb8-a5e5-4f96-8447-c57f43c818af tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:05.706512 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b35e5bb8-a5e5-4f96-8447-c57f43c818af tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:05.707258 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b35e5bb8-a5e5-4f96-8447-c57f43c818af tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:05.707767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b35e5bb8-a5e5-4f96-8447-c57f43c818af tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:05.708229 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b35e5bb8-a5e5-4f96-8447-c57f43c818af tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:05.780652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b35e5bb8-a5e5-4f96-8447-c57f43c818af tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:05.790795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b35e5bb8-a5e5-4f96-8447-c57f43c818af tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.097514 Aug 28 01:22:05.791350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1136/2495] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:05 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:05.792132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-de0b7cc9-0b20-4fb8-87b6-83e01d318d7f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:05.792893 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-de0b7cc9-0b20-4fb8-87b6-83e01d318d7f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:05.799662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de0b7cc9-0b20-4fb8-87b6-83e01d318d7f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:05.800212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de0b7cc9-0b20-4fb8-87b6-83e01d318d7f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:05.800651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de0b7cc9-0b20-4fb8-87b6-83e01d318d7f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:05.801388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de0b7cc9-0b20-4fb8-87b6-83e01d318d7f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:05.801872 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de0b7cc9-0b20-4fb8-87b6-83e01d318d7f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:05.802291 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-de0b7cc9-0b20-4fb8-87b6-83e01d318d7f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:05.824002 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-175416ac-3d1c-4636-924a-3eb972f69198 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:05.824613 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-175416ac-3d1c-4636-924a-3eb972f69198 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:05.830694 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-175416ac-3d1c-4636-924a-3eb972f69198 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:05.831203 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-175416ac-3d1c-4636-924a-3eb972f69198 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:05.831632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-175416ac-3d1c-4636-924a-3eb972f69198 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:05.832316 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-175416ac-3d1c-4636-924a-3eb972f69198 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:05.832758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-175416ac-3d1c-4636-924a-3eb972f69198 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:05.833208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-175416ac-3d1c-4636-924a-3eb972f69198 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:05.865489 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-de0b7cc9-0b20-4fb8-87b6-83e01d318d7f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:05.873941 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-de0b7cc9-0b20-4fb8-87b6-83e01d318d7f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.085854 Aug 28 01:22:05.874507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1360/2496] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:05 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:22:05.965003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-175416ac-3d1c-4636-924a-3eb972f69198 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:05.976789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-175416ac-3d1c-4636-924a-3eb972f69198 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.156304 Aug 28 01:22:05.977670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1137/2497] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:05 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:22:06.158765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c88cd96a-eabb-401d-be0e-f7d5398ca15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:06.159363 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c88cd96a-eabb-401d-be0e-f7d5398ca15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:06.165978 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c88cd96a-eabb-401d-be0e-f7d5398ca15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:06.166485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c88cd96a-eabb-401d-be0e-f7d5398ca15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:06.166915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c88cd96a-eabb-401d-be0e-f7d5398ca15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:06.167625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c88cd96a-eabb-401d-be0e-f7d5398ca15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:06.168050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c88cd96a-eabb-401d-be0e-f7d5398ca15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:06.168455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c88cd96a-eabb-401d-be0e-f7d5398ca15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:06.300826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c88cd96a-eabb-401d-be0e-f7d5398ca15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:06.309521 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c88cd96a-eabb-401d-be0e-f7d5398ca15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:06.311005 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c88cd96a-eabb-401d-be0e-f7d5398ca15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:06.312159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c88cd96a-eabb-401d-be0e-f7d5398ca15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.157358 Aug 28 01:22:06.312715 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1361/2498] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:06 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:06.811984 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3e98eb57-8c78-464d-9313-066aac101f0e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:06.812613 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3e98eb57-8c78-464d-9313-066aac101f0e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:06.819914 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3e98eb57-8c78-464d-9313-066aac101f0e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:06.820396 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3e98eb57-8c78-464d-9313-066aac101f0e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:06.820819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3e98eb57-8c78-464d-9313-066aac101f0e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:06.821545 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3e98eb57-8c78-464d-9313-066aac101f0e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:06.821987 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3e98eb57-8c78-464d-9313-066aac101f0e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:06.822396 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3e98eb57-8c78-464d-9313-066aac101f0e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:06.892319 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3e98eb57-8c78-464d-9313-066aac101f0e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:06.894577 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e1dc80c6-02a0-4657-8034-a51957702b2a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:06.895187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e1dc80c6-02a0-4657-8034-a51957702b2a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:06.901434 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3e98eb57-8c78-464d-9313-066aac101f0e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.093302 Aug 28 01:22:06.901838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1dc80c6-02a0-4657-8034-a51957702b2a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:06.902028 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1138/2499] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:06 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:06.902312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1dc80c6-02a0-4657-8034-a51957702b2a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:06.902744 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1dc80c6-02a0-4657-8034-a51957702b2a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:06.903431 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1dc80c6-02a0-4657-8034-a51957702b2a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:06.903881 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1dc80c6-02a0-4657-8034-a51957702b2a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:06.904295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1dc80c6-02a0-4657-8034-a51957702b2a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:06.967199 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e1dc80c6-02a0-4657-8034-a51957702b2a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:06.975793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e1dc80c6-02a0-4657-8034-a51957702b2a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.085205 Aug 28 01:22:06.976357 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1362/2500] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:06 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:06.998438 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b3f677f5-bda9-4bd8-bd15-9463eb6477f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:06.999044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b3f677f5-bda9-4bd8-bd15-9463eb6477f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:07.005206 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3f677f5-bda9-4bd8-bd15-9463eb6477f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:07.005679 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3f677f5-bda9-4bd8-bd15-9463eb6477f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:07.006107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3f677f5-bda9-4bd8-bd15-9463eb6477f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:07.006780 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3f677f5-bda9-4bd8-bd15-9463eb6477f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:07.007219 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3f677f5-bda9-4bd8-bd15-9463eb6477f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:07.007630 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b3f677f5-bda9-4bd8-bd15-9463eb6477f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:07.153103 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b3f677f5-bda9-4bd8-bd15-9463eb6477f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:07.161622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b3f677f5-bda9-4bd8-bd15-9463eb6477f7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.166914 Aug 28 01:22:07.162199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1139/2501] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:06 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:07.336472 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e892df5d-3eb4-4815-bb47-05cf979bcd40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:07.337106 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e892df5d-3eb4-4815-bb47-05cf979bcd40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:07.343789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e892df5d-3eb4-4815-bb47-05cf979bcd40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:07.344205 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e892df5d-3eb4-4815-bb47-05cf979bcd40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:07.344632 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e892df5d-3eb4-4815-bb47-05cf979bcd40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:07.345347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e892df5d-3eb4-4815-bb47-05cf979bcd40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:07.345830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e892df5d-3eb4-4815-bb47-05cf979bcd40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:07.346243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e892df5d-3eb4-4815-bb47-05cf979bcd40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:07.485638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e892df5d-3eb4-4815-bb47-05cf979bcd40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:07.494851 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e892df5d-3eb4-4815-bb47-05cf979bcd40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:07.496257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e892df5d-3eb4-4815-bb47-05cf979bcd40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:07.497421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e892df5d-3eb4-4815-bb47-05cf979bcd40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.165119 Aug 28 01:22:07.497980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1363/2502] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:07 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:07.924693 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f9f84e69-11e8-4bd4-aeff-d0d33cd06ef2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:07.925362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f9f84e69-11e8-4bd4-aeff-d0d33cd06ef2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:07.935797 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9f84e69-11e8-4bd4-aeff-d0d33cd06ef2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:07.936439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9f84e69-11e8-4bd4-aeff-d0d33cd06ef2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:07.937028 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9f84e69-11e8-4bd4-aeff-d0d33cd06ef2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:07.938170 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9f84e69-11e8-4bd4-aeff-d0d33cd06ef2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:07.938897 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9f84e69-11e8-4bd4-aeff-d0d33cd06ef2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:07.939600 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9f84e69-11e8-4bd4-aeff-d0d33cd06ef2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:07.997603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-dab28269-94c9-4c81-b021-b80f5972e88c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:07.998236 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-dab28269-94c9-4c81-b021-b80f5972e88c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:08.005138 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dab28269-94c9-4c81-b021-b80f5972e88c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:08.005353 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f9f84e69-11e8-4bd4-aeff-d0d33cd06ef2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:08.005615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dab28269-94c9-4c81-b021-b80f5972e88c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:08.006047 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dab28269-94c9-4c81-b021-b80f5972e88c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:08.006726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dab28269-94c9-4c81-b021-b80f5972e88c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:08.007168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dab28269-94c9-4c81-b021-b80f5972e88c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:08.007582 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dab28269-94c9-4c81-b021-b80f5972e88c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:08.014459 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f9f84e69-11e8-4bd4-aeff-d0d33cd06ef2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.095110 Aug 28 01:22:08.015337 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1140/2503] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:07 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:08.071324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dab28269-94c9-4c81-b021-b80f5972e88c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:08.083688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-dab28269-94c9-4c81-b021-b80f5972e88c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089648 Aug 28 01:22:08.084516 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1364/2504] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:07 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:22:08.183463 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1de0f51e-b670-48c7-aef3-00c950bec738 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:08.183974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1de0f51e-b670-48c7-aef3-00c950bec738 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:08.190444 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1de0f51e-b670-48c7-aef3-00c950bec738 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:08.191340 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1de0f51e-b670-48c7-aef3-00c950bec738 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:08.191708 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1de0f51e-b670-48c7-aef3-00c950bec738 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:08.192503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1de0f51e-b670-48c7-aef3-00c950bec738 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:08.193025 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1de0f51e-b670-48c7-aef3-00c950bec738 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:08.193450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1de0f51e-b670-48c7-aef3-00c950bec738 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:08.334478 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1de0f51e-b670-48c7-aef3-00c950bec738 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:08.342765 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1de0f51e-b670-48c7-aef3-00c950bec738 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.163296 Aug 28 01:22:08.343338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1141/2505] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:08 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:08.520710 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-29ace2a2-5a8f-40bd-893b-ce42b4f9ac48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:08.521264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-29ace2a2-5a8f-40bd-893b-ce42b4f9ac48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:08.528001 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29ace2a2-5a8f-40bd-893b-ce42b4f9ac48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:08.528490 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29ace2a2-5a8f-40bd-893b-ce42b4f9ac48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:08.528922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29ace2a2-5a8f-40bd-893b-ce42b4f9ac48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:08.529658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29ace2a2-5a8f-40bd-893b-ce42b4f9ac48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:08.530108 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29ace2a2-5a8f-40bd-893b-ce42b4f9ac48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:08.530531 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29ace2a2-5a8f-40bd-893b-ce42b4f9ac48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:08.680127 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-29ace2a2-5a8f-40bd-893b-ce42b4f9ac48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:08.694653 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-29ace2a2-5a8f-40bd-893b-ce42b4f9ac48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:08.696076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-29ace2a2-5a8f-40bd-893b-ce42b4f9ac48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:08.697263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-29ace2a2-5a8f-40bd-893b-ce42b4f9ac48 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.180815 Aug 28 01:22:08.697853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1365/2506] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:08 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:09.035601 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fd7f798e-b160-4fac-a5b9-9f3be37dde08 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:09.036234 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fd7f798e-b160-4fac-a5b9-9f3be37dde08 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:09.043376 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd7f798e-b160-4fac-a5b9-9f3be37dde08 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:09.043866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd7f798e-b160-4fac-a5b9-9f3be37dde08 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:09.044301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd7f798e-b160-4fac-a5b9-9f3be37dde08 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:09.045038 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd7f798e-b160-4fac-a5b9-9f3be37dde08 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:09.045496 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd7f798e-b160-4fac-a5b9-9f3be37dde08 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:09.045927 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd7f798e-b160-4fac-a5b9-9f3be37dde08 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:09.109518 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fc15462b-a864-4718-b7e1-143a73f87844 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:09.110665 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fc15462b-a864-4718-b7e1-143a73f87844 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:09.119988 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fd7f798e-b160-4fac-a5b9-9f3be37dde08 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:09.123190 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc15462b-a864-4718-b7e1-143a73f87844 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:09.124616 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc15462b-a864-4718-b7e1-143a73f87844 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:09.124616 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc15462b-a864-4718-b7e1-143a73f87844 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:09.125774 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc15462b-a864-4718-b7e1-143a73f87844 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:09.126627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc15462b-a864-4718-b7e1-143a73f87844 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:09.127402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc15462b-a864-4718-b7e1-143a73f87844 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:09.131636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fd7f798e-b160-4fac-a5b9-9f3be37dde08 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.099867 Aug 28 01:22:09.132239 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1142/2507] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:09 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:09.204819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fc15462b-a864-4718-b7e1-143a73f87844 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:09.214193 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fc15462b-a864-4718-b7e1-143a73f87844 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.112210 Aug 28 01:22:09.214784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1366/2508] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:09 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:22:09.372371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fcb21d71-e58a-4aa2-8cd8-885c4210d642 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:09.372990 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fcb21d71-e58a-4aa2-8cd8-885c4210d642 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:09.380026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fcb21d71-e58a-4aa2-8cd8-885c4210d642 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:09.380521 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fcb21d71-e58a-4aa2-8cd8-885c4210d642 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:09.380954 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fcb21d71-e58a-4aa2-8cd8-885c4210d642 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:09.381697 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fcb21d71-e58a-4aa2-8cd8-885c4210d642 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:09.382144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fcb21d71-e58a-4aa2-8cd8-885c4210d642 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:09.382567 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fcb21d71-e58a-4aa2-8cd8-885c4210d642 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:09.528918 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fcb21d71-e58a-4aa2-8cd8-885c4210d642 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:09.537690 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fcb21d71-e58a-4aa2-8cd8-885c4210d642 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.169688 Aug 28 01:22:09.538319 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1143/2509] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:09 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:09.732284 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-36a80144-25af-4bbd-a6d0-eee29eba6817 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:09.733364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-36a80144-25af-4bbd-a6d0-eee29eba6817 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:09.741556 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-36a80144-25af-4bbd-a6d0-eee29eba6817 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:09.742221 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-36a80144-25af-4bbd-a6d0-eee29eba6817 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:09.742694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-36a80144-25af-4bbd-a6d0-eee29eba6817 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:09.743675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-36a80144-25af-4bbd-a6d0-eee29eba6817 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:09.744155 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-36a80144-25af-4bbd-a6d0-eee29eba6817 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:09.744574 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-36a80144-25af-4bbd-a6d0-eee29eba6817 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:09.889482 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-36a80144-25af-4bbd-a6d0-eee29eba6817 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:09.898709 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-36a80144-25af-4bbd-a6d0-eee29eba6817 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:09.900138 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-36a80144-25af-4bbd-a6d0-eee29eba6817 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:09.901384 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-36a80144-25af-4bbd-a6d0-eee29eba6817 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.176684 Aug 28 01:22:09.901949 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1367/2510] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:09 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:10.155814 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f95369de-ed67-4bdf-b6d9-661c195fa70e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:10.156745 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f95369de-ed67-4bdf-b6d9-661c195fa70e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:10.164874 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f95369de-ed67-4bdf-b6d9-661c195fa70e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:10.165392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f95369de-ed67-4bdf-b6d9-661c195fa70e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:10.165822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f95369de-ed67-4bdf-b6d9-661c195fa70e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:10.166512 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f95369de-ed67-4bdf-b6d9-661c195fa70e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:10.166958 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f95369de-ed67-4bdf-b6d9-661c195fa70e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:10.167370 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f95369de-ed67-4bdf-b6d9-661c195fa70e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:10.169976 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:22:10.231852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f95369de-ed67-4bdf-b6d9-661c195fa70e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:10.236606 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-26aa92bf-bd3c-4c6e-94f3-b7859b3aeb1d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:10.237388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-26aa92bf-bd3c-4c6e-94f3-b7859b3aeb1d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:10.241393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f95369de-ed67-4bdf-b6d9-661c195fa70e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.091072 Aug 28 01:22:10.241997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1144/2511] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:10 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:10.245562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26aa92bf-bd3c-4c6e-94f3-b7859b3aeb1d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:10.246047 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26aa92bf-bd3c-4c6e-94f3-b7859b3aeb1d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:10.246498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26aa92bf-bd3c-4c6e-94f3-b7859b3aeb1d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:10.247200 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26aa92bf-bd3c-4c6e-94f3-b7859b3aeb1d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:10.247764 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26aa92bf-bd3c-4c6e-94f3-b7859b3aeb1d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:10.248186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26aa92bf-bd3c-4c6e-94f3-b7859b3aeb1d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:10.315447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-26aa92bf-bd3c-4c6e-94f3-b7859b3aeb1d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:10.324309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-26aa92bf-bd3c-4c6e-94f3-b7859b3aeb1d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.091853 Aug 28 01:22:10.324894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1368/2512] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:10 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:10.558425 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-703e9989-21e2-4e29-ad66-c6516c73880b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:10.559047 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-703e9989-21e2-4e29-ad66-c6516c73880b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:10.565472 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-703e9989-21e2-4e29-ad66-c6516c73880b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:10.566291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-703e9989-21e2-4e29-ad66-c6516c73880b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:10.566707 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-703e9989-21e2-4e29-ad66-c6516c73880b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:10.567395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-703e9989-21e2-4e29-ad66-c6516c73880b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:10.567837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-703e9989-21e2-4e29-ad66-c6516c73880b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:10.568251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-703e9989-21e2-4e29-ad66-c6516c73880b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:10.705701 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-703e9989-21e2-4e29-ad66-c6516c73880b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:10.714582 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-703e9989-21e2-4e29-ad66-c6516c73880b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.160025 Aug 28 01:22:10.715176 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1145/2513] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:10 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:10.924439 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bfb4b13b-29b6-4b33-a9ce-5eaf5ae87195 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:10.925064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bfb4b13b-29b6-4b33-a9ce-5eaf5ae87195 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:10.933674 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bfb4b13b-29b6-4b33-a9ce-5eaf5ae87195 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:10.934483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bfb4b13b-29b6-4b33-a9ce-5eaf5ae87195 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:10.935163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bfb4b13b-29b6-4b33-a9ce-5eaf5ae87195 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:10.936333 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bfb4b13b-29b6-4b33-a9ce-5eaf5ae87195 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:10.936996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bfb4b13b-29b6-4b33-a9ce-5eaf5ae87195 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:10.937613 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bfb4b13b-29b6-4b33-a9ce-5eaf5ae87195 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:11.084464 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bfb4b13b-29b6-4b33-a9ce-5eaf5ae87195 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:11.094121 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bfb4b13b-29b6-4b33-a9ce-5eaf5ae87195 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:11.095681 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bfb4b13b-29b6-4b33-a9ce-5eaf5ae87195 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:11.096892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bfb4b13b-29b6-4b33-a9ce-5eaf5ae87195 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.176619 Aug 28 01:22:11.097467 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1369/2514] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:10 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:11.263113 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f9a17078-c640-4b25-9a4c-aea20eed4f25 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:11.263780 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f9a17078-c640-4b25-9a4c-aea20eed4f25 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:11.271381 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9a17078-c640-4b25-9a4c-aea20eed4f25 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:11.271876 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9a17078-c640-4b25-9a4c-aea20eed4f25 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:11.272357 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9a17078-c640-4b25-9a4c-aea20eed4f25 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:11.273196 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9a17078-c640-4b25-9a4c-aea20eed4f25 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:11.273658 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9a17078-c640-4b25-9a4c-aea20eed4f25 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:11.274084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9a17078-c640-4b25-9a4c-aea20eed4f25 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:11.343687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f9a17078-c640-4b25-9a4c-aea20eed4f25 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:11.345160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-26476591-5ce5-43c3-8ddc-8a2488dac902 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:11.345779 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-26476591-5ce5-43c3-8ddc-8a2488dac902 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:11.352799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26476591-5ce5-43c3-8ddc-8a2488dac902 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:11.353368 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26476591-5ce5-43c3-8ddc-8a2488dac902 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:11.353624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f9a17078-c640-4b25-9a4c-aea20eed4f25 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.094368 Aug 28 01:22:11.354002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26476591-5ce5-43c3-8ddc-8a2488dac902 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:11.354134 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1146/2515] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:11 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:11.354549 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26476591-5ce5-43c3-8ddc-8a2488dac902 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:11.354984 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26476591-5ce5-43c3-8ddc-8a2488dac902 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:11.355405 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26476591-5ce5-43c3-8ddc-8a2488dac902 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:11.420437 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-26476591-5ce5-43c3-8ddc-8a2488dac902 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:11.429831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-26476591-5ce5-43c3-8ddc-8a2488dac902 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.088511 Aug 28 01:22:11.431428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1370/2516] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:11 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:11.737786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5e5f5c8f-5153-44ce-89b7-6476bf0f7627 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:11.738684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5e5f5c8f-5153-44ce-89b7-6476bf0f7627 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:11.747491 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e5f5c8f-5153-44ce-89b7-6476bf0f7627 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:11.748242 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e5f5c8f-5153-44ce-89b7-6476bf0f7627 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:11.748675 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e5f5c8f-5153-44ce-89b7-6476bf0f7627 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:11.749549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e5f5c8f-5153-44ce-89b7-6476bf0f7627 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:11.750098 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e5f5c8f-5153-44ce-89b7-6476bf0f7627 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:11.750799 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e5f5c8f-5153-44ce-89b7-6476bf0f7627 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:11.892054 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5e5f5c8f-5153-44ce-89b7-6476bf0f7627 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:11.902589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5e5f5c8f-5153-44ce-89b7-6476bf0f7627 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.169789 Aug 28 01:22:11.903156 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1147/2517] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:11 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:12.123561 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0f2364da-39e3-4fcd-b6d5-13cba34aeb40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:12.124168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0f2364da-39e3-4fcd-b6d5-13cba34aeb40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:12.131487 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f2364da-39e3-4fcd-b6d5-13cba34aeb40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:12.132061 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f2364da-39e3-4fcd-b6d5-13cba34aeb40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:12.132832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f2364da-39e3-4fcd-b6d5-13cba34aeb40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:12.133846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f2364da-39e3-4fcd-b6d5-13cba34aeb40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:12.134304 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f2364da-39e3-4fcd-b6d5-13cba34aeb40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:12.134876 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0f2364da-39e3-4fcd-b6d5-13cba34aeb40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:12.277280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0f2364da-39e3-4fcd-b6d5-13cba34aeb40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:12.286744 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0f2364da-39e3-4fcd-b6d5-13cba34aeb40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:12.288418 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0f2364da-39e3-4fcd-b6d5-13cba34aeb40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:12.289615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0f2364da-39e3-4fcd-b6d5-13cba34aeb40 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.170103 Aug 28 01:22:12.290428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1371/2518] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:12 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:12.374906 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0fe8cc67-b391-4cdf-b3f3-cd2a16210cf0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:12.375492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0fe8cc67-b391-4cdf-b3f3-cd2a16210cf0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:12.382215 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0fe8cc67-b391-4cdf-b3f3-cd2a16210cf0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:12.382697 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0fe8cc67-b391-4cdf-b3f3-cd2a16210cf0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:12.383121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0fe8cc67-b391-4cdf-b3f3-cd2a16210cf0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:12.383812 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0fe8cc67-b391-4cdf-b3f3-cd2a16210cf0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:12.384259 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0fe8cc67-b391-4cdf-b3f3-cd2a16210cf0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:12.384700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0fe8cc67-b391-4cdf-b3f3-cd2a16210cf0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:12.453684 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d365c64a-6f48-46a3-8b26-871fc6ef6def tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:12.454336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d365c64a-6f48-46a3-8b26-871fc6ef6def tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:12.454761 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0fe8cc67-b391-4cdf-b3f3-cd2a16210cf0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:12.460951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d365c64a-6f48-46a3-8b26-871fc6ef6def tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:12.461468 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d365c64a-6f48-46a3-8b26-871fc6ef6def tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:12.461923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d365c64a-6f48-46a3-8b26-871fc6ef6def tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:12.462611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d365c64a-6f48-46a3-8b26-871fc6ef6def tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:12.463065 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d365c64a-6f48-46a3-8b26-871fc6ef6def tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:12.463196 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0fe8cc67-b391-4cdf-b3f3-cd2a16210cf0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.092205 Aug 28 01:22:12.463486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d365c64a-6f48-46a3-8b26-871fc6ef6def tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:12.463957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1148/2519] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:12 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:12.773597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d365c64a-6f48-46a3-8b26-871fc6ef6def tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:12.782734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d365c64a-6f48-46a3-8b26-871fc6ef6def tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.333469 Aug 28 01:22:12.783359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1372/2520] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:12 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:12.923954 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c691a6fd-4176-4500-bc3a-31d4c27338d4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:12.924612 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c691a6fd-4176-4500-bc3a-31d4c27338d4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:12.931168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c691a6fd-4176-4500-bc3a-31d4c27338d4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:12.931658 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c691a6fd-4176-4500-bc3a-31d4c27338d4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:12.932088 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c691a6fd-4176-4500-bc3a-31d4c27338d4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:12.932789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c691a6fd-4176-4500-bc3a-31d4c27338d4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:12.933266 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c691a6fd-4176-4500-bc3a-31d4c27338d4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:12.933696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c691a6fd-4176-4500-bc3a-31d4c27338d4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:13.079222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c691a6fd-4176-4500-bc3a-31d4c27338d4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:13.088268 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c691a6fd-4176-4500-bc3a-31d4c27338d4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.168458 Aug 28 01:22:13.088875 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1149/2521] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:12 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:13.314332 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f229ea1b-6962-4804-bb2b-19091e9d59e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:13.314998 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f229ea1b-6962-4804-bb2b-19091e9d59e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:13.322082 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f229ea1b-6962-4804-bb2b-19091e9d59e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:13.322553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f229ea1b-6962-4804-bb2b-19091e9d59e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:13.322992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f229ea1b-6962-4804-bb2b-19091e9d59e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:13.323681 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f229ea1b-6962-4804-bb2b-19091e9d59e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:13.324124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f229ea1b-6962-4804-bb2b-19091e9d59e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:13.324555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f229ea1b-6962-4804-bb2b-19091e9d59e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:13.339748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:22:13.466559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f229ea1b-6962-4804-bb2b-19091e9d59e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:13.475738 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f229ea1b-6962-4804-bb2b-19091e9d59e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:13.477192 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f229ea1b-6962-4804-bb2b-19091e9d59e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:13.478365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f229ea1b-6962-4804-bb2b-19091e9d59e9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.168052 Aug 28 01:22:13.478995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1373/2522] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:13 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:13.484061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-feca717f-8135-4823-84b0-5b61add10fb0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:13.484854 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-feca717f-8135-4823-84b0-5b61add10fb0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:13.491789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-feca717f-8135-4823-84b0-5b61add10fb0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:13.492280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-feca717f-8135-4823-84b0-5b61add10fb0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:13.492703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-feca717f-8135-4823-84b0-5b61add10fb0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:13.493428 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-feca717f-8135-4823-84b0-5b61add10fb0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:13.493881 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-feca717f-8135-4823-84b0-5b61add10fb0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:13.494304 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-feca717f-8135-4823-84b0-5b61add10fb0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:13.559364 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-feca717f-8135-4823-84b0-5b61add10fb0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:13.568625 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-feca717f-8135-4823-84b0-5b61add10fb0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.088472 Aug 28 01:22:13.569148 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1150/2523] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:13 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:13.805907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b8dc271c-bcf0-4639-a522-9f8355b8f80a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:13.806657 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b8dc271c-bcf0-4639-a522-9f8355b8f80a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:13.813863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8dc271c-bcf0-4639-a522-9f8355b8f80a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:13.814337 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8dc271c-bcf0-4639-a522-9f8355b8f80a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:13.814761 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8dc271c-bcf0-4639-a522-9f8355b8f80a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:13.815436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8dc271c-bcf0-4639-a522-9f8355b8f80a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:13.816221 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8dc271c-bcf0-4639-a522-9f8355b8f80a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:13.816680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b8dc271c-bcf0-4639-a522-9f8355b8f80a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:13.887997 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b8dc271c-bcf0-4639-a522-9f8355b8f80a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:13.897386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b8dc271c-bcf0-4639-a522-9f8355b8f80a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.097071 Aug 28 01:22:13.897963 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1374/2524] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:13 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:14.110212 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cbffad16-56e1-4f6f-a4e1-dd94c3b14d53 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:14.110827 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cbffad16-56e1-4f6f-a4e1-dd94c3b14d53 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:14.117394 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cbffad16-56e1-4f6f-a4e1-dd94c3b14d53 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:14.118203 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cbffad16-56e1-4f6f-a4e1-dd94c3b14d53 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:14.118661 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cbffad16-56e1-4f6f-a4e1-dd94c3b14d53 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:14.119334 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cbffad16-56e1-4f6f-a4e1-dd94c3b14d53 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:14.119947 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cbffad16-56e1-4f6f-a4e1-dd94c3b14d53 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:14.120572 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cbffad16-56e1-4f6f-a4e1-dd94c3b14d53 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:14.298967 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-cbffad16-56e1-4f6f-a4e1-dd94c3b14d53 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:14.308395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cbffad16-56e1-4f6f-a4e1-dd94c3b14d53 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.202112 Aug 28 01:22:14.309221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1151/2525] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:14 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:14.502924 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-88b0e3b3-78c0-488b-a929-80f139507ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:14.502924 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-88b0e3b3-78c0-488b-a929-80f139507ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:14.509780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88b0e3b3-78c0-488b-a929-80f139507ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:14.510260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88b0e3b3-78c0-488b-a929-80f139507ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:14.510681 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88b0e3b3-78c0-488b-a929-80f139507ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:14.511391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88b0e3b3-78c0-488b-a929-80f139507ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:14.511836 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88b0e3b3-78c0-488b-a929-80f139507ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:14.512264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88b0e3b3-78c0-488b-a929-80f139507ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:14.590298 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-36a5d33b-c2ff-4b39-af10-978f07869bc1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:14.591586 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-36a5d33b-c2ff-4b39-af10-978f07869bc1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:14.598282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36a5d33b-c2ff-4b39-af10-978f07869bc1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:14.598829 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36a5d33b-c2ff-4b39-af10-978f07869bc1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:14.599300 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36a5d33b-c2ff-4b39-af10-978f07869bc1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:14.600001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36a5d33b-c2ff-4b39-af10-978f07869bc1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:14.601199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36a5d33b-c2ff-4b39-af10-978f07869bc1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:14.601628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36a5d33b-c2ff-4b39-af10-978f07869bc1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:14.691838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-36a5d33b-c2ff-4b39-af10-978f07869bc1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:14.703579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-36a5d33b-c2ff-4b39-af10-978f07869bc1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.117030 Aug 28 01:22:14.704168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1152/2526] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:14 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:14.743453 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-88b0e3b3-78c0-488b-a929-80f139507ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:14.753890 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-88b0e3b3-78c0-488b-a929-80f139507ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:14.755354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-88b0e3b3-78c0-488b-a929-80f139507ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:14.756524 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-88b0e3b3-78c0-488b-a929-80f139507ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.258180 Aug 28 01:22:14.757148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1375/2527] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:14 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:14.919925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3d01e87f-229f-485b-8f7b-2e7c5875cf30 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:14.921041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3d01e87f-229f-485b-8f7b-2e7c5875cf30 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:14.929598 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d01e87f-229f-485b-8f7b-2e7c5875cf30 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:14.930089 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d01e87f-229f-485b-8f7b-2e7c5875cf30 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:14.930516 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d01e87f-229f-485b-8f7b-2e7c5875cf30 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:14.931212 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d01e87f-229f-485b-8f7b-2e7c5875cf30 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:14.931651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d01e87f-229f-485b-8f7b-2e7c5875cf30 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:14.932097 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d01e87f-229f-485b-8f7b-2e7c5875cf30 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:14.996425 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3d01e87f-229f-485b-8f7b-2e7c5875cf30 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:15.005679 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3d01e87f-229f-485b-8f7b-2e7c5875cf30 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089445 Aug 28 01:22:15.006439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1153/2528] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:14 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:15.329979 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-75d336a7-fdcc-49fd-993c-fd5ba3465172 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:15.330585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-75d336a7-fdcc-49fd-993c-fd5ba3465172 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:15.337243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75d336a7-fdcc-49fd-993c-fd5ba3465172 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:15.337730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75d336a7-fdcc-49fd-993c-fd5ba3465172 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:15.338158 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75d336a7-fdcc-49fd-993c-fd5ba3465172 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:15.338846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75d336a7-fdcc-49fd-993c-fd5ba3465172 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:15.339280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75d336a7-fdcc-49fd-993c-fd5ba3465172 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:15.339694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75d336a7-fdcc-49fd-993c-fd5ba3465172 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:15.479629 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-75d336a7-fdcc-49fd-993c-fd5ba3465172 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:15.490154 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-75d336a7-fdcc-49fd-993c-fd5ba3465172 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.163701 Aug 28 01:22:15.490952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1376/2529] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:15 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:15.725942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7b257d0e-80a5-4794-974b-d47cae4b4800 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:15.726580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7b257d0e-80a5-4794-974b-d47cae4b4800 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:15.733823 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7b257d0e-80a5-4794-974b-d47cae4b4800 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:15.734319 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7b257d0e-80a5-4794-974b-d47cae4b4800 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:15.734852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7b257d0e-80a5-4794-974b-d47cae4b4800 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:15.735464 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7b257d0e-80a5-4794-974b-d47cae4b4800 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:15.735917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7b257d0e-80a5-4794-974b-d47cae4b4800 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:15.736342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7b257d0e-80a5-4794-974b-d47cae4b4800 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:15.786840 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d4d19f27-f063-4c73-b821-8938f24e82fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:15.787745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d4d19f27-f063-4c73-b821-8938f24e82fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:15.797939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4d19f27-f063-4c73-b821-8938f24e82fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:15.798559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4d19f27-f063-4c73-b821-8938f24e82fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:15.799161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4d19f27-f063-4c73-b821-8938f24e82fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:15.800294 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4d19f27-f063-4c73-b821-8938f24e82fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:15.801008 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4d19f27-f063-4c73-b821-8938f24e82fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:15.801519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d4d19f27-f063-4c73-b821-8938f24e82fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:15.812216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7b257d0e-80a5-4794-974b-d47cae4b4800 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:15.822219 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7b257d0e-80a5-4794-974b-d47cae4b4800 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.100135 Aug 28 01:22:15.835616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1154/2530] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:15 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:16.022572 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d4d19f27-f063-4c73-b821-8938f24e82fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:16.028419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f88bca67-e6f7-40db-bf81-a13ee3fb6f4a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:16.029062 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f88bca67-e6f7-40db-bf81-a13ee3fb6f4a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:16.031603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d4d19f27-f063-4c73-b821-8938f24e82fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:16.033056 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d4d19f27-f063-4c73-b821-8938f24e82fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:16.034255 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d4d19f27-f063-4c73-b821-8938f24e82fa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.253945 Aug 28 01:22:16.034830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1377/2531] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:15 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:16.035235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f88bca67-e6f7-40db-bf81-a13ee3fb6f4a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:16.035769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f88bca67-e6f7-40db-bf81-a13ee3fb6f4a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:16.036211 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f88bca67-e6f7-40db-bf81-a13ee3fb6f4a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:16.036900 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f88bca67-e6f7-40db-bf81-a13ee3fb6f4a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:16.037363 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f88bca67-e6f7-40db-bf81-a13ee3fb6f4a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:16.037778 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f88bca67-e6f7-40db-bf81-a13ee3fb6f4a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:16.111713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f88bca67-e6f7-40db-bf81-a13ee3fb6f4a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:16.120787 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f88bca67-e6f7-40db-bf81-a13ee3fb6f4a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.096235 Aug 28 01:22:16.121532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1155/2532] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:16 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:22:16.512231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1d28095b-ef8b-46f7-83e4-99dd0565905c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:16.512922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1d28095b-ef8b-46f7-83e4-99dd0565905c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:16.519036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1d28095b-ef8b-46f7-83e4-99dd0565905c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:16.519601 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1d28095b-ef8b-46f7-83e4-99dd0565905c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:16.520032 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1d28095b-ef8b-46f7-83e4-99dd0565905c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:16.520706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1d28095b-ef8b-46f7-83e4-99dd0565905c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:16.521197 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1d28095b-ef8b-46f7-83e4-99dd0565905c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:16.521575 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1d28095b-ef8b-46f7-83e4-99dd0565905c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:16.655785 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1d28095b-ef8b-46f7-83e4-99dd0565905c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:16.664073 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1d28095b-ef8b-46f7-83e4-99dd0565905c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.155729 Aug 28 01:22:16.664716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1378/2533] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:16 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:16.847192 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a5cd9558-6717-460f-a302-2e1c7d9742e9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:16.848004 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a5cd9558-6717-460f-a302-2e1c7d9742e9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:16.854843 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5cd9558-6717-460f-a302-2e1c7d9742e9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:16.855315 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5cd9558-6717-460f-a302-2e1c7d9742e9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:16.855746 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5cd9558-6717-460f-a302-2e1c7d9742e9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:16.856421 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5cd9558-6717-460f-a302-2e1c7d9742e9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:16.856881 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5cd9558-6717-460f-a302-2e1c7d9742e9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:16.857331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5cd9558-6717-460f-a302-2e1c7d9742e9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:16.979222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a5cd9558-6717-460f-a302-2e1c7d9742e9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:16.988673 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a5cd9558-6717-460f-a302-2e1c7d9742e9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.147611 Aug 28 01:22:16.989210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1156/2534] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:16 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:17.063122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8100b32c-d2c6-493d-9fed-1403948bbf60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:17.064246 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8100b32c-d2c6-493d-9fed-1403948bbf60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:17.072590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8100b32c-d2c6-493d-9fed-1403948bbf60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:17.073115 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8100b32c-d2c6-493d-9fed-1403948bbf60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:17.073549 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8100b32c-d2c6-493d-9fed-1403948bbf60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:17.074261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8100b32c-d2c6-493d-9fed-1403948bbf60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:17.074709 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8100b32c-d2c6-493d-9fed-1403948bbf60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:17.075136 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8100b32c-d2c6-493d-9fed-1403948bbf60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:17.144860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f1dab5ee-4fe5-45d9-8f92-c2e5750b5bff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:17.145603 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f1dab5ee-4fe5-45d9-8f92-c2e5750b5bff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:17.152340 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1dab5ee-4fe5-45d9-8f92-c2e5750b5bff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:17.152837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1dab5ee-4fe5-45d9-8f92-c2e5750b5bff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:17.153348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1dab5ee-4fe5-45d9-8f92-c2e5750b5bff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:17.154049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1dab5ee-4fe5-45d9-8f92-c2e5750b5bff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:17.154500 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1dab5ee-4fe5-45d9-8f92-c2e5750b5bff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:17.154925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1dab5ee-4fe5-45d9-8f92-c2e5750b5bff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:17.236815 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f1dab5ee-4fe5-45d9-8f92-c2e5750b5bff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:17.245796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f1dab5ee-4fe5-45d9-8f92-c2e5750b5bff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.107167 Aug 28 01:22:17.246328 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1157/2535] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:17 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:22:17.266278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8100b32c-d2c6-493d-9fed-1403948bbf60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:17.275254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8100b32c-d2c6-493d-9fed-1403948bbf60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:17.276688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8100b32c-d2c6-493d-9fed-1403948bbf60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:17.277884 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8100b32c-d2c6-493d-9fed-1403948bbf60 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.222241 Aug 28 01:22:17.278427 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1379/2536] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:17 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:17.688619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0586682f-c404-4e50-b35b-39d743799e99 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:17.689485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0586682f-c404-4e50-b35b-39d743799e99 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:17.696739 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0586682f-c404-4e50-b35b-39d743799e99 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:17.697282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0586682f-c404-4e50-b35b-39d743799e99 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:17.697715 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0586682f-c404-4e50-b35b-39d743799e99 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:17.698402 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0586682f-c404-4e50-b35b-39d743799e99 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:17.698850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0586682f-c404-4e50-b35b-39d743799e99 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:17.699253 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0586682f-c404-4e50-b35b-39d743799e99 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:17.840039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0586682f-c404-4e50-b35b-39d743799e99 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:17.848557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0586682f-c404-4e50-b35b-39d743799e99 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.165765 Aug 28 01:22:17.849373 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1158/2537] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:17 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:18.012223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-98f83b36-5c69-4eb8-a5d3-88ee353158b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:18.012832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-98f83b36-5c69-4eb8-a5d3-88ee353158b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:18.019002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98f83b36-5c69-4eb8-a5d3-88ee353158b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:18.019526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98f83b36-5c69-4eb8-a5d3-88ee353158b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:18.019968 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98f83b36-5c69-4eb8-a5d3-88ee353158b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:18.020644 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98f83b36-5c69-4eb8-a5d3-88ee353158b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:18.021127 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98f83b36-5c69-4eb8-a5d3-88ee353158b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:18.021551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98f83b36-5c69-4eb8-a5d3-88ee353158b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:18.093280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-98f83b36-5c69-4eb8-a5d3-88ee353158b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:18.102229 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-98f83b36-5c69-4eb8-a5d3-88ee353158b1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.094142 Aug 28 01:22:18.102800 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1380/2538] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:18 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:18.267329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b930b0e0-9548-4192-b07e-675fc555b034 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:18.267840 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b930b0e0-9548-4192-b07e-675fc555b034 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:18.275073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b930b0e0-9548-4192-b07e-675fc555b034 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:18.275540 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b930b0e0-9548-4192-b07e-675fc555b034 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:18.275990 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b930b0e0-9548-4192-b07e-675fc555b034 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:18.276696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b930b0e0-9548-4192-b07e-675fc555b034 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:18.277188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b930b0e0-9548-4192-b07e-675fc555b034 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:18.277610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b930b0e0-9548-4192-b07e-675fc555b034 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:18.301279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-914945c9-e277-432e-bb83-7da7d4e21a74 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:18.301887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-914945c9-e277-432e-bb83-7da7d4e21a74 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:18.308340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-914945c9-e277-432e-bb83-7da7d4e21a74 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:18.308824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-914945c9-e277-432e-bb83-7da7d4e21a74 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:18.309282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-914945c9-e277-432e-bb83-7da7d4e21a74 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:18.309964 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-914945c9-e277-432e-bb83-7da7d4e21a74 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:18.310441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-914945c9-e277-432e-bb83-7da7d4e21a74 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:18.310836 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-914945c9-e277-432e-bb83-7da7d4e21a74 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:18.341983 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b930b0e0-9548-4192-b07e-675fc555b034 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:18.359500 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b930b0e0-9548-4192-b07e-675fc555b034 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.096062 Aug 28 01:22:18.360093 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1159/2539] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:18 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:18.456931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-914945c9-e277-432e-bb83-7da7d4e21a74 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:18.469474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-914945c9-e277-432e-bb83-7da7d4e21a74 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:18.471546 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-914945c9-e277-432e-bb83-7da7d4e21a74 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:18.473155 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-914945c9-e277-432e-bb83-7da7d4e21a74 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.175860 Aug 28 01:22:18.473712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1381/2540] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:18 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:18.873430 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a303b20a-0c50-4aff-a4ab-15a06c4d5706 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:18.874102 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a303b20a-0c50-4aff-a4ab-15a06c4d5706 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:18.883048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a303b20a-0c50-4aff-a4ab-15a06c4d5706 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:18.883867 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a303b20a-0c50-4aff-a4ab-15a06c4d5706 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:18.884595 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a303b20a-0c50-4aff-a4ab-15a06c4d5706 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:18.885723 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a303b20a-0c50-4aff-a4ab-15a06c4d5706 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:18.886480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a303b20a-0c50-4aff-a4ab-15a06c4d5706 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:18.887096 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a303b20a-0c50-4aff-a4ab-15a06c4d5706 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:19.042183 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a303b20a-0c50-4aff-a4ab-15a06c4d5706 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:19.044484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:22:19.051799 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a303b20a-0c50-4aff-a4ab-15a06c4d5706 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.184628 Aug 28 01:22:19.052391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1160/2541] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:18 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:19.125057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-da75bbfb-1649-4e55-ae0a-436a08090a1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:19.125653 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-da75bbfb-1649-4e55-ae0a-436a08090a1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:19.134128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da75bbfb-1649-4e55-ae0a-436a08090a1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:19.134780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da75bbfb-1649-4e55-ae0a-436a08090a1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:19.135298 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da75bbfb-1649-4e55-ae0a-436a08090a1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:19.136214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da75bbfb-1649-4e55-ae0a-436a08090a1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:19.136694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da75bbfb-1649-4e55-ae0a-436a08090a1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:19.137165 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da75bbfb-1649-4e55-ae0a-436a08090a1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:19.207861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-da75bbfb-1649-4e55-ae0a-436a08090a1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:19.220656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-da75bbfb-1649-4e55-ae0a-436a08090a1a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.101065 Aug 28 01:22:19.221334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1382/2542] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:19 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:19.383657 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6f674fe0-ea02-4d04-b809-89c58e129bb1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:19.384267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6f674fe0-ea02-4d04-b809-89c58e129bb1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:19.393119 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f674fe0-ea02-4d04-b809-89c58e129bb1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:19.393604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f674fe0-ea02-4d04-b809-89c58e129bb1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:19.394028 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f674fe0-ea02-4d04-b809-89c58e129bb1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:19.394785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f674fe0-ea02-4d04-b809-89c58e129bb1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:19.395231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f674fe0-ea02-4d04-b809-89c58e129bb1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:19.395641 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f674fe0-ea02-4d04-b809-89c58e129bb1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:19.464974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6f674fe0-ea02-4d04-b809-89c58e129bb1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:19.473824 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6f674fe0-ea02-4d04-b809-89c58e129bb1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.094487 Aug 28 01:22:19.474405 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1161/2543] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:19 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:19.495885 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d93cc0b0-8f6b-4bde-bbc0-2abfac2e07b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:19.496494 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d93cc0b0-8f6b-4bde-bbc0-2abfac2e07b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:19.502721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d93cc0b0-8f6b-4bde-bbc0-2abfac2e07b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:19.503212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d93cc0b0-8f6b-4bde-bbc0-2abfac2e07b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:19.503645 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d93cc0b0-8f6b-4bde-bbc0-2abfac2e07b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:19.504344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d93cc0b0-8f6b-4bde-bbc0-2abfac2e07b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:19.504798 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d93cc0b0-8f6b-4bde-bbc0-2abfac2e07b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:19.505261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d93cc0b0-8f6b-4bde-bbc0-2abfac2e07b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:19.643660 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d93cc0b0-8f6b-4bde-bbc0-2abfac2e07b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:19.653664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d93cc0b0-8f6b-4bde-bbc0-2abfac2e07b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:19.655433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d93cc0b0-8f6b-4bde-bbc0-2abfac2e07b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:19.657027 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d93cc0b0-8f6b-4bde-bbc0-2abfac2e07b5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.164680 Aug 28 01:22:19.657368 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1383/2544] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:19 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:20.073707 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-95c951c6-8fce-4f7f-afb9-f2e7c05d228d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:20.074312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-95c951c6-8fce-4f7f-afb9-f2e7c05d228d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:20.080584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95c951c6-8fce-4f7f-afb9-f2e7c05d228d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:20.081091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95c951c6-8fce-4f7f-afb9-f2e7c05d228d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:20.081525 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95c951c6-8fce-4f7f-afb9-f2e7c05d228d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:20.082202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95c951c6-8fce-4f7f-afb9-f2e7c05d228d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:20.082647 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95c951c6-8fce-4f7f-afb9-f2e7c05d228d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:20.083060 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95c951c6-8fce-4f7f-afb9-f2e7c05d228d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:20.222170 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-95c951c6-8fce-4f7f-afb9-f2e7c05d228d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:20.234335 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-95c951c6-8fce-4f7f-afb9-f2e7c05d228d tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.164383 Aug 28 01:22:20.235069 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1162/2545] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:20 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:20.242018 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c3f3c4dc-ac16-4904-841d-1e47c0175d9b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:20.242583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c3f3c4dc-ac16-4904-841d-1e47c0175d9b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:20.249419 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3f3c4dc-ac16-4904-841d-1e47c0175d9b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:20.249900 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3f3c4dc-ac16-4904-841d-1e47c0175d9b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:20.250321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3f3c4dc-ac16-4904-841d-1e47c0175d9b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:20.251010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3f3c4dc-ac16-4904-841d-1e47c0175d9b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:20.251456 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3f3c4dc-ac16-4904-841d-1e47c0175d9b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:20.251869 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3f3c4dc-ac16-4904-841d-1e47c0175d9b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:20.327872 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c3f3c4dc-ac16-4904-841d-1e47c0175d9b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:20.335918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c3f3c4dc-ac16-4904-841d-1e47c0175d9b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.097741 Aug 28 01:22:20.336507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1384/2546] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:20 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:20.495295 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fc952439-b59f-4625-b84e-02d81e93c734 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:20.495904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fc952439-b59f-4625-b84e-02d81e93c734 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:20.502780 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc952439-b59f-4625-b84e-02d81e93c734 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:20.503270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc952439-b59f-4625-b84e-02d81e93c734 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:20.503733 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc952439-b59f-4625-b84e-02d81e93c734 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:20.504446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc952439-b59f-4625-b84e-02d81e93c734 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:20.504898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc952439-b59f-4625-b84e-02d81e93c734 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:20.505351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fc952439-b59f-4625-b84e-02d81e93c734 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:20.577688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fc952439-b59f-4625-b84e-02d81e93c734 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:20.586796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fc952439-b59f-4625-b84e-02d81e93c734 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.095260 Aug 28 01:22:20.587300 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1163/2547] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:20 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:20.679612 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-914dfb6f-e5d4-4a5a-a763-3acf1c54ed6c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:20.680215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-914dfb6f-e5d4-4a5a-a763-3acf1c54ed6c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:20.686895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-914dfb6f-e5d4-4a5a-a763-3acf1c54ed6c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:20.687370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-914dfb6f-e5d4-4a5a-a763-3acf1c54ed6c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:20.687793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-914dfb6f-e5d4-4a5a-a763-3acf1c54ed6c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:20.688486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-914dfb6f-e5d4-4a5a-a763-3acf1c54ed6c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:20.688917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-914dfb6f-e5d4-4a5a-a763-3acf1c54ed6c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:20.689370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-914dfb6f-e5d4-4a5a-a763-3acf1c54ed6c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:20.829120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-914dfb6f-e5d4-4a5a-a763-3acf1c54ed6c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:20.838947 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-914dfb6f-e5d4-4a5a-a763-3acf1c54ed6c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:20.840295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-914dfb6f-e5d4-4a5a-a763-3acf1c54ed6c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:20.841593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-914dfb6f-e5d4-4a5a-a763-3acf1c54ed6c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.165985 Aug 28 01:22:20.842156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1385/2548] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:20 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:21.256252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f4e6f0c2-6865-4297-b24a-7e9468357b12 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:21.256953 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f4e6f0c2-6865-4297-b24a-7e9468357b12 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:21.263333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4e6f0c2-6865-4297-b24a-7e9468357b12 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:21.263838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4e6f0c2-6865-4297-b24a-7e9468357b12 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:21.264255 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4e6f0c2-6865-4297-b24a-7e9468357b12 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:21.264997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4e6f0c2-6865-4297-b24a-7e9468357b12 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:21.265448 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4e6f0c2-6865-4297-b24a-7e9468357b12 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:21.265898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4e6f0c2-6865-4297-b24a-7e9468357b12 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:21.365586 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a853fd8b-8274-455b-9635-e8f69c4e87b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:21.366943 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a853fd8b-8274-455b-9635-e8f69c4e87b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:21.374944 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a853fd8b-8274-455b-9635-e8f69c4e87b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:21.375181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a853fd8b-8274-455b-9635-e8f69c4e87b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:21.375639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a853fd8b-8274-455b-9635-e8f69c4e87b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:21.376280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a853fd8b-8274-455b-9635-e8f69c4e87b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:21.376825 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a853fd8b-8274-455b-9635-e8f69c4e87b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:21.377405 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a853fd8b-8274-455b-9635-e8f69c4e87b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:21.439798 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f4e6f0c2-6865-4297-b24a-7e9468357b12 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:21.449035 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f4e6f0c2-6865-4297-b24a-7e9468357b12 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.196690 Aug 28 01:22:21.449646 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1164/2549] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:21 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:21.463407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a853fd8b-8274-455b-9635-e8f69c4e87b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:21.473203 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a853fd8b-8274-455b-9635-e8f69c4e87b6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.113182 Aug 28 01:22:21.473786 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1386/2550] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:21 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:21.608338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-95f3c619-e700-40ca-b0de-64c006d90322 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:21.609086 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-95f3c619-e700-40ca-b0de-64c006d90322 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:21.617993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95f3c619-e700-40ca-b0de-64c006d90322 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:21.618539 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95f3c619-e700-40ca-b0de-64c006d90322 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:21.618972 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95f3c619-e700-40ca-b0de-64c006d90322 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:21.619728 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95f3c619-e700-40ca-b0de-64c006d90322 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:21.620175 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95f3c619-e700-40ca-b0de-64c006d90322 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:21.620594 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-95f3c619-e700-40ca-b0de-64c006d90322 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:21.693451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-95f3c619-e700-40ca-b0de-64c006d90322 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:21.701938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-95f3c619-e700-40ca-b0de-64c006d90322 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.097584 Aug 28 01:22:21.702489 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1165/2551] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:21 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:21.865470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a303bdc1-f9b9-454f-8596-e1222f38e732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:21.866123 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a303bdc1-f9b9-454f-8596-e1222f38e732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:21.872799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a303bdc1-f9b9-454f-8596-e1222f38e732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:21.873320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a303bdc1-f9b9-454f-8596-e1222f38e732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:21.873750 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a303bdc1-f9b9-454f-8596-e1222f38e732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:21.874427 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a303bdc1-f9b9-454f-8596-e1222f38e732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:21.874882 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a303bdc1-f9b9-454f-8596-e1222f38e732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:21.875295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a303bdc1-f9b9-454f-8596-e1222f38e732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:22.011970 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a303bdc1-f9b9-454f-8596-e1222f38e732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:22.021997 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a303bdc1-f9b9-454f-8596-e1222f38e732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:22.023451 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a303bdc1-f9b9-454f-8596-e1222f38e732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:22.024615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a303bdc1-f9b9-454f-8596-e1222f38e732 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.163431 Aug 28 01:22:22.025223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1387/2552] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:21 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:22.470644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8ebc069a-d557-49c1-8b93-4414bc363084 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:22.471285 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8ebc069a-d557-49c1-8b93-4414bc363084 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:22.478071 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ebc069a-d557-49c1-8b93-4414bc363084 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:22.478627 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ebc069a-d557-49c1-8b93-4414bc363084 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:22.479112 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ebc069a-d557-49c1-8b93-4414bc363084 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:22.480244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ebc069a-d557-49c1-8b93-4414bc363084 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:22.480760 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ebc069a-d557-49c1-8b93-4414bc363084 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:22.481215 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ebc069a-d557-49c1-8b93-4414bc363084 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:22.494466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5a439de7-832a-43df-827b-eca25c4459b2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:22.495082 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5a439de7-832a-43df-827b-eca25c4459b2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:22.503664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5a439de7-832a-43df-827b-eca25c4459b2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:22.504291 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5a439de7-832a-43df-827b-eca25c4459b2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:22.504729 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5a439de7-832a-43df-827b-eca25c4459b2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:22.505585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5a439de7-832a-43df-827b-eca25c4459b2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:22.506441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5a439de7-832a-43df-827b-eca25c4459b2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:22.507722 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5a439de7-832a-43df-827b-eca25c4459b2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:22.591376 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5a439de7-832a-43df-827b-eca25c4459b2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:22.601023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5a439de7-832a-43df-827b-eca25c4459b2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.110380 Aug 28 01:22:22.601616 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1388/2553] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:22 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:22.654390 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8ebc069a-d557-49c1-8b93-4414bc363084 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:22.662925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8ebc069a-d557-49c1-8b93-4414bc363084 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.196512 Aug 28 01:22:22.663504 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1166/2554] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:22 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:22.722536 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3da1c8c3-7c4a-4bf4-9cd2-270cf0f5db9e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:22.723052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3da1c8c3-7c4a-4bf4-9cd2-270cf0f5db9e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:22.729311 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3da1c8c3-7c4a-4bf4-9cd2-270cf0f5db9e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:22.729807 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3da1c8c3-7c4a-4bf4-9cd2-270cf0f5db9e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:22.730243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3da1c8c3-7c4a-4bf4-9cd2-270cf0f5db9e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:22.730948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3da1c8c3-7c4a-4bf4-9cd2-270cf0f5db9e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:22.731396 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3da1c8c3-7c4a-4bf4-9cd2-270cf0f5db9e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:22.731816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3da1c8c3-7c4a-4bf4-9cd2-270cf0f5db9e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:22.795047 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3da1c8c3-7c4a-4bf4-9cd2-270cf0f5db9e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:22.803670 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3da1c8c3-7c4a-4bf4-9cd2-270cf0f5db9e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.085111 Aug 28 01:22:22.804265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1389/2555] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:22 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:23.048624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fd03113e-ffe5-4f61-bf0a-19a8b0fdfd1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:23.049173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fd03113e-ffe5-4f61-bf0a-19a8b0fdfd1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:23.057187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd03113e-ffe5-4f61-bf0a-19a8b0fdfd1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:23.057675 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd03113e-ffe5-4f61-bf0a-19a8b0fdfd1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:23.058095 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd03113e-ffe5-4f61-bf0a-19a8b0fdfd1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:23.058804 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd03113e-ffe5-4f61-bf0a-19a8b0fdfd1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:23.059253 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd03113e-ffe5-4f61-bf0a-19a8b0fdfd1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:23.059674 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd03113e-ffe5-4f61-bf0a-19a8b0fdfd1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:23.210914 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fd03113e-ffe5-4f61-bf0a-19a8b0fdfd1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:23.223795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fd03113e-ffe5-4f61-bf0a-19a8b0fdfd1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:23.225261 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fd03113e-ffe5-4f61-bf0a-19a8b0fdfd1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:23.226441 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fd03113e-ffe5-4f61-bf0a-19a8b0fdfd1f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.181951 Aug 28 01:22:23.227009 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1167/2556] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:23 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 01:22:23.626203 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4755ab27-b9bd-4bc3-b916-d1fdd51830bf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:23.626986 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4755ab27-b9bd-4bc3-b916-d1fdd51830bf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:23.635115 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4755ab27-b9bd-4bc3-b916-d1fdd51830bf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:23.635585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4755ab27-b9bd-4bc3-b916-d1fdd51830bf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:23.636014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4755ab27-b9bd-4bc3-b916-d1fdd51830bf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:23.636786 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4755ab27-b9bd-4bc3-b916-d1fdd51830bf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:23.637177 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4755ab27-b9bd-4bc3-b916-d1fdd51830bf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:23.637593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4755ab27-b9bd-4bc3-b916-d1fdd51830bf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:23.640182 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:22:23.687210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bc11d355-9396-4c53-9d07-04a7a8fb8149 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:23.688109 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bc11d355-9396-4c53-9d07-04a7a8fb8149 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:23.696338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc11d355-9396-4c53-9d07-04a7a8fb8149 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:23.696914 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc11d355-9396-4c53-9d07-04a7a8fb8149 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:23.697447 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc11d355-9396-4c53-9d07-04a7a8fb8149 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:23.698280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc11d355-9396-4c53-9d07-04a7a8fb8149 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:23.698803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc11d355-9396-4c53-9d07-04a7a8fb8149 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:23.699301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc11d355-9396-4c53-9d07-04a7a8fb8149 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:23.708728 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4755ab27-b9bd-4bc3-b916-d1fdd51830bf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:23.717254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4755ab27-b9bd-4bc3-b916-d1fdd51830bf tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.097511 Aug 28 01:22:23.717796 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1390/2557] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:23 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:23.825783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-79631fd2-596c-4cca-a099-e547cb7c61ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:23.826428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-79631fd2-596c-4cca-a099-e547cb7c61ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:23.833438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79631fd2-596c-4cca-a099-e547cb7c61ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:23.833912 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bc11d355-9396-4c53-9d07-04a7a8fb8149 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:23.834088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79631fd2-596c-4cca-a099-e547cb7c61ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:23.834341 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79631fd2-596c-4cca-a099-e547cb7c61ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:23.835034 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79631fd2-596c-4cca-a099-e547cb7c61ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:23.835486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79631fd2-596c-4cca-a099-e547cb7c61ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:23.835907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79631fd2-596c-4cca-a099-e547cb7c61ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:23.842243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bc11d355-9396-4c53-9d07-04a7a8fb8149 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.160551 Aug 28 01:22:23.842815 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1168/2558] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:23 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:22:23.908460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-79631fd2-596c-4cca-a099-e547cb7c61ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:23.922253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-79631fd2-596c-4cca-a099-e547cb7c61ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.100101 Aug 28 01:22:23.923161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1391/2559] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:23 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:24.250310 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-29dbb526-800f-4acd-a301-40c6d8e400cd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:24.250818 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-29dbb526-800f-4acd-a301-40c6d8e400cd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:24.257308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-29dbb526-800f-4acd-a301-40c6d8e400cd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:24.257786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-29dbb526-800f-4acd-a301-40c6d8e400cd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:24.258210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-29dbb526-800f-4acd-a301-40c6d8e400cd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:24.258886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-29dbb526-800f-4acd-a301-40c6d8e400cd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:24.259353 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-29dbb526-800f-4acd-a301-40c6d8e400cd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:24.259745 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-29dbb526-800f-4acd-a301-40c6d8e400cd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:24.400894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-29dbb526-800f-4acd-a301-40c6d8e400cd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:24.410165 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-29dbb526-800f-4acd-a301-40c6d8e400cd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:24.411740 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-29dbb526-800f-4acd-a301-40c6d8e400cd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:24.412873 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-29dbb526-800f-4acd-a301-40c6d8e400cd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.166640 Aug 28 01:22:24.413459 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1169/2560] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:24 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:24.739344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e89f4f7a-4670-49f8-86da-1cf12f769d43 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:24.739979 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e89f4f7a-4670-49f8-86da-1cf12f769d43 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:24.747671 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e89f4f7a-4670-49f8-86da-1cf12f769d43 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:24.748309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e89f4f7a-4670-49f8-86da-1cf12f769d43 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:24.748846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e89f4f7a-4670-49f8-86da-1cf12f769d43 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:24.749895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e89f4f7a-4670-49f8-86da-1cf12f769d43 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:24.750523 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e89f4f7a-4670-49f8-86da-1cf12f769d43 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:24.751099 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e89f4f7a-4670-49f8-86da-1cf12f769d43 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:24.825498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e89f4f7a-4670-49f8-86da-1cf12f769d43 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:24.836601 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e89f4f7a-4670-49f8-86da-1cf12f769d43 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.101368 Aug 28 01:22:24.837212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1392/2561] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:24 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:24.864475 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f11fb61b-c373-43bd-9228-9f57b756a745 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:24.865110 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f11fb61b-c373-43bd-9228-9f57b756a745 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:24.872409 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f11fb61b-c373-43bd-9228-9f57b756a745 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:24.872910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f11fb61b-c373-43bd-9228-9f57b756a745 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:24.873351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f11fb61b-c373-43bd-9228-9f57b756a745 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:24.874045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f11fb61b-c373-43bd-9228-9f57b756a745 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:24.874482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f11fb61b-c373-43bd-9228-9f57b756a745 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:24.874890 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f11fb61b-c373-43bd-9228-9f57b756a745 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:24.945711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2b291876-f350-46bc-84c3-1510c380833b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:24.946316 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2b291876-f350-46bc-84c3-1510c380833b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:24.952302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2b291876-f350-46bc-84c3-1510c380833b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:24.952785 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2b291876-f350-46bc-84c3-1510c380833b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:24.953256 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2b291876-f350-46bc-84c3-1510c380833b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:24.953950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2b291876-f350-46bc-84c3-1510c380833b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:24.954392 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2b291876-f350-46bc-84c3-1510c380833b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:24.954906 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2b291876-f350-46bc-84c3-1510c380833b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:25.024260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2b291876-f350-46bc-84c3-1510c380833b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:25.032817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2b291876-f350-46bc-84c3-1510c380833b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.092132 Aug 28 01:22:25.033431 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1393/2562] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:24 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:25.038852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f11fb61b-c373-43bd-9228-9f57b756a745 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:25.047207 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f11fb61b-c373-43bd-9228-9f57b756a745 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.186607 Aug 28 01:22:25.047765 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1170/2563] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:24 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:25.436727 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2f2291ef-9110-4278-bf75-cd302bbad7b6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:25.437403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2f2291ef-9110-4278-bf75-cd302bbad7b6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:25.444242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f2291ef-9110-4278-bf75-cd302bbad7b6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:25.444796 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f2291ef-9110-4278-bf75-cd302bbad7b6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:25.451526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f2291ef-9110-4278-bf75-cd302bbad7b6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:25.451526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f2291ef-9110-4278-bf75-cd302bbad7b6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:25.451526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f2291ef-9110-4278-bf75-cd302bbad7b6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:25.451526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f2291ef-9110-4278-bf75-cd302bbad7b6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:25.580916 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2f2291ef-9110-4278-bf75-cd302bbad7b6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:25.589709 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2f2291ef-9110-4278-bf75-cd302bbad7b6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:25.591148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2f2291ef-9110-4278-bf75-cd302bbad7b6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:25.592308 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2f2291ef-9110-4278-bf75-cd302bbad7b6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.160400 Aug 28 01:22:25.593064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1394/2564] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:25 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:25.860145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-86a23530-6d1d-496f-af3a-673d2514c030 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:25.860868 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-86a23530-6d1d-496f-af3a-673d2514c030 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:25.869986 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86a23530-6d1d-496f-af3a-673d2514c030 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:25.870826 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86a23530-6d1d-496f-af3a-673d2514c030 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:25.871227 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86a23530-6d1d-496f-af3a-673d2514c030 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:25.872210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86a23530-6d1d-496f-af3a-673d2514c030 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:25.872819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86a23530-6d1d-496f-af3a-673d2514c030 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:25.873444 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86a23530-6d1d-496f-af3a-673d2514c030 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:25.942451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-86a23530-6d1d-496f-af3a-673d2514c030 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:25.950904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-86a23530-6d1d-496f-af3a-673d2514c030 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.096071 Aug 28 01:22:25.951464 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1171/2565] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:25 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:26.056056 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e1d81091-b763-4e99-8226-eb20e48be535 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:26.056727 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e1d81091-b763-4e99-8226-eb20e48be535 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:26.064873 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1d81091-b763-4e99-8226-eb20e48be535 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:26.065376 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1d81091-b763-4e99-8226-eb20e48be535 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:26.066142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1d81091-b763-4e99-8226-eb20e48be535 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:26.066959 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1d81091-b763-4e99-8226-eb20e48be535 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:26.067453 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1d81091-b763-4e99-8226-eb20e48be535 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:26.067872 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1d81091-b763-4e99-8226-eb20e48be535 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:26.068876 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6cfaac40-f81b-4bc0-a167-c358e4f15c56 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:26.069529 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6cfaac40-f81b-4bc0-a167-c358e4f15c56 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:26.075720 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6cfaac40-f81b-4bc0-a167-c358e4f15c56 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:26.076233 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6cfaac40-f81b-4bc0-a167-c358e4f15c56 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:26.076665 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6cfaac40-f81b-4bc0-a167-c358e4f15c56 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:26.077373 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6cfaac40-f81b-4bc0-a167-c358e4f15c56 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:26.077812 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6cfaac40-f81b-4bc0-a167-c358e4f15c56 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:26.078226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6cfaac40-f81b-4bc0-a167-c358e4f15c56 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:26.137804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e1d81091-b763-4e99-8226-eb20e48be535 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:26.146128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e1d81091-b763-4e99-8226-eb20e48be535 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.095666 Aug 28 01:22:26.146675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1395/2566] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:26 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:26.233910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6cfaac40-f81b-4bc0-a167-c358e4f15c56 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:26.242525 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6cfaac40-f81b-4bc0-a167-c358e4f15c56 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.178262 Aug 28 01:22:26.243102 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1172/2567] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:26 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:22:26.619593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-42a9d033-d0ef-4e7c-a6cd-dc0a639e1631 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:26.620541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-42a9d033-d0ef-4e7c-a6cd-dc0a639e1631 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:26.629880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42a9d033-d0ef-4e7c-a6cd-dc0a639e1631 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:26.630365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42a9d033-d0ef-4e7c-a6cd-dc0a639e1631 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:26.630793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42a9d033-d0ef-4e7c-a6cd-dc0a639e1631 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:26.631486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42a9d033-d0ef-4e7c-a6cd-dc0a639e1631 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:26.631932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42a9d033-d0ef-4e7c-a6cd-dc0a639e1631 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:26.632360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42a9d033-d0ef-4e7c-a6cd-dc0a639e1631 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:26.781804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-42a9d033-d0ef-4e7c-a6cd-dc0a639e1631 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:26.791061 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-42a9d033-d0ef-4e7c-a6cd-dc0a639e1631 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:26.792495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-42a9d033-d0ef-4e7c-a6cd-dc0a639e1631 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:26.793687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-42a9d033-d0ef-4e7c-a6cd-dc0a639e1631 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.180840 Aug 28 01:22:26.794251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1396/2568] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:26 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:26.972554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a01e18c5-ead2-4880-af67-9d8b158248c6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:26.973222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a01e18c5-ead2-4880-af67-9d8b158248c6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:26.980939 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a01e18c5-ead2-4880-af67-9d8b158248c6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:26.981451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a01e18c5-ead2-4880-af67-9d8b158248c6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:26.981907 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a01e18c5-ead2-4880-af67-9d8b158248c6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:26.982592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a01e18c5-ead2-4880-af67-9d8b158248c6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:26.983062 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a01e18c5-ead2-4880-af67-9d8b158248c6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:26.983479 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a01e18c5-ead2-4880-af67-9d8b158248c6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:27.048655 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a01e18c5-ead2-4880-af67-9d8b158248c6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:27.058150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a01e18c5-ead2-4880-af67-9d8b158248c6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.089505 Aug 28 01:22:27.058960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1173/2569] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:26 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:27.168043 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a4ebedc1-ae09-404f-9a27-199dbd247abd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:27.168675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a4ebedc1-ae09-404f-9a27-199dbd247abd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:27.175756 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4ebedc1-ae09-404f-9a27-199dbd247abd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:27.176239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4ebedc1-ae09-404f-9a27-199dbd247abd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:27.176671 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4ebedc1-ae09-404f-9a27-199dbd247abd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:27.177395 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4ebedc1-ae09-404f-9a27-199dbd247abd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:27.177844 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4ebedc1-ae09-404f-9a27-199dbd247abd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:27.178393 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4ebedc1-ae09-404f-9a27-199dbd247abd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:27.245195 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a4ebedc1-ae09-404f-9a27-199dbd247abd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:27.253654 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a4ebedc1-ae09-404f-9a27-199dbd247abd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089586 Aug 28 01:22:27.254212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1397/2570] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:27 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:27.262759 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-58ca6dc0-e32b-4b29-966a-e194893e3d20 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:27.263369 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-58ca6dc0-e32b-4b29-966a-e194893e3d20 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:27.269549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-58ca6dc0-e32b-4b29-966a-e194893e3d20 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:27.270022 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-58ca6dc0-e32b-4b29-966a-e194893e3d20 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:27.270457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-58ca6dc0-e32b-4b29-966a-e194893e3d20 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:27.271137 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-58ca6dc0-e32b-4b29-966a-e194893e3d20 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:27.271578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-58ca6dc0-e32b-4b29-966a-e194893e3d20 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:27.271993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-58ca6dc0-e32b-4b29-966a-e194893e3d20 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:27.405599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-58ca6dc0-e32b-4b29-966a-e194893e3d20 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:27.414149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-58ca6dc0-e32b-4b29-966a-e194893e3d20 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.155223 Aug 28 01:22:27.414714 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1174/2571] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:27 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:22:27.818901 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-81406b63-6424-4e37-9b15-0b0a830a6b61 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:27.819478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-81406b63-6424-4e37-9b15-0b0a830a6b61 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:27.826810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81406b63-6424-4e37-9b15-0b0a830a6b61 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:27.827327 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81406b63-6424-4e37-9b15-0b0a830a6b61 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:27.827754 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81406b63-6424-4e37-9b15-0b0a830a6b61 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:27.828471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81406b63-6424-4e37-9b15-0b0a830a6b61 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:27.828927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81406b63-6424-4e37-9b15-0b0a830a6b61 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:27.829387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81406b63-6424-4e37-9b15-0b0a830a6b61 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:27.977205 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-81406b63-6424-4e37-9b15-0b0a830a6b61 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:27.990360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-81406b63-6424-4e37-9b15-0b0a830a6b61 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:27.992562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-81406b63-6424-4e37-9b15-0b0a830a6b61 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:27.994283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-81406b63-6424-4e37-9b15-0b0a830a6b61 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.180414 Aug 28 01:22:27.995105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1398/2572] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:27 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:28.083025 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8fac07c0-b9cc-4daf-a0a6-3b274536ce57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:28.083603 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8fac07c0-b9cc-4daf-a0a6-3b274536ce57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:28.091089 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fac07c0-b9cc-4daf-a0a6-3b274536ce57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:28.091641 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fac07c0-b9cc-4daf-a0a6-3b274536ce57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:28.092125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fac07c0-b9cc-4daf-a0a6-3b274536ce57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:28.092858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fac07c0-b9cc-4daf-a0a6-3b274536ce57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:28.093429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fac07c0-b9cc-4daf-a0a6-3b274536ce57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:28.093910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fac07c0-b9cc-4daf-a0a6-3b274536ce57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:28.157970 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8fac07c0-b9cc-4daf-a0a6-3b274536ce57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:28.168097 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8fac07c0-b9cc-4daf-a0a6-3b274536ce57 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.089499 Aug 28 01:22:28.168709 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1175/2573] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:28 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:28.278014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cf4c4bad-e403-4632-abf2-f3f1bb3eb1e0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:28.278786 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cf4c4bad-e403-4632-abf2-f3f1bb3eb1e0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:28.288186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf4c4bad-e403-4632-abf2-f3f1bb3eb1e0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:28.288664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf4c4bad-e403-4632-abf2-f3f1bb3eb1e0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:28.289122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf4c4bad-e403-4632-abf2-f3f1bb3eb1e0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:28.289806 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf4c4bad-e403-4632-abf2-f3f1bb3eb1e0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:28.290252 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf4c4bad-e403-4632-abf2-f3f1bb3eb1e0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:28.290683 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf4c4bad-e403-4632-abf2-f3f1bb3eb1e0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:28.352805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-cf4c4bad-e403-4632-abf2-f3f1bb3eb1e0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:28.361741 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cf4c4bad-e403-4632-abf2-f3f1bb3eb1e0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089093 Aug 28 01:22:28.362467 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1399/2574] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:28 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:28.436053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-dbd15c62-fa77-460f-92c4-b01a340c444e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:28.436648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-dbd15c62-fa77-460f-92c4-b01a340c444e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:28.443194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dbd15c62-fa77-460f-92c4-b01a340c444e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:28.443668 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dbd15c62-fa77-460f-92c4-b01a340c444e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:28.444091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dbd15c62-fa77-460f-92c4-b01a340c444e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:28.444767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dbd15c62-fa77-460f-92c4-b01a340c444e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:28.445250 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dbd15c62-fa77-460f-92c4-b01a340c444e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:28.445652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dbd15c62-fa77-460f-92c4-b01a340c444e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:28.588721 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-dbd15c62-fa77-460f-92c4-b01a340c444e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:28.597499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-dbd15c62-fa77-460f-92c4-b01a340c444e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.165370 Aug 28 01:22:28.598141 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1176/2575] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:28 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:29.024272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-be372704-63a6-4ad8-b59e-5e0fcc87b5e8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:29.024876 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-be372704-63a6-4ad8-b59e-5e0fcc87b5e8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:29.033499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-be372704-63a6-4ad8-b59e-5e0fcc87b5e8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:29.034139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-be372704-63a6-4ad8-b59e-5e0fcc87b5e8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:29.034696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-be372704-63a6-4ad8-b59e-5e0fcc87b5e8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:29.035715 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-be372704-63a6-4ad8-b59e-5e0fcc87b5e8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:29.036376 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-be372704-63a6-4ad8-b59e-5e0fcc87b5e8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:29.036988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-be372704-63a6-4ad8-b59e-5e0fcc87b5e8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:29.181783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-be372704-63a6-4ad8-b59e-5e0fcc87b5e8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:29.189527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-445ff172-3397-4893-b261-6dbe42c4de40 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:29.190124 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-445ff172-3397-4893-b261-6dbe42c4de40 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:29.192170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-be372704-63a6-4ad8-b59e-5e0fcc87b5e8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:29.193644 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-be372704-63a6-4ad8-b59e-5e0fcc87b5e8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:29.195243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-be372704-63a6-4ad8-b59e-5e0fcc87b5e8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.176829 Aug 28 01:22:29.196079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1400/2576] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:29 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:29.198795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-445ff172-3397-4893-b261-6dbe42c4de40 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:29.199368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-445ff172-3397-4893-b261-6dbe42c4de40 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:29.199902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-445ff172-3397-4893-b261-6dbe42c4de40 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:29.200932 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-445ff172-3397-4893-b261-6dbe42c4de40 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:29.201486 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-445ff172-3397-4893-b261-6dbe42c4de40 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:29.202251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-445ff172-3397-4893-b261-6dbe42c4de40 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:29.219609 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:22:29.267242 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-445ff172-3397-4893-b261-6dbe42c4de40 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:29.275547 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-445ff172-3397-4893-b261-6dbe42c4de40 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.089801 Aug 28 01:22:29.276044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1177/2577] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:29 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:29.385574 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-db3a6389-fe25-4b1b-a757-306e78dc35eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:29.386378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-db3a6389-fe25-4b1b-a757-306e78dc35eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:29.394465 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db3a6389-fe25-4b1b-a757-306e78dc35eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:29.395036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db3a6389-fe25-4b1b-a757-306e78dc35eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:29.395473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db3a6389-fe25-4b1b-a757-306e78dc35eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:29.396305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db3a6389-fe25-4b1b-a757-306e78dc35eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:29.396921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db3a6389-fe25-4b1b-a757-306e78dc35eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:29.397375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-db3a6389-fe25-4b1b-a757-306e78dc35eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:29.469573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-db3a6389-fe25-4b1b-a757-306e78dc35eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:29.477917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-db3a6389-fe25-4b1b-a757-306e78dc35eb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.096988 Aug 28 01:22:29.478499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1401/2578] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:29 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:29.622985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c8670c37-41c9-49e4-ba32-860ae97ba2ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:29.623993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c8670c37-41c9-49e4-ba32-860ae97ba2ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:29.633613 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8670c37-41c9-49e4-ba32-860ae97ba2ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:29.634099 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8670c37-41c9-49e4-ba32-860ae97ba2ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:29.634528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8670c37-41c9-49e4-ba32-860ae97ba2ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:29.635230 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8670c37-41c9-49e4-ba32-860ae97ba2ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:29.635669 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8670c37-41c9-49e4-ba32-860ae97ba2ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:29.636090 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8670c37-41c9-49e4-ba32-860ae97ba2ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:29.779514 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c8670c37-41c9-49e4-ba32-860ae97ba2ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:29.788183 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c8670c37-41c9-49e4-ba32-860ae97ba2ad tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.172199 Aug 28 01:22:29.788854 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1178/2579] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:29 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:30.226081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-79f8edb1-0cc0-43c3-b250-eaec79df0aaa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:30.226810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-79f8edb1-0cc0-43c3-b250-eaec79df0aaa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:30.233448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79f8edb1-0cc0-43c3-b250-eaec79df0aaa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:30.233927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79f8edb1-0cc0-43c3-b250-eaec79df0aaa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:30.234358 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79f8edb1-0cc0-43c3-b250-eaec79df0aaa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:30.235038 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79f8edb1-0cc0-43c3-b250-eaec79df0aaa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:30.235488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79f8edb1-0cc0-43c3-b250-eaec79df0aaa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:30.235907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-79f8edb1-0cc0-43c3-b250-eaec79df0aaa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:30.296015 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c8e7e334-b148-4926-ad77-01e93e960ef4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:30.296673 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c8e7e334-b148-4926-ad77-01e93e960ef4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:30.299037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:22:30.304582 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8e7e334-b148-4926-ad77-01e93e960ef4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:30.305096 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8e7e334-b148-4926-ad77-01e93e960ef4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:30.305537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8e7e334-b148-4926-ad77-01e93e960ef4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:30.306255 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8e7e334-b148-4926-ad77-01e93e960ef4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:30.306702 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8e7e334-b148-4926-ad77-01e93e960ef4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:30.307111 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8e7e334-b148-4926-ad77-01e93e960ef4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:30.380584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c8e7e334-b148-4926-ad77-01e93e960ef4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:30.389149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c8e7e334-b148-4926-ad77-01e93e960ef4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.096989 Aug 28 01:22:30.391822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1179/2580] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:30 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:30.403496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-79f8edb1-0cc0-43c3-b250-eaec79df0aaa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:30.415778 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-79f8edb1-0cc0-43c3-b250-eaec79df0aaa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:30.417246 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-79f8edb1-0cc0-43c3-b250-eaec79df0aaa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:30.418399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-79f8edb1-0cc0-43c3-b250-eaec79df0aaa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.198750 Aug 28 01:22:30.418994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1402/2581] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:30 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:30.501904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3f9b7d44-32a0-49a8-b333-f58d0bada6cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:30.502704 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3f9b7d44-32a0-49a8-b333-f58d0bada6cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:30.510400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3f9b7d44-32a0-49a8-b333-f58d0bada6cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:30.510889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3f9b7d44-32a0-49a8-b333-f58d0bada6cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:30.511314 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3f9b7d44-32a0-49a8-b333-f58d0bada6cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:30.512007 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3f9b7d44-32a0-49a8-b333-f58d0bada6cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:30.512481 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3f9b7d44-32a0-49a8-b333-f58d0bada6cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:30.512898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3f9b7d44-32a0-49a8-b333-f58d0bada6cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:30.580852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3f9b7d44-32a0-49a8-b333-f58d0bada6cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:30.590470 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3f9b7d44-32a0-49a8-b333-f58d0bada6cc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.094901 Aug 28 01:22:30.591043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1180/2582] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:30 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:22:30.810818 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8dc66b92-ac70-4bc4-9877-f137bfd8ec35 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:30.811480 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8dc66b92-ac70-4bc4-9877-f137bfd8ec35 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:30.821505 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8dc66b92-ac70-4bc4-9877-f137bfd8ec35 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:30.822322 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8dc66b92-ac70-4bc4-9877-f137bfd8ec35 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:30.823034 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8dc66b92-ac70-4bc4-9877-f137bfd8ec35 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:30.824157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8dc66b92-ac70-4bc4-9877-f137bfd8ec35 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:30.824905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8dc66b92-ac70-4bc4-9877-f137bfd8ec35 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:30.825655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8dc66b92-ac70-4bc4-9877-f137bfd8ec35 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:30.966870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8dc66b92-ac70-4bc4-9877-f137bfd8ec35 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:30.975407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8dc66b92-ac70-4bc4-9877-f137bfd8ec35 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.168858 Aug 28 01:22:30.975952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1403/2583] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:30 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:31.414333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2dafb68b-b95d-4753-92c8-1da7ef29a5c9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:31.415065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2dafb68b-b95d-4753-92c8-1da7ef29a5c9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:31.424645 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2dafb68b-b95d-4753-92c8-1da7ef29a5c9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:31.425581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2dafb68b-b95d-4753-92c8-1da7ef29a5c9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:31.426013 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2dafb68b-b95d-4753-92c8-1da7ef29a5c9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:31.426850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2dafb68b-b95d-4753-92c8-1da7ef29a5c9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:31.427336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2dafb68b-b95d-4753-92c8-1da7ef29a5c9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:31.427808 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2dafb68b-b95d-4753-92c8-1da7ef29a5c9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:31.442062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1e38a5fa-766c-4e59-bc98-2c08131ad926 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:31.442655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1e38a5fa-766c-4e59-bc98-2c08131ad926 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:31.449688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1e38a5fa-766c-4e59-bc98-2c08131ad926 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:31.450227 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1e38a5fa-766c-4e59-bc98-2c08131ad926 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:31.450763 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1e38a5fa-766c-4e59-bc98-2c08131ad926 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:31.451501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1e38a5fa-766c-4e59-bc98-2c08131ad926 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:31.451978 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1e38a5fa-766c-4e59-bc98-2c08131ad926 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:31.452375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1e38a5fa-766c-4e59-bc98-2c08131ad926 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:31.507606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2dafb68b-b95d-4753-92c8-1da7ef29a5c9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:31.519451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2dafb68b-b95d-4753-92c8-1da7ef29a5c9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.111077 Aug 28 01:22:31.520073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1181/2584] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:31 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:31.608160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1e38a5fa-766c-4e59-bc98-2c08131ad926 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:31.611418 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c38c54c1-09c4-461a-a5fa-e7dc1a15491f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:31.612025 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c38c54c1-09c4-461a-a5fa-e7dc1a15491f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:31.617163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1e38a5fa-766c-4e59-bc98-2c08131ad926 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:31.618084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c38c54c1-09c4-461a-a5fa-e7dc1a15491f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:31.618575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c38c54c1-09c4-461a-a5fa-e7dc1a15491f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:31.618700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1e38a5fa-766c-4e59-bc98-2c08131ad926 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:31.619000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c38c54c1-09c4-461a-a5fa-e7dc1a15491f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:31.619793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1e38a5fa-766c-4e59-bc98-2c08131ad926 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.182127 Aug 28 01:22:31.619904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c38c54c1-09c4-461a-a5fa-e7dc1a15491f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:31.620240 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c38c54c1-09c4-461a-a5fa-e7dc1a15491f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:31.620660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c38c54c1-09c4-461a-a5fa-e7dc1a15491f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:31.625153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1404/2585] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:31 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:31.686409 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c38c54c1-09c4-461a-a5fa-e7dc1a15491f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:31.695374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c38c54c1-09c4-461a-a5fa-e7dc1a15491f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.087821 Aug 28 01:22:31.695981 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1182/2586] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:31 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:31.997892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-76f9263e-3187-45a2-84f5-151f0bfa9f0b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:31.998531 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-76f9263e-3187-45a2-84f5-151f0bfa9f0b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:32.000380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:22:32.005902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76f9263e-3187-45a2-84f5-151f0bfa9f0b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:32.006386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76f9263e-3187-45a2-84f5-151f0bfa9f0b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:32.006818 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76f9263e-3187-45a2-84f5-151f0bfa9f0b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:32.007537 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76f9263e-3187-45a2-84f5-151f0bfa9f0b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:32.007968 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76f9263e-3187-45a2-84f5-151f0bfa9f0b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:32.008384 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76f9263e-3187-45a2-84f5-151f0bfa9f0b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:32.150025 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-76f9263e-3187-45a2-84f5-151f0bfa9f0b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:32.158340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-76f9263e-3187-45a2-84f5-151f0bfa9f0b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.165355 Aug 28 01:22:32.158900 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1405/2587] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:31 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:32.539899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b135200e-0e64-42de-b684-f72a6343ec89 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:32.540550 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b135200e-0e64-42de-b684-f72a6343ec89 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:32.547454 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b135200e-0e64-42de-b684-f72a6343ec89 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:32.547952 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b135200e-0e64-42de-b684-f72a6343ec89 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:32.548366 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b135200e-0e64-42de-b684-f72a6343ec89 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:32.549110 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b135200e-0e64-42de-b684-f72a6343ec89 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:32.549560 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b135200e-0e64-42de-b684-f72a6343ec89 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:32.549983 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b135200e-0e64-42de-b684-f72a6343ec89 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:32.620319 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b135200e-0e64-42de-b684-f72a6343ec89 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:32.632229 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b135200e-0e64-42de-b684-f72a6343ec89 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.095957 Aug 28 01:22:32.632977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1183/2588] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:32 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:32.642902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bc4ad119-df63-49bd-aea4-c8e2b1329c37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:32.643464 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bc4ad119-df63-49bd-aea4-c8e2b1329c37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:32.653073 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc4ad119-df63-49bd-aea4-c8e2b1329c37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:32.653609 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc4ad119-df63-49bd-aea4-c8e2b1329c37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:32.654182 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc4ad119-df63-49bd-aea4-c8e2b1329c37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:32.655309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc4ad119-df63-49bd-aea4-c8e2b1329c37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:32.656011 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc4ad119-df63-49bd-aea4-c8e2b1329c37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:32.656662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc4ad119-df63-49bd-aea4-c8e2b1329c37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:32.715797 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-622664d1-15a2-4a33-9b66-64821b8219bb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:32.716406 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-622664d1-15a2-4a33-9b66-64821b8219bb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:32.722939 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-622664d1-15a2-4a33-9b66-64821b8219bb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:32.723434 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-622664d1-15a2-4a33-9b66-64821b8219bb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:32.723867 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-622664d1-15a2-4a33-9b66-64821b8219bb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:32.724577 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-622664d1-15a2-4a33-9b66-64821b8219bb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:32.725472 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-622664d1-15a2-4a33-9b66-64821b8219bb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:32.726101 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-622664d1-15a2-4a33-9b66-64821b8219bb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:32.797499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-622664d1-15a2-4a33-9b66-64821b8219bb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:32.806049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-622664d1-15a2-4a33-9b66-64821b8219bb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.094125 Aug 28 01:22:32.806687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1184/2589] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:32 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:32.819778 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bc4ad119-df63-49bd-aea4-c8e2b1329c37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:32.822054 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:22:32.829643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bc4ad119-df63-49bd-aea4-c8e2b1329c37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:32.831060 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bc4ad119-df63-49bd-aea4-c8e2b1329c37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:32.832324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bc4ad119-df63-49bd-aea4-c8e2b1329c37 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.193475 Aug 28 01:22:32.832922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1406/2590] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:32 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:33.180359 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9b3cc8b4-a39b-407f-a9a4-aed77ab841e8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:33.181507 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9b3cc8b4-a39b-407f-a9a4-aed77ab841e8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:33.191736 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b3cc8b4-a39b-407f-a9a4-aed77ab841e8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:33.192222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b3cc8b4-a39b-407f-a9a4-aed77ab841e8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:33.192654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b3cc8b4-a39b-407f-a9a4-aed77ab841e8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:33.193389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b3cc8b4-a39b-407f-a9a4-aed77ab841e8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:33.193834 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b3cc8b4-a39b-407f-a9a4-aed77ab841e8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:33.194253 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b3cc8b4-a39b-407f-a9a4-aed77ab841e8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:33.346501 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9b3cc8b4-a39b-407f-a9a4-aed77ab841e8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:33.355272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9b3cc8b4-a39b-407f-a9a4-aed77ab841e8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.180132 Aug 28 01:22:33.355867 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1185/2591] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:33 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:33.655607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-48665319-30c0-410a-a482-f0112e27dea7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:33.656259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-48665319-30c0-410a-a482-f0112e27dea7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:33.663218 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48665319-30c0-410a-a482-f0112e27dea7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:33.663662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48665319-30c0-410a-a482-f0112e27dea7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:33.664092 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48665319-30c0-410a-a482-f0112e27dea7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:33.664771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48665319-30c0-410a-a482-f0112e27dea7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:33.665247 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48665319-30c0-410a-a482-f0112e27dea7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:33.665663 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48665319-30c0-410a-a482-f0112e27dea7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:33.729277 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-48665319-30c0-410a-a482-f0112e27dea7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:33.741988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-48665319-30c0-410a-a482-f0112e27dea7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.090308 Aug 28 01:22:33.742785 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1407/2592] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:33 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:33.828712 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ba751ff1-4f5d-4f4c-a740-f1ab2e97cdcd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:33.829368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ba751ff1-4f5d-4f4c-a740-f1ab2e97cdcd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:33.836321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba751ff1-4f5d-4f4c-a740-f1ab2e97cdcd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:33.836808 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba751ff1-4f5d-4f4c-a740-f1ab2e97cdcd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:33.837274 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba751ff1-4f5d-4f4c-a740-f1ab2e97cdcd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:33.837980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba751ff1-4f5d-4f4c-a740-f1ab2e97cdcd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:33.838419 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba751ff1-4f5d-4f4c-a740-f1ab2e97cdcd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:33.838842 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ba751ff1-4f5d-4f4c-a740-f1ab2e97cdcd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:33.855824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2358902e-e9cf-492a-a65b-06ad19b81489 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:33.856422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2358902e-e9cf-492a-a65b-06ad19b81489 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:33.863399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2358902e-e9cf-492a-a65b-06ad19b81489 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:33.863903 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2358902e-e9cf-492a-a65b-06ad19b81489 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:33.864319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2358902e-e9cf-492a-a65b-06ad19b81489 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:33.864993 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2358902e-e9cf-492a-a65b-06ad19b81489 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:33.865476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2358902e-e9cf-492a-a65b-06ad19b81489 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:33.865890 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2358902e-e9cf-492a-a65b-06ad19b81489 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:33.901750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ba751ff1-4f5d-4f4c-a740-f1ab2e97cdcd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:33.910522 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ba751ff1-4f5d-4f4c-a740-f1ab2e97cdcd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.085569 Aug 28 01:22:33.910983 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1186/2593] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:33 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:33.999082 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2358902e-e9cf-492a-a65b-06ad19b81489 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:34.008252 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2358902e-e9cf-492a-a65b-06ad19b81489 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:34.009757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2358902e-e9cf-492a-a65b-06ad19b81489 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:34.010945 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2358902e-e9cf-492a-a65b-06ad19b81489 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.159060 Aug 28 01:22:34.011495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1408/2594] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:33 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:34.377755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bd70acc1-d403-4594-ac4a-9204c5c2bde8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:34.378370 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bd70acc1-d403-4594-ac4a-9204c5c2bde8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:34.384700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd70acc1-d403-4594-ac4a-9204c5c2bde8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:34.385214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd70acc1-d403-4594-ac4a-9204c5c2bde8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:34.385626 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd70acc1-d403-4594-ac4a-9204c5c2bde8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:34.386306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd70acc1-d403-4594-ac4a-9204c5c2bde8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:34.386756 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd70acc1-d403-4594-ac4a-9204c5c2bde8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:34.387170 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bd70acc1-d403-4594-ac4a-9204c5c2bde8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:34.518732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bd70acc1-d403-4594-ac4a-9204c5c2bde8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:34.527045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bd70acc1-d403-4594-ac4a-9204c5c2bde8 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.153275 Aug 28 01:22:34.527610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1187/2595] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:34 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:34.764163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9bf66321-af8b-4e32-b0f7-2549ba2a0296 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:34.764764 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9bf66321-af8b-4e32-b0f7-2549ba2a0296 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:34.771408 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9bf66321-af8b-4e32-b0f7-2549ba2a0296 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:34.771879 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9bf66321-af8b-4e32-b0f7-2549ba2a0296 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:34.772305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9bf66321-af8b-4e32-b0f7-2549ba2a0296 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:34.773005 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9bf66321-af8b-4e32-b0f7-2549ba2a0296 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:34.773460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9bf66321-af8b-4e32-b0f7-2549ba2a0296 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:34.773879 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9bf66321-af8b-4e32-b0f7-2549ba2a0296 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:34.839355 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9bf66321-af8b-4e32-b0f7-2549ba2a0296 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:34.852443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9bf66321-af8b-4e32-b0f7-2549ba2a0296 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.092122 Aug 28 01:22:34.853252 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1409/2596] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:34 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:34.934005 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6481a3f7-b448-47c8-b13d-376addeeecb1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:34.934849 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6481a3f7-b448-47c8-b13d-376addeeecb1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:34.945716 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6481a3f7-b448-47c8-b13d-376addeeecb1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:34.946501 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6481a3f7-b448-47c8-b13d-376addeeecb1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:34.947124 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6481a3f7-b448-47c8-b13d-376addeeecb1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:34.948076 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6481a3f7-b448-47c8-b13d-376addeeecb1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:34.948744 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6481a3f7-b448-47c8-b13d-376addeeecb1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:34.949215 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6481a3f7-b448-47c8-b13d-376addeeecb1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:35.015380 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6481a3f7-b448-47c8-b13d-376addeeecb1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:35.027410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6481a3f7-b448-47c8-b13d-376addeeecb1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.099030 Aug 28 01:22:35.027985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1188/2597] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:34 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:35.043544 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5e494588-4aa7-4b14-8eb8-466916739514 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:35.044317 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5e494588-4aa7-4b14-8eb8-466916739514 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:35.053030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e494588-4aa7-4b14-8eb8-466916739514 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:35.053627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e494588-4aa7-4b14-8eb8-466916739514 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:35.054146 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e494588-4aa7-4b14-8eb8-466916739514 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:35.055118 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e494588-4aa7-4b14-8eb8-466916739514 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:35.055543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e494588-4aa7-4b14-8eb8-466916739514 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:35.056026 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e494588-4aa7-4b14-8eb8-466916739514 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:35.191241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5e494588-4aa7-4b14-8eb8-466916739514 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:35.200753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5e494588-4aa7-4b14-8eb8-466916739514 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:35.202285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5e494588-4aa7-4b14-8eb8-466916739514 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:35.203549 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5e494588-4aa7-4b14-8eb8-466916739514 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.164648 Aug 28 01:22:35.204334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1410/2598] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:35 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:35.548574 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4a2c164b-a406-41f3-8633-633d67282f4f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:35.549282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4a2c164b-a406-41f3-8633-633d67282f4f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:35.556003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a2c164b-a406-41f3-8633-633d67282f4f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:35.556492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a2c164b-a406-41f3-8633-633d67282f4f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:35.556930 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a2c164b-a406-41f3-8633-633d67282f4f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:35.557660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a2c164b-a406-41f3-8633-633d67282f4f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:35.558103 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a2c164b-a406-41f3-8633-633d67282f4f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:35.558532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a2c164b-a406-41f3-8633-633d67282f4f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:35.691973 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4a2c164b-a406-41f3-8633-633d67282f4f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:35.700416 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4a2c164b-a406-41f3-8633-633d67282f4f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.155787 Aug 28 01:22:35.700992 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1189/2599] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:35 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:35.875948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5e9a13ed-5e3f-46d6-85a2-4592488c343b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:35.876653 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5e9a13ed-5e3f-46d6-85a2-4592488c343b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:35.884214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e9a13ed-5e3f-46d6-85a2-4592488c343b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:35.884744 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e9a13ed-5e3f-46d6-85a2-4592488c343b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:35.885265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e9a13ed-5e3f-46d6-85a2-4592488c343b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:35.886005 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e9a13ed-5e3f-46d6-85a2-4592488c343b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:35.886520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e9a13ed-5e3f-46d6-85a2-4592488c343b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:35.887053 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5e9a13ed-5e3f-46d6-85a2-4592488c343b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:35.958038 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5e9a13ed-5e3f-46d6-85a2-4592488c343b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:35.966820 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5e9a13ed-5e3f-46d6-85a2-4592488c343b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.095215 Aug 28 01:22:35.967355 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1411/2600] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:35 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:36.049013 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-644ad6a7-0652-445e-af49-c04c92a3784e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:36.049731 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-644ad6a7-0652-445e-af49-c04c92a3784e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:36.056611 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-644ad6a7-0652-445e-af49-c04c92a3784e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:36.057125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-644ad6a7-0652-445e-af49-c04c92a3784e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:36.057521 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-644ad6a7-0652-445e-af49-c04c92a3784e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:36.058235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-644ad6a7-0652-445e-af49-c04c92a3784e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:36.058680 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-644ad6a7-0652-445e-af49-c04c92a3784e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:36.059136 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-644ad6a7-0652-445e-af49-c04c92a3784e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:36.130773 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-644ad6a7-0652-445e-af49-c04c92a3784e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:36.141902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-644ad6a7-0652-445e-af49-c04c92a3784e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.096685 Aug 28 01:22:36.142398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1190/2601] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:36 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:36.228196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-66ae8b39-93e2-42e8-b3b9-f3f4ab076508 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:36.228798 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-66ae8b39-93e2-42e8-b3b9-f3f4ab076508 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:36.235433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66ae8b39-93e2-42e8-b3b9-f3f4ab076508 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:36.235917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66ae8b39-93e2-42e8-b3b9-f3f4ab076508 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:36.236346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66ae8b39-93e2-42e8-b3b9-f3f4ab076508 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:36.237082 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66ae8b39-93e2-42e8-b3b9-f3f4ab076508 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:36.237528 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66ae8b39-93e2-42e8-b3b9-f3f4ab076508 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:36.237950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66ae8b39-93e2-42e8-b3b9-f3f4ab076508 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:36.376011 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-66ae8b39-93e2-42e8-b3b9-f3f4ab076508 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:36.384931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-66ae8b39-93e2-42e8-b3b9-f3f4ab076508 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:36.386398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-66ae8b39-93e2-42e8-b3b9-f3f4ab076508 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:36.387676 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-66ae8b39-93e2-42e8-b3b9-f3f4ab076508 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.163467 Aug 28 01:22:36.388237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1412/2602] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:36 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:36.723436 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d2c120de-eb9e-4383-a2a8-c21518f4b59b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:36.724175 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d2c120de-eb9e-4383-a2a8-c21518f4b59b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:36.732178 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2c120de-eb9e-4383-a2a8-c21518f4b59b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:36.732803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2c120de-eb9e-4383-a2a8-c21518f4b59b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:36.733368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2c120de-eb9e-4383-a2a8-c21518f4b59b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:36.734291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2c120de-eb9e-4383-a2a8-c21518f4b59b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:36.734845 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2c120de-eb9e-4383-a2a8-c21518f4b59b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:36.735360 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2c120de-eb9e-4383-a2a8-c21518f4b59b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:36.882079 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d2c120de-eb9e-4383-a2a8-c21518f4b59b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:36.892312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d2c120de-eb9e-4383-a2a8-c21518f4b59b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.174116 Aug 28 01:22:36.892871 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1191/2603] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:36 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:36.987107 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-832130d9-6eb1-4c61-bd2c-b2c823a8193f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:36.987703 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-832130d9-6eb1-4c61-bd2c-b2c823a8193f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:36.994578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-832130d9-6eb1-4c61-bd2c-b2c823a8193f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:36.995067 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-832130d9-6eb1-4c61-bd2c-b2c823a8193f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:36.995495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-832130d9-6eb1-4c61-bd2c-b2c823a8193f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:36.996179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-832130d9-6eb1-4c61-bd2c-b2c823a8193f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:36.996631 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-832130d9-6eb1-4c61-bd2c-b2c823a8193f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:36.997085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-832130d9-6eb1-4c61-bd2c-b2c823a8193f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:37.065976 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-832130d9-6eb1-4c61-bd2c-b2c823a8193f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:37.075019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-832130d9-6eb1-4c61-bd2c-b2c823a8193f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.091766 Aug 28 01:22:37.075558 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1413/2604] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:36 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:37.164140 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-49f40e37-65ad-4d16-ab0a-abd4f0feb1f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:37.164712 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-49f40e37-65ad-4d16-ab0a-abd4f0feb1f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:37.171791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49f40e37-65ad-4d16-ab0a-abd4f0feb1f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:37.172244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49f40e37-65ad-4d16-ab0a-abd4f0feb1f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:37.172691 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49f40e37-65ad-4d16-ab0a-abd4f0feb1f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:37.173416 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49f40e37-65ad-4d16-ab0a-abd4f0feb1f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:37.173860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49f40e37-65ad-4d16-ab0a-abd4f0feb1f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:37.174276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49f40e37-65ad-4d16-ab0a-abd4f0feb1f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:37.244716 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-49f40e37-65ad-4d16-ab0a-abd4f0feb1f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:37.254011 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-49f40e37-65ad-4d16-ab0a-abd4f0feb1f3 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.093999 Aug 28 01:22:37.254609 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1192/2605] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:37 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:37.413452 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a0cac792-95fa-4090-9863-df19afb1f8ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:37.414228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a0cac792-95fa-4090-9863-df19afb1f8ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:37.423398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a0cac792-95fa-4090-9863-df19afb1f8ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:37.424053 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a0cac792-95fa-4090-9863-df19afb1f8ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:37.424580 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a0cac792-95fa-4090-9863-df19afb1f8ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:37.425400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a0cac792-95fa-4090-9863-df19afb1f8ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:37.425887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a0cac792-95fa-4090-9863-df19afb1f8ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:37.426469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a0cac792-95fa-4090-9863-df19afb1f8ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:37.564701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a0cac792-95fa-4090-9863-df19afb1f8ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:37.576710 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a0cac792-95fa-4090-9863-df19afb1f8ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:37.578214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a0cac792-95fa-4090-9863-df19afb1f8ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:37.579465 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a0cac792-95fa-4090-9863-df19afb1f8ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.172231 Aug 28 01:22:37.580066 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1414/2606] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:37 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:37.917146 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b5425f66-4db6-435e-bab6-b45e2d76c819 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:37.918088 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b5425f66-4db6-435e-bab6-b45e2d76c819 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:37.924724 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5425f66-4db6-435e-bab6-b45e2d76c819 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:37.925224 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5425f66-4db6-435e-bab6-b45e2d76c819 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:37.925645 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5425f66-4db6-435e-bab6-b45e2d76c819 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:37.926327 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5425f66-4db6-435e-bab6-b45e2d76c819 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:37.926782 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5425f66-4db6-435e-bab6-b45e2d76c819 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:37.927196 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5425f66-4db6-435e-bab6-b45e2d76c819 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:38.067911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b5425f66-4db6-435e-bab6-b45e2d76c819 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:38.080653 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b5425f66-4db6-435e-bab6-b45e2d76c819 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.169151 Aug 28 01:22:38.081694 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1193/2607] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:37 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:38.096255 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9379616d-3e0e-427c-a142-40bf690492b5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:38.096937 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9379616d-3e0e-427c-a142-40bf690492b5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:38.107199 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9379616d-3e0e-427c-a142-40bf690492b5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:38.108054 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9379616d-3e0e-427c-a142-40bf690492b5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:38.108657 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9379616d-3e0e-427c-a142-40bf690492b5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:38.109696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9379616d-3e0e-427c-a142-40bf690492b5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:38.110247 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9379616d-3e0e-427c-a142-40bf690492b5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:38.110773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9379616d-3e0e-427c-a142-40bf690492b5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:38.174293 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9379616d-3e0e-427c-a142-40bf690492b5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:38.182915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9379616d-3e0e-427c-a142-40bf690492b5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.090501 Aug 28 01:22:38.183486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1415/2608] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:38 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:38.278130 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b2831d59-3c60-4d1f-ada3-e926df78f95b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:38.278735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b2831d59-3c60-4d1f-ada3-e926df78f95b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:38.285671 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b2831d59-3c60-4d1f-ada3-e926df78f95b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:38.286156 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b2831d59-3c60-4d1f-ada3-e926df78f95b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:38.286587 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b2831d59-3c60-4d1f-ada3-e926df78f95b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:38.287280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b2831d59-3c60-4d1f-ada3-e926df78f95b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:38.287729 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b2831d59-3c60-4d1f-ada3-e926df78f95b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:38.288146 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b2831d59-3c60-4d1f-ada3-e926df78f95b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:38.290741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:22:38.356096 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b2831d59-3c60-4d1f-ada3-e926df78f95b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:38.365093 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b2831d59-3c60-4d1f-ada3-e926df78f95b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.093118 Aug 28 01:22:38.365701 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1194/2609] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:38 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:38.606935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d40d73c1-0309-49d6-bc66-5765da80acc3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:38.607441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d40d73c1-0309-49d6-bc66-5765da80acc3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:38.613752 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d40d73c1-0309-49d6-bc66-5765da80acc3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:38.614229 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d40d73c1-0309-49d6-bc66-5765da80acc3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:38.614656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d40d73c1-0309-49d6-bc66-5765da80acc3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:38.615331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d40d73c1-0309-49d6-bc66-5765da80acc3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:38.615773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d40d73c1-0309-49d6-bc66-5765da80acc3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:38.616172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d40d73c1-0309-49d6-bc66-5765da80acc3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:38.754321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d40d73c1-0309-49d6-bc66-5765da80acc3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:38.763433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d40d73c1-0309-49d6-bc66-5765da80acc3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:38.764846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d40d73c1-0309-49d6-bc66-5765da80acc3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:38.765996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d40d73c1-0309-49d6-bc66-5765da80acc3 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.165406 Aug 28 01:22:38.766580 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1416/2610] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:38 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:39.106246 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-23c7b324-8d0a-4732-9d49-d16a223e6be0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:39.106854 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-23c7b324-8d0a-4732-9d49-d16a223e6be0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:39.113363 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23c7b324-8d0a-4732-9d49-d16a223e6be0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:39.113825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23c7b324-8d0a-4732-9d49-d16a223e6be0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:39.114257 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23c7b324-8d0a-4732-9d49-d16a223e6be0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:39.114945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23c7b324-8d0a-4732-9d49-d16a223e6be0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:39.115396 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23c7b324-8d0a-4732-9d49-d16a223e6be0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:39.115827 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23c7b324-8d0a-4732-9d49-d16a223e6be0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:39.204015 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-666bf587-0e84-4fb7-a688-edd9b024c0ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:39.204784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-666bf587-0e84-4fb7-a688-edd9b024c0ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:39.211768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-666bf587-0e84-4fb7-a688-edd9b024c0ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:39.212264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-666bf587-0e84-4fb7-a688-edd9b024c0ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:39.212705 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-666bf587-0e84-4fb7-a688-edd9b024c0ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:39.213438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-666bf587-0e84-4fb7-a688-edd9b024c0ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:39.213899 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-666bf587-0e84-4fb7-a688-edd9b024c0ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:39.214316 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-666bf587-0e84-4fb7-a688-edd9b024c0ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:39.251239 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-23c7b324-8d0a-4732-9d49-d16a223e6be0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:39.259755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-23c7b324-8d0a-4732-9d49-d16a223e6be0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.157529 Aug 28 01:22:39.260350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1195/2611] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:39 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:39.282507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-666bf587-0e84-4fb7-a688-edd9b024c0ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:39.290908 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-666bf587-0e84-4fb7-a688-edd9b024c0ee tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.090753 Aug 28 01:22:39.291462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1417/2612] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:39 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:39.386132 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9e10b968-3497-4365-a944-daf99f94f95f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:39.386710 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9e10b968-3497-4365-a944-daf99f94f95f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:39.393803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e10b968-3497-4365-a944-daf99f94f95f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:39.394282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e10b968-3497-4365-a944-daf99f94f95f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:39.394707 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e10b968-3497-4365-a944-daf99f94f95f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:39.395388 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e10b968-3497-4365-a944-daf99f94f95f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:39.395835 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e10b968-3497-4365-a944-daf99f94f95f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:39.396250 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e10b968-3497-4365-a944-daf99f94f95f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:39.476607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9e10b968-3497-4365-a944-daf99f94f95f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:39.490023 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9e10b968-3497-4365-a944-daf99f94f95f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.107503 Aug 28 01:22:39.490902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1196/2613] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:39 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:39.790459 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d7ea2b8d-0355-4136-b015-0e20e3045044 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:39.791030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d7ea2b8d-0355-4136-b015-0e20e3045044 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:39.798040 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7ea2b8d-0355-4136-b015-0e20e3045044 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:39.798520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7ea2b8d-0355-4136-b015-0e20e3045044 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:39.798949 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7ea2b8d-0355-4136-b015-0e20e3045044 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:39.799628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7ea2b8d-0355-4136-b015-0e20e3045044 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:39.800067 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7ea2b8d-0355-4136-b015-0e20e3045044 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:39.800483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7ea2b8d-0355-4136-b015-0e20e3045044 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:39.931538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d7ea2b8d-0355-4136-b015-0e20e3045044 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:39.940268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d7ea2b8d-0355-4136-b015-0e20e3045044 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:39.942113 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d7ea2b8d-0355-4136-b015-0e20e3045044 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:39.943264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d7ea2b8d-0355-4136-b015-0e20e3045044 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.157656 Aug 28 01:22:39.943825 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1418/2614] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:39 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:40.282857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7ab84cd7-da64-451a-9961-0c065e85877e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:40.283461 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7ab84cd7-da64-451a-9961-0c065e85877e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:40.289834 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ab84cd7-da64-451a-9961-0c065e85877e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:40.290315 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ab84cd7-da64-451a-9961-0c065e85877e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:40.290747 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ab84cd7-da64-451a-9961-0c065e85877e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:40.291434 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ab84cd7-da64-451a-9961-0c065e85877e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:40.291878 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ab84cd7-da64-451a-9961-0c065e85877e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:40.292297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ab84cd7-da64-451a-9961-0c065e85877e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:40.312157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3e48fbea-58a0-4ffb-8cb9-86bfeca70df0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:40.312879 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3e48fbea-58a0-4ffb-8cb9-86bfeca70df0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:40.320132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e48fbea-58a0-4ffb-8cb9-86bfeca70df0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:40.320617 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e48fbea-58a0-4ffb-8cb9-86bfeca70df0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:40.321070 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e48fbea-58a0-4ffb-8cb9-86bfeca70df0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:40.321767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e48fbea-58a0-4ffb-8cb9-86bfeca70df0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:40.322214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e48fbea-58a0-4ffb-8cb9-86bfeca70df0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:40.322762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e48fbea-58a0-4ffb-8cb9-86bfeca70df0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:40.388678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3e48fbea-58a0-4ffb-8cb9-86bfeca70df0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:40.396984 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3e48fbea-58a0-4ffb-8cb9-86bfeca70df0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.088955 Aug 28 01:22:40.397801 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1419/2615] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:40 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:40.448147 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7ab84cd7-da64-451a-9961-0c065e85877e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:40.456681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7ab84cd7-da64-451a-9961-0c065e85877e tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.178203 Aug 28 01:22:40.457288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1197/2616] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:40 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:40.516345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4fc43ea6-eb4d-4a3a-a55a-3593955f63aa tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:40.516955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4fc43ea6-eb4d-4a3a-a55a-3593955f63aa tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:40.523383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fc43ea6-eb4d-4a3a-a55a-3593955f63aa tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:40.523865 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fc43ea6-eb4d-4a3a-a55a-3593955f63aa tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:40.524299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fc43ea6-eb4d-4a3a-a55a-3593955f63aa tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:40.525002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fc43ea6-eb4d-4a3a-a55a-3593955f63aa tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:40.525467 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fc43ea6-eb4d-4a3a-a55a-3593955f63aa tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:40.525907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fc43ea6-eb4d-4a3a-a55a-3593955f63aa tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:40.593621 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4fc43ea6-eb4d-4a3a-a55a-3593955f63aa tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:40.602251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4fc43ea6-eb4d-4a3a-a55a-3593955f63aa tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.090770 Aug 28 01:22:40.602823 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1420/2617] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:40 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:40.967384 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-03330cb8-fff8-4c9d-8a8b-f64b06912f38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:40.967994 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-03330cb8-fff8-4c9d-8a8b-f64b06912f38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:40.974410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-03330cb8-fff8-4c9d-8a8b-f64b06912f38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:40.974891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-03330cb8-fff8-4c9d-8a8b-f64b06912f38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:40.975324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-03330cb8-fff8-4c9d-8a8b-f64b06912f38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:40.976005 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-03330cb8-fff8-4c9d-8a8b-f64b06912f38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:40.976453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-03330cb8-fff8-4c9d-8a8b-f64b06912f38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:40.976874 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-03330cb8-fff8-4c9d-8a8b-f64b06912f38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:41.107521 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-03330cb8-fff8-4c9d-8a8b-f64b06912f38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:41.116406 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-03330cb8-fff8-4c9d-8a8b-f64b06912f38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:41.117851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-03330cb8-fff8-4c9d-8a8b-f64b06912f38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:41.119048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-03330cb8-fff8-4c9d-8a8b-f64b06912f38 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.156273 Aug 28 01:22:41.119598 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1198/2618] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:40 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:41.417608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-77e4e2f0-23cd-41ef-91f7-adad8b2b2772 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:41.418248 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-77e4e2f0-23cd-41ef-91f7-adad8b2b2772 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:41.425450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-77e4e2f0-23cd-41ef-91f7-adad8b2b2772 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:41.425938 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-77e4e2f0-23cd-41ef-91f7-adad8b2b2772 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:41.426369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-77e4e2f0-23cd-41ef-91f7-adad8b2b2772 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:41.427075 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-77e4e2f0-23cd-41ef-91f7-adad8b2b2772 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:41.427525 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-77e4e2f0-23cd-41ef-91f7-adad8b2b2772 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:41.427996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-77e4e2f0-23cd-41ef-91f7-adad8b2b2772 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:41.478270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9bf0d6ef-1753-4e81-ab5b-2ab81ca720db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:41.479512 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9bf0d6ef-1753-4e81-ab5b-2ab81ca720db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:41.485688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bf0d6ef-1753-4e81-ab5b-2ab81ca720db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:41.486164 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bf0d6ef-1753-4e81-ab5b-2ab81ca720db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:41.486597 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bf0d6ef-1753-4e81-ab5b-2ab81ca720db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:41.487284 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bf0d6ef-1753-4e81-ab5b-2ab81ca720db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:41.487723 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bf0d6ef-1753-4e81-ab5b-2ab81ca720db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:41.488141 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bf0d6ef-1753-4e81-ab5b-2ab81ca720db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:41.492526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-77e4e2f0-23cd-41ef-91f7-adad8b2b2772 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:41.502253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-77e4e2f0-23cd-41ef-91f7-adad8b2b2772 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.088630 Aug 28 01:22:41.502853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1421/2619] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:41 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:41.623021 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9bf0d6ef-1753-4e81-ab5b-2ab81ca720db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:41.623789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8cb57d54-3f82-449d-88ac-6e8cebdfb4f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:41.624396 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8cb57d54-3f82-449d-88ac-6e8cebdfb4f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:41.630615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8cb57d54-3f82-449d-88ac-6e8cebdfb4f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:41.631095 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8cb57d54-3f82-449d-88ac-6e8cebdfb4f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:41.631354 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9bf0d6ef-1753-4e81-ab5b-2ab81ca720db tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.157306 Aug 28 01:22:41.631500 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8cb57d54-3f82-449d-88ac-6e8cebdfb4f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:41.631930 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1199/2620] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:41 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:41.632185 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8cb57d54-3f82-449d-88ac-6e8cebdfb4f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:41.632600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8cb57d54-3f82-449d-88ac-6e8cebdfb4f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:41.633042 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8cb57d54-3f82-449d-88ac-6e8cebdfb4f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:41.704783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8cb57d54-3f82-449d-88ac-6e8cebdfb4f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:41.713568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8cb57d54-3f82-449d-88ac-6e8cebdfb4f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.093713 Aug 28 01:22:41.714149 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1422/2621] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:41 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:42.143347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c9843edb-b0ca-4465-b645-30eb2f11aed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:42.143944 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c9843edb-b0ca-4465-b645-30eb2f11aed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:42.150374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9843edb-b0ca-4465-b645-30eb2f11aed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:42.150857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9843edb-b0ca-4465-b645-30eb2f11aed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:42.151288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9843edb-b0ca-4465-b645-30eb2f11aed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:42.151952 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9843edb-b0ca-4465-b645-30eb2f11aed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:42.152407 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9843edb-b0ca-4465-b645-30eb2f11aed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:42.152819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9843edb-b0ca-4465-b645-30eb2f11aed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:42.298070 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c9843edb-b0ca-4465-b645-30eb2f11aed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:42.307318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c9843edb-b0ca-4465-b645-30eb2f11aed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:42.308736 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c9843edb-b0ca-4465-b645-30eb2f11aed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:42.309912 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c9843edb-b0ca-4465-b645-30eb2f11aed4 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.170589 Aug 28 01:22:42.310512 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1200/2622] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:42 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:42.522901 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8118c37b-6ae9-4c10-941f-2b4158ec2ea2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:42.523548 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8118c37b-6ae9-4c10-941f-2b4158ec2ea2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:42.530116 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8118c37b-6ae9-4c10-941f-2b4158ec2ea2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:42.530593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8118c37b-6ae9-4c10-941f-2b4158ec2ea2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:42.531044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8118c37b-6ae9-4c10-941f-2b4158ec2ea2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:42.531724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8118c37b-6ae9-4c10-941f-2b4158ec2ea2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:42.532164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8118c37b-6ae9-4c10-941f-2b4158ec2ea2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:42.532573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8118c37b-6ae9-4c10-941f-2b4158ec2ea2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:42.547513 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:22:42.595104 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8118c37b-6ae9-4c10-941f-2b4158ec2ea2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:42.603330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8118c37b-6ae9-4c10-941f-2b4158ec2ea2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.084411 Aug 28 01:22:42.603892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1423/2623] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:42 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:42.654561 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e263bd8a-46e5-4bd9-91db-092f22a072ba tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:42.655141 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e263bd8a-46e5-4bd9-91db-092f22a072ba tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:42.665491 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e263bd8a-46e5-4bd9-91db-092f22a072ba tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:42.666258 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e263bd8a-46e5-4bd9-91db-092f22a072ba tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:42.666913 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e263bd8a-46e5-4bd9-91db-092f22a072ba tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:42.667966 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e263bd8a-46e5-4bd9-91db-092f22a072ba tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:42.668684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e263bd8a-46e5-4bd9-91db-092f22a072ba tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:42.669398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e263bd8a-46e5-4bd9-91db-092f22a072ba tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:42.734886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-dc744af6-2231-46ed-8cb1-257f58d2221b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:42.735523 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-dc744af6-2231-46ed-8cb1-257f58d2221b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:42.741704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc744af6-2231-46ed-8cb1-257f58d2221b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:42.742202 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc744af6-2231-46ed-8cb1-257f58d2221b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:42.742635 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc744af6-2231-46ed-8cb1-257f58d2221b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:42.743336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc744af6-2231-46ed-8cb1-257f58d2221b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:42.743788 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc744af6-2231-46ed-8cb1-257f58d2221b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:42.744208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc744af6-2231-46ed-8cb1-257f58d2221b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:42.819459 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dc744af6-2231-46ed-8cb1-257f58d2221b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:42.824133 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e263bd8a-46e5-4bd9-91db-092f22a072ba tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:42.828144 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-dc744af6-2231-46ed-8cb1-257f58d2221b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.097467 Aug 28 01:22:42.828733 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1424/2624] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:42 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:42.832414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e263bd8a-46e5-4bd9-91db-092f22a072ba tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.183591 Aug 28 01:22:42.832977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1201/2625] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:42 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:43.335954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-81fdac29-6150-4702-a1c5-09845bb814aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:43.336765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-81fdac29-6150-4702-a1c5-09845bb814aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:43.345609 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81fdac29-6150-4702-a1c5-09845bb814aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:43.346215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81fdac29-6150-4702-a1c5-09845bb814aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:43.346799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81fdac29-6150-4702-a1c5-09845bb814aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:43.347694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81fdac29-6150-4702-a1c5-09845bb814aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:43.348244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81fdac29-6150-4702-a1c5-09845bb814aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:43.348781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81fdac29-6150-4702-a1c5-09845bb814aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:43.489291 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-81fdac29-6150-4702-a1c5-09845bb814aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:43.502323 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-81fdac29-6150-4702-a1c5-09845bb814aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:43.504325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-81fdac29-6150-4702-a1c5-09845bb814aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:43.506215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-81fdac29-6150-4702-a1c5-09845bb814aa tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.175691 Aug 28 01:22:43.506788 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1425/2626] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:43 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:43.627246 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ed587ee5-3cfa-4a74-a0ac-d6e657cf891b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:43.627907 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ed587ee5-3cfa-4a74-a0ac-d6e657cf891b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:43.635019 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ed587ee5-3cfa-4a74-a0ac-d6e657cf891b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:43.635488 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ed587ee5-3cfa-4a74-a0ac-d6e657cf891b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:43.635914 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ed587ee5-3cfa-4a74-a0ac-d6e657cf891b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:43.636605 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ed587ee5-3cfa-4a74-a0ac-d6e657cf891b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:43.637089 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ed587ee5-3cfa-4a74-a0ac-d6e657cf891b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:43.637510 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ed587ee5-3cfa-4a74-a0ac-d6e657cf891b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:43.704710 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ed587ee5-3cfa-4a74-a0ac-d6e657cf891b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:43.713244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ed587ee5-3cfa-4a74-a0ac-d6e657cf891b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.090711 Aug 28 01:22:43.713816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1202/2627] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:43 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:43.855755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-71b85503-bebd-4268-983f-87ee5743bae0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:43.856627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-71b85503-bebd-4268-983f-87ee5743bae0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:43.859035 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b9547b7e-d291-4cef-81ff-559e64121f0f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:43.859961 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b9547b7e-d291-4cef-81ff-559e64121f0f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:43.867419 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-71b85503-bebd-4268-983f-87ee5743bae0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:43.868096 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-71b85503-bebd-4268-983f-87ee5743bae0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:43.868657 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-71b85503-bebd-4268-983f-87ee5743bae0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:43.869282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b9547b7e-d291-4cef-81ff-559e64121f0f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:43.869703 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-71b85503-bebd-4268-983f-87ee5743bae0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:43.870045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b9547b7e-d291-4cef-81ff-559e64121f0f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:43.870430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-71b85503-bebd-4268-983f-87ee5743bae0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:43.870763 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b9547b7e-d291-4cef-81ff-559e64121f0f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:43.871131 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-71b85503-bebd-4268-983f-87ee5743bae0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:43.871905 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b9547b7e-d291-4cef-81ff-559e64121f0f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:43.872615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b9547b7e-d291-4cef-81ff-559e64121f0f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:43.873332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b9547b7e-d291-4cef-81ff-559e64121f0f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:43.961808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-71b85503-bebd-4268-983f-87ee5743bae0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:43.970472 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-71b85503-bebd-4268-983f-87ee5743bae0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.120716 Aug 28 01:22:43.971069 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1426/2628] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:43 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:44.030700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b9547b7e-d291-4cef-81ff-559e64121f0f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:44.039449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b9547b7e-d291-4cef-81ff-559e64121f0f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.186667 Aug 28 01:22:44.040048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1203/2629] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:43 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:44.532248 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-267cd21a-19c3-4e22-a500-efaa8657c60e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:44.532909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-267cd21a-19c3-4e22-a500-efaa8657c60e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:44.539323 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-267cd21a-19c3-4e22-a500-efaa8657c60e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:44.539815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-267cd21a-19c3-4e22-a500-efaa8657c60e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:44.540261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-267cd21a-19c3-4e22-a500-efaa8657c60e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:44.541165 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-267cd21a-19c3-4e22-a500-efaa8657c60e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:44.541757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-267cd21a-19c3-4e22-a500-efaa8657c60e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:44.541757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-267cd21a-19c3-4e22-a500-efaa8657c60e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:44.675567 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-267cd21a-19c3-4e22-a500-efaa8657c60e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:44.689197 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-267cd21a-19c3-4e22-a500-efaa8657c60e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:44.691736 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-267cd21a-19c3-4e22-a500-efaa8657c60e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:44.693535 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-267cd21a-19c3-4e22-a500-efaa8657c60e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.165027 Aug 28 01:22:44.694289 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1427/2630] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:44 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:44.735761 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1816b7cb-4934-4c50-9c6a-d6ead4cf0d58 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:44.736410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1816b7cb-4934-4c50-9c6a-d6ead4cf0d58 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:44.746924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1816b7cb-4934-4c50-9c6a-d6ead4cf0d58 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:44.747436 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1816b7cb-4934-4c50-9c6a-d6ead4cf0d58 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:44.747995 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1816b7cb-4934-4c50-9c6a-d6ead4cf0d58 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:44.749015 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1816b7cb-4934-4c50-9c6a-d6ead4cf0d58 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:44.749688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1816b7cb-4934-4c50-9c6a-d6ead4cf0d58 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:44.750218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1816b7cb-4934-4c50-9c6a-d6ead4cf0d58 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:44.819837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1816b7cb-4934-4c50-9c6a-d6ead4cf0d58 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:44.828571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1816b7cb-4934-4c50-9c6a-d6ead4cf0d58 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.098173 Aug 28 01:22:44.829131 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1204/2631] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:44 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:44.994283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e6bb0565-f06e-4e60-928e-019be21e95c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:44.994919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e6bb0565-f06e-4e60-928e-019be21e95c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:45.003018 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6bb0565-f06e-4e60-928e-019be21e95c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:45.003485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6bb0565-f06e-4e60-928e-019be21e95c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:45.003980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6bb0565-f06e-4e60-928e-019be21e95c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:45.004754 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6bb0565-f06e-4e60-928e-019be21e95c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:45.005317 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6bb0565-f06e-4e60-928e-019be21e95c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:45.005795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6bb0565-f06e-4e60-928e-019be21e95c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:45.059813 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4fafd49c-6b50-401d-be99-935ae1b9bce4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:45.060547 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4fafd49c-6b50-401d-be99-935ae1b9bce4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:45.069564 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e6bb0565-f06e-4e60-928e-019be21e95c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:45.070581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4fafd49c-6b50-401d-be99-935ae1b9bce4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:45.071308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4fafd49c-6b50-401d-be99-935ae1b9bce4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:45.071958 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4fafd49c-6b50-401d-be99-935ae1b9bce4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:45.073031 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4fafd49c-6b50-401d-be99-935ae1b9bce4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:45.073726 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4fafd49c-6b50-401d-be99-935ae1b9bce4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:45.074377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4fafd49c-6b50-401d-be99-935ae1b9bce4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:45.077992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e6bb0565-f06e-4e60-928e-019be21e95c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.088131 Aug 28 01:22:45.078577 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1428/2632] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:44 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:45.223252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4fafd49c-6b50-401d-be99-935ae1b9bce4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:45.232137 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4fafd49c-6b50-401d-be99-935ae1b9bce4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.176205 Aug 28 01:22:45.232822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1205/2633] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:45 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:45.719802 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-99161c3c-4048-44dc-ae46-aad9cbbb7eb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:45.720453 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-99161c3c-4048-44dc-ae46-aad9cbbb7eb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:45.727261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-99161c3c-4048-44dc-ae46-aad9cbbb7eb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:45.727774 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-99161c3c-4048-44dc-ae46-aad9cbbb7eb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:45.728228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-99161c3c-4048-44dc-ae46-aad9cbbb7eb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:45.728977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-99161c3c-4048-44dc-ae46-aad9cbbb7eb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:45.729463 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-99161c3c-4048-44dc-ae46-aad9cbbb7eb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:45.729909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-99161c3c-4048-44dc-ae46-aad9cbbb7eb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:45.849265 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c4ac3e3f-03c9-4223-b786-03bb52c90121 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:45.850107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c4ac3e3f-03c9-4223-b786-03bb52c90121 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:45.856515 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c4ac3e3f-03c9-4223-b786-03bb52c90121 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:45.857030 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c4ac3e3f-03c9-4223-b786-03bb52c90121 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:45.857471 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c4ac3e3f-03c9-4223-b786-03bb52c90121 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:45.858171 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c4ac3e3f-03c9-4223-b786-03bb52c90121 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:45.858619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c4ac3e3f-03c9-4223-b786-03bb52c90121 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:45.859034 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c4ac3e3f-03c9-4223-b786-03bb52c90121 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:46.115052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-99161c3c-4048-44dc-ae46-aad9cbbb7eb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:46.125621 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-99161c3c-4048-44dc-ae46-aad9cbbb7eb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:46.127273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-99161c3c-4048-44dc-ae46-aad9cbbb7eb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:46.128587 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-99161c3c-4048-44dc-ae46-aad9cbbb7eb2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.412989 Aug 28 01:22:46.129090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1429/2634] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:45 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:46.131687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c4ac3e3f-03c9-4223-b786-03bb52c90121 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:46.133632 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-47ccaaf8-ab9d-4423-93d8-3ea13a499a9b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:46.134463 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-47ccaaf8-ab9d-4423-93d8-3ea13a499a9b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:46.140372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47ccaaf8-ab9d-4423-93d8-3ea13a499a9b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:46.140775 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c4ac3e3f-03c9-4223-b786-03bb52c90121 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.295539 Aug 28 01:22:46.140918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47ccaaf8-ab9d-4423-93d8-3ea13a499a9b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:46.141587 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47ccaaf8-ab9d-4423-93d8-3ea13a499a9b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:46.141714 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1206/2635] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:45 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:46.142273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47ccaaf8-ab9d-4423-93d8-3ea13a499a9b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:46.142730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47ccaaf8-ab9d-4423-93d8-3ea13a499a9b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:46.143147 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-47ccaaf8-ab9d-4423-93d8-3ea13a499a9b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:46.213722 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-47ccaaf8-ab9d-4423-93d8-3ea13a499a9b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:46.222345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-47ccaaf8-ab9d-4423-93d8-3ea13a499a9b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.092260 Aug 28 01:22:46.222912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1430/2636] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:46 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:46.252821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7cab7426-3d6d-486c-aa44-d2499236fffb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:46.253534 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7cab7426-3d6d-486c-aa44-d2499236fffb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:46.259565 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7cab7426-3d6d-486c-aa44-d2499236fffb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:46.260038 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7cab7426-3d6d-486c-aa44-d2499236fffb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:46.260460 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7cab7426-3d6d-486c-aa44-d2499236fffb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:46.261191 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7cab7426-3d6d-486c-aa44-d2499236fffb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:46.261634 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7cab7426-3d6d-486c-aa44-d2499236fffb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:46.262051 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7cab7426-3d6d-486c-aa44-d2499236fffb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:46.405292 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7cab7426-3d6d-486c-aa44-d2499236fffb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:46.413859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7cab7426-3d6d-486c-aa44-d2499236fffb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.164883 Aug 28 01:22:46.414430 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1207/2637] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:46 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:47.151954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-68f7c47b-f5a1-4ee1-8fab-8c3b8e614b2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:47.152631 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-68f7c47b-f5a1-4ee1-8fab-8c3b8e614b2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:47.160860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-68f7c47b-f5a1-4ee1-8fab-8c3b8e614b2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:47.161422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-68f7c47b-f5a1-4ee1-8fab-8c3b8e614b2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:47.161858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-68f7c47b-f5a1-4ee1-8fab-8c3b8e614b2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:47.162616 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-68f7c47b-f5a1-4ee1-8fab-8c3b8e614b2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:47.163078 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-68f7c47b-f5a1-4ee1-8fab-8c3b8e614b2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:47.163236 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-67b32d66-8b73-4b2e-8033-5b0eb43ec447 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:47.163702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-68f7c47b-f5a1-4ee1-8fab-8c3b8e614b2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:47.163817 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-67b32d66-8b73-4b2e-8033-5b0eb43ec447 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:47.171079 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67b32d66-8b73-4b2e-8033-5b0eb43ec447 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:47.171563 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67b32d66-8b73-4b2e-8033-5b0eb43ec447 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:47.171986 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67b32d66-8b73-4b2e-8033-5b0eb43ec447 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:47.172707 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67b32d66-8b73-4b2e-8033-5b0eb43ec447 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:47.173152 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67b32d66-8b73-4b2e-8033-5b0eb43ec447 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:47.173595 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67b32d66-8b73-4b2e-8033-5b0eb43ec447 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:47.259093 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-67b32d66-8b73-4b2e-8033-5b0eb43ec447 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:47.268679 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-67b32d66-8b73-4b2e-8033-5b0eb43ec447 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.109385 Aug 28 01:22:47.269288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1208/2638] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:47 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:47.273727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6632d1f7-6787-4282-b428-c805ecab8a57 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:47.274338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6632d1f7-6787-4282-b428-c805ecab8a57 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:47.280517 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6632d1f7-6787-4282-b428-c805ecab8a57 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:47.281309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6632d1f7-6787-4282-b428-c805ecab8a57 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:47.281743 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6632d1f7-6787-4282-b428-c805ecab8a57 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:47.282438 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6632d1f7-6787-4282-b428-c805ecab8a57 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:47.282888 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6632d1f7-6787-4282-b428-c805ecab8a57 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:47.283308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6632d1f7-6787-4282-b428-c805ecab8a57 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:47.333995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-68f7c47b-f5a1-4ee1-8fab-8c3b8e614b2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:47.343038 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-68f7c47b-f5a1-4ee1-8fab-8c3b8e614b2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:47.344642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-68f7c47b-f5a1-4ee1-8fab-8c3b8e614b2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:47.345815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-68f7c47b-f5a1-4ee1-8fab-8c3b8e614b2a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.198085 Aug 28 01:22:47.346367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1431/2639] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:47 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:47.349592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6632d1f7-6787-4282-b428-c805ecab8a57 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:47.358162 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6632d1f7-6787-4282-b428-c805ecab8a57 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.087917 Aug 28 01:22:47.358723 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1209/2640] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:47 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:47.437737 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d479be4b-f3fa-4ef4-80b4-16a716590930 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:47.438251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d479be4b-f3fa-4ef4-80b4-16a716590930 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:47.445220 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d479be4b-f3fa-4ef4-80b4-16a716590930 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:47.445700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d479be4b-f3fa-4ef4-80b4-16a716590930 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:47.446129 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d479be4b-f3fa-4ef4-80b4-16a716590930 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:47.446962 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d479be4b-f3fa-4ef4-80b4-16a716590930 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:47.447413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d479be4b-f3fa-4ef4-80b4-16a716590930 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:47.447830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d479be4b-f3fa-4ef4-80b4-16a716590930 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:47.597451 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d479be4b-f3fa-4ef4-80b4-16a716590930 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:47.605909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d479be4b-f3fa-4ef4-80b4-16a716590930 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.173303 Aug 28 01:22:47.606480 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1432/2641] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:47 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:48.290988 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bb55841e-7061-4c64-9039-b6375ef25db8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:48.291695 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bb55841e-7061-4c64-9039-b6375ef25db8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:48.299951 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb55841e-7061-4c64-9039-b6375ef25db8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:48.300519 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb55841e-7061-4c64-9039-b6375ef25db8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:48.301024 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb55841e-7061-4c64-9039-b6375ef25db8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:48.301864 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb55841e-7061-4c64-9039-b6375ef25db8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:48.302390 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb55841e-7061-4c64-9039-b6375ef25db8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:48.302880 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb55841e-7061-4c64-9039-b6375ef25db8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:48.369051 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bb55841e-7061-4c64-9039-b6375ef25db8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:48.370908 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c0bb1127-d6aa-41e4-be5a-7c895ad95699 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:48.371697 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c0bb1127-d6aa-41e4-be5a-7c895ad95699 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:48.371866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:22:48.379056 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0bb1127-d6aa-41e4-be5a-7c895ad95699 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:48.380014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0bb1127-d6aa-41e4-be5a-7c895ad95699 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:48.380412 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bb55841e-7061-4c64-9039-b6375ef25db8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.093713 Aug 28 01:22:48.380569 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0bb1127-d6aa-41e4-be5a-7c895ad95699 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:48.381436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0bb1127-d6aa-41e4-be5a-7c895ad95699 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:48.381828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0bb1127-d6aa-41e4-be5a-7c895ad95699 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:48.382323 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0bb1127-d6aa-41e4-be5a-7c895ad95699 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:48.386815 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1210/2642] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:48 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:48.391883 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-aac9cc32-909e-4d93-ae47-bdc1907416b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:48.392479 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-aac9cc32-909e-4d93-ae47-bdc1907416b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:48.399389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aac9cc32-909e-4d93-ae47-bdc1907416b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:48.399838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aac9cc32-909e-4d93-ae47-bdc1907416b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:48.400532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aac9cc32-909e-4d93-ae47-bdc1907416b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:48.401223 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aac9cc32-909e-4d93-ae47-bdc1907416b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:48.402362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aac9cc32-909e-4d93-ae47-bdc1907416b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:48.402362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aac9cc32-909e-4d93-ae47-bdc1907416b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:48.485586 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-aac9cc32-909e-4d93-ae47-bdc1907416b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:48.494840 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-aac9cc32-909e-4d93-ae47-bdc1907416b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.106985 Aug 28 01:22:48.495410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1211/2643] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:48 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:48.542229 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c0bb1127-d6aa-41e4-be5a-7c895ad95699 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:48.550925 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c0bb1127-d6aa-41e4-be5a-7c895ad95699 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:48.552342 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c0bb1127-d6aa-41e4-be5a-7c895ad95699 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:48.553526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c0bb1127-d6aa-41e4-be5a-7c895ad95699 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.187743 Aug 28 01:22:48.554085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1433/2644] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:48 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:48.626537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-afe7a83d-9c2c-40eb-83ce-3df2c12a4dfd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:48.627146 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-afe7a83d-9c2c-40eb-83ce-3df2c12a4dfd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:48.633636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-afe7a83d-9c2c-40eb-83ce-3df2c12a4dfd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:48.634135 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-afe7a83d-9c2c-40eb-83ce-3df2c12a4dfd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:48.634571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-afe7a83d-9c2c-40eb-83ce-3df2c12a4dfd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:48.635309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-afe7a83d-9c2c-40eb-83ce-3df2c12a4dfd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:48.635830 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-afe7a83d-9c2c-40eb-83ce-3df2c12a4dfd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:48.636262 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-afe7a83d-9c2c-40eb-83ce-3df2c12a4dfd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:48.784935 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-afe7a83d-9c2c-40eb-83ce-3df2c12a4dfd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:48.793604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-afe7a83d-9c2c-40eb-83ce-3df2c12a4dfd tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.170914 Aug 28 01:22:48.794153 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1212/2645] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:48 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:49.400537 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-737bd70d-3146-4ce9-9010-a18cf8213e2a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:49.401167 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-737bd70d-3146-4ce9-9010-a18cf8213e2a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:49.408052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-737bd70d-3146-4ce9-9010-a18cf8213e2a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:49.408527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-737bd70d-3146-4ce9-9010-a18cf8213e2a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:49.408975 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-737bd70d-3146-4ce9-9010-a18cf8213e2a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:49.409664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-737bd70d-3146-4ce9-9010-a18cf8213e2a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:49.410115 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-737bd70d-3146-4ce9-9010-a18cf8213e2a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:49.410527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-737bd70d-3146-4ce9-9010-a18cf8213e2a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:49.478733 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-737bd70d-3146-4ce9-9010-a18cf8213e2a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:49.487553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-737bd70d-3146-4ce9-9010-a18cf8213e2a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.090887 Aug 28 01:22:49.488119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1434/2646] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:49 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:49.515852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f3bcd985-3cb0-4082-ab14-da097cab1f6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:49.516426 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f3bcd985-3cb0-4082-ab14-da097cab1f6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:49.523446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f3bcd985-3cb0-4082-ab14-da097cab1f6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:49.523938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f3bcd985-3cb0-4082-ab14-da097cab1f6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:49.524379 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f3bcd985-3cb0-4082-ab14-da097cab1f6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:49.525034 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f3bcd985-3cb0-4082-ab14-da097cab1f6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:49.525503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f3bcd985-3cb0-4082-ab14-da097cab1f6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:49.525915 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f3bcd985-3cb0-4082-ab14-da097cab1f6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:49.577427 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d43f2104-ac10-4b63-b49a-f18f05ca3282 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:49.578044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d43f2104-ac10-4b63-b49a-f18f05ca3282 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:49.584886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d43f2104-ac10-4b63-b49a-f18f05ca3282 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:49.585556 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d43f2104-ac10-4b63-b49a-f18f05ca3282 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:49.585838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d43f2104-ac10-4b63-b49a-f18f05ca3282 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:49.586541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d43f2104-ac10-4b63-b49a-f18f05ca3282 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:49.586985 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d43f2104-ac10-4b63-b49a-f18f05ca3282 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:49.587408 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d43f2104-ac10-4b63-b49a-f18f05ca3282 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:49.598174 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f3bcd985-3cb0-4082-ab14-da097cab1f6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:49.606756 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f3bcd985-3cb0-4082-ab14-da097cab1f6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.094748 Aug 28 01:22:49.607338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1213/2647] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:49 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:49.716919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d43f2104-ac10-4b63-b49a-f18f05ca3282 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:49.726588 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d43f2104-ac10-4b63-b49a-f18f05ca3282 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:49.728031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d43f2104-ac10-4b63-b49a-f18f05ca3282 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:49.729200 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d43f2104-ac10-4b63-b49a-f18f05ca3282 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.155823 Aug 28 01:22:49.729775 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1435/2648] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:49 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:49.816068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3cea5804-43ba-44a2-ba6e-1e0a83c53b67 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:49.816684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3cea5804-43ba-44a2-ba6e-1e0a83c53b67 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:49.823461 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cea5804-43ba-44a2-ba6e-1e0a83c53b67 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:49.823949 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cea5804-43ba-44a2-ba6e-1e0a83c53b67 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:49.824384 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cea5804-43ba-44a2-ba6e-1e0a83c53b67 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:49.825113 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cea5804-43ba-44a2-ba6e-1e0a83c53b67 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:49.825566 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cea5804-43ba-44a2-ba6e-1e0a83c53b67 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:49.825998 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3cea5804-43ba-44a2-ba6e-1e0a83c53b67 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:49.966107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3cea5804-43ba-44a2-ba6e-1e0a83c53b67 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:49.978239 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3cea5804-43ba-44a2-ba6e-1e0a83c53b67 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.166341 Aug 28 01:22:49.978927 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1214/2649] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:49 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:50.510814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d437ebb4-598b-4f80-a322-7701c2479c0a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:50.511724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d437ebb4-598b-4f80-a322-7701c2479c0a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:50.521515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d437ebb4-598b-4f80-a322-7701c2479c0a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:50.522051 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d437ebb4-598b-4f80-a322-7701c2479c0a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:50.522669 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d437ebb4-598b-4f80-a322-7701c2479c0a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:50.523635 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d437ebb4-598b-4f80-a322-7701c2479c0a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:50.524247 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d437ebb4-598b-4f80-a322-7701c2479c0a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:50.524784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d437ebb4-598b-4f80-a322-7701c2479c0a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:50.589958 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d437ebb4-598b-4f80-a322-7701c2479c0a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:50.598280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d437ebb4-598b-4f80-a322-7701c2479c0a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.093100 Aug 28 01:22:50.598840 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1436/2650] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:50 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:50.630408 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d3b0c11e-3c1a-4d0b-979d-68eaa4a46442 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:50.631402 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d3b0c11e-3c1a-4d0b-979d-68eaa4a46442 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:50.641335 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d3b0c11e-3c1a-4d0b-979d-68eaa4a46442 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:50.641823 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d3b0c11e-3c1a-4d0b-979d-68eaa4a46442 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:50.642286 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d3b0c11e-3c1a-4d0b-979d-68eaa4a46442 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:50.642972 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d3b0c11e-3c1a-4d0b-979d-68eaa4a46442 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:50.643416 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d3b0c11e-3c1a-4d0b-979d-68eaa4a46442 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:50.643864 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d3b0c11e-3c1a-4d0b-979d-68eaa4a46442 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:50.714455 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d3b0c11e-3c1a-4d0b-979d-68eaa4a46442 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:50.722964 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d3b0c11e-3c1a-4d0b-979d-68eaa4a46442 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.098374 Aug 28 01:22:50.723532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1215/2651] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:50 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:50.755278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-31898ac7-af1e-42de-85d5-f0397a1a6aeb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:50.756112 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-31898ac7-af1e-42de-85d5-f0397a1a6aeb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:50.765281 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-31898ac7-af1e-42de-85d5-f0397a1a6aeb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:50.765699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-31898ac7-af1e-42de-85d5-f0397a1a6aeb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:50.766133 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-31898ac7-af1e-42de-85d5-f0397a1a6aeb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:50.766816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-31898ac7-af1e-42de-85d5-f0397a1a6aeb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:50.767258 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-31898ac7-af1e-42de-85d5-f0397a1a6aeb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:50.767668 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-31898ac7-af1e-42de-85d5-f0397a1a6aeb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:50.908172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-31898ac7-af1e-42de-85d5-f0397a1a6aeb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:50.918209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-31898ac7-af1e-42de-85d5-f0397a1a6aeb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:50.919624 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-31898ac7-af1e-42de-85d5-f0397a1a6aeb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:50.920800 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-31898ac7-af1e-42de-85d5-f0397a1a6aeb tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.171958 Aug 28 01:22:50.921396 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1437/2652] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:50 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:51.003232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8caa1f0d-a119-4cd3-ae1f-ee47d3de271c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:51.003834 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8caa1f0d-a119-4cd3-ae1f-ee47d3de271c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:51.010501 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8caa1f0d-a119-4cd3-ae1f-ee47d3de271c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:51.010984 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8caa1f0d-a119-4cd3-ae1f-ee47d3de271c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:51.011416 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8caa1f0d-a119-4cd3-ae1f-ee47d3de271c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:51.012108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8caa1f0d-a119-4cd3-ae1f-ee47d3de271c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:51.012556 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8caa1f0d-a119-4cd3-ae1f-ee47d3de271c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:51.013008 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8caa1f0d-a119-4cd3-ae1f-ee47d3de271c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:51.152757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8caa1f0d-a119-4cd3-ae1f-ee47d3de271c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:51.161359 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8caa1f0d-a119-4cd3-ae1f-ee47d3de271c tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.162621 Aug 28 01:22:51.161928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1216/2653] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:50 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:51.622827 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-42bafaee-6255-4719-88a4-39e59883c4a9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:51.623681 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-42bafaee-6255-4719-88a4-39e59883c4a9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:51.634387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42bafaee-6255-4719-88a4-39e59883c4a9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:51.635116 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42bafaee-6255-4719-88a4-39e59883c4a9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:51.635869 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42bafaee-6255-4719-88a4-39e59883c4a9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:51.637199 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42bafaee-6255-4719-88a4-39e59883c4a9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:51.637863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42bafaee-6255-4719-88a4-39e59883c4a9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:51.638485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42bafaee-6255-4719-88a4-39e59883c4a9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:51.702121 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-42bafaee-6255-4719-88a4-39e59883c4a9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:51.710907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-42bafaee-6255-4719-88a4-39e59883c4a9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.094509 Aug 28 01:22:51.711478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1438/2654] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:51 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:51.745047 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2f34d899-ab75-4871-8354-40dc06dbbc6a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:51.745670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2f34d899-ab75-4871-8354-40dc06dbbc6a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:51.752698 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2f34d899-ab75-4871-8354-40dc06dbbc6a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:51.753202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2f34d899-ab75-4871-8354-40dc06dbbc6a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:51.753634 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2f34d899-ab75-4871-8354-40dc06dbbc6a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:51.754314 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2f34d899-ab75-4871-8354-40dc06dbbc6a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:51.754761 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2f34d899-ab75-4871-8354-40dc06dbbc6a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:51.755175 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2f34d899-ab75-4871-8354-40dc06dbbc6a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:51.826825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2f34d899-ab75-4871-8354-40dc06dbbc6a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:51.835597 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2f34d899-ab75-4871-8354-40dc06dbbc6a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.094561 Aug 28 01:22:51.836245 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1217/2655] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:51 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:22:51.943988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b9b1de9a-e525-4e7d-b652-44e550dfc3da tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:51.944592 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b9b1de9a-e525-4e7d-b652-44e550dfc3da tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:51.953266 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9b1de9a-e525-4e7d-b652-44e550dfc3da tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:51.954040 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9b1de9a-e525-4e7d-b652-44e550dfc3da tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:51.954570 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9b1de9a-e525-4e7d-b652-44e550dfc3da tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:51.955748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9b1de9a-e525-4e7d-b652-44e550dfc3da tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:51.956485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9b1de9a-e525-4e7d-b652-44e550dfc3da tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:51.957146 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9b1de9a-e525-4e7d-b652-44e550dfc3da tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:52.099335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b9b1de9a-e525-4e7d-b652-44e550dfc3da tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:52.108077 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b9b1de9a-e525-4e7d-b652-44e550dfc3da tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:52.109498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b9b1de9a-e525-4e7d-b652-44e550dfc3da tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:52.110668 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b9b1de9a-e525-4e7d-b652-44e550dfc3da tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.171003 Aug 28 01:22:52.111281 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1439/2656] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:51 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:52.182747 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-479870f9-58d4-45b6-a32d-0abdc2359429 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:52.183352 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-479870f9-58d4-45b6-a32d-0abdc2359429 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:52.189830 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-479870f9-58d4-45b6-a32d-0abdc2359429 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:52.190307 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-479870f9-58d4-45b6-a32d-0abdc2359429 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:52.191077 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-479870f9-58d4-45b6-a32d-0abdc2359429 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:52.191751 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-479870f9-58d4-45b6-a32d-0abdc2359429 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:52.192197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-479870f9-58d4-45b6-a32d-0abdc2359429 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:52.192604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-479870f9-58d4-45b6-a32d-0abdc2359429 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:52.327330 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-479870f9-58d4-45b6-a32d-0abdc2359429 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:52.335967 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-479870f9-58d4-45b6-a32d-0abdc2359429 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.157129 Aug 28 01:22:52.336545 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1218/2657] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:52 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:52.732015 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-29b1b3c4-7229-4b5d-8cee-ed687f610ac8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:52.732677 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-29b1b3c4-7229-4b5d-8cee-ed687f610ac8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:52.739695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29b1b3c4-7229-4b5d-8cee-ed687f610ac8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:52.740184 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29b1b3c4-7229-4b5d-8cee-ed687f610ac8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:52.740616 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29b1b3c4-7229-4b5d-8cee-ed687f610ac8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:52.741362 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29b1b3c4-7229-4b5d-8cee-ed687f610ac8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:52.741811 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29b1b3c4-7229-4b5d-8cee-ed687f610ac8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:52.742234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29b1b3c4-7229-4b5d-8cee-ed687f610ac8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:52.744921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:22:52.811414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-29b1b3c4-7229-4b5d-8cee-ed687f610ac8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:52.825043 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-29b1b3c4-7229-4b5d-8cee-ed687f610ac8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.096721 Aug 28 01:22:52.825825 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1440/2658] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:52 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:52.857498 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fda3dc28-ea54-4b34-9ca5-f9e94f2c709c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:52.858132 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fda3dc28-ea54-4b34-9ca5-f9e94f2c709c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:52.865323 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fda3dc28-ea54-4b34-9ca5-f9e94f2c709c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:52.865898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fda3dc28-ea54-4b34-9ca5-f9e94f2c709c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:52.866338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fda3dc28-ea54-4b34-9ca5-f9e94f2c709c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:52.867031 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fda3dc28-ea54-4b34-9ca5-f9e94f2c709c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:52.867712 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fda3dc28-ea54-4b34-9ca5-f9e94f2c709c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:52.868129 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fda3dc28-ea54-4b34-9ca5-f9e94f2c709c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:52.939735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fda3dc28-ea54-4b34-9ca5-f9e94f2c709c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:52.951724 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fda3dc28-ea54-4b34-9ca5-f9e94f2c709c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.097820 Aug 28 01:22:52.952525 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1219/2659] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:52 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:53.133895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d7c2aacb-4bb2-44ec-ba03-fd5a4f28fc33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:53.134454 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d7c2aacb-4bb2-44ec-ba03-fd5a4f28fc33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:53.140629 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c2aacb-4bb2-44ec-ba03-fd5a4f28fc33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:53.141139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c2aacb-4bb2-44ec-ba03-fd5a4f28fc33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:53.141653 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c2aacb-4bb2-44ec-ba03-fd5a4f28fc33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:53.142457 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c2aacb-4bb2-44ec-ba03-fd5a4f28fc33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:53.143028 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c2aacb-4bb2-44ec-ba03-fd5a4f28fc33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:53.143487 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c2aacb-4bb2-44ec-ba03-fd5a4f28fc33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:53.277332 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d7c2aacb-4bb2-44ec-ba03-fd5a4f28fc33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:53.286225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d7c2aacb-4bb2-44ec-ba03-fd5a4f28fc33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:53.287696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d7c2aacb-4bb2-44ec-ba03-fd5a4f28fc33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:53.288829 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d7c2aacb-4bb2-44ec-ba03-fd5a4f28fc33 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.158967 Aug 28 01:22:53.298861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1441/2660] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:53 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:53.357205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ae072072-7b3f-4071-b777-a9b05ff1ebe6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:53.357817 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ae072072-7b3f-4071-b777-a9b05ff1ebe6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:53.364756 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae072072-7b3f-4071-b777-a9b05ff1ebe6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:53.365290 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae072072-7b3f-4071-b777-a9b05ff1ebe6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:53.365719 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae072072-7b3f-4071-b777-a9b05ff1ebe6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:53.366420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae072072-7b3f-4071-b777-a9b05ff1ebe6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:53.366872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae072072-7b3f-4071-b777-a9b05ff1ebe6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:53.367315 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae072072-7b3f-4071-b777-a9b05ff1ebe6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:53.520161 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ae072072-7b3f-4071-b777-a9b05ff1ebe6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:53.528853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ae072072-7b3f-4071-b777-a9b05ff1ebe6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.175577 Aug 28 01:22:53.529461 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1220/2661] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:53 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:53.851727 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c2f882ad-26d5-4962-a13e-d5620c92729a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:53.852618 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c2f882ad-26d5-4962-a13e-d5620c92729a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:53.862671 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2f882ad-26d5-4962-a13e-d5620c92729a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:53.863153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2f882ad-26d5-4962-a13e-d5620c92729a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:53.863580 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2f882ad-26d5-4962-a13e-d5620c92729a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:53.864266 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2f882ad-26d5-4962-a13e-d5620c92729a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:53.864713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2f882ad-26d5-4962-a13e-d5620c92729a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:53.865157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2f882ad-26d5-4962-a13e-d5620c92729a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:53.928930 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c2f882ad-26d5-4962-a13e-d5620c92729a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:53.937404 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c2f882ad-26d5-4962-a13e-d5620c92729a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.092547 Aug 28 01:22:53.937979 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1442/2662] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:53 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:53.974441 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-07494713-80df-4b0a-a117-40db14be3979 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:53.975066 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-07494713-80df-4b0a-a117-40db14be3979 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:53.982046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07494713-80df-4b0a-a117-40db14be3979 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:53.982553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07494713-80df-4b0a-a117-40db14be3979 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:53.982988 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07494713-80df-4b0a-a117-40db14be3979 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:53.983665 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07494713-80df-4b0a-a117-40db14be3979 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:53.984115 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07494713-80df-4b0a-a117-40db14be3979 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:53.984533 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07494713-80df-4b0a-a117-40db14be3979 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:54.057912 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-07494713-80df-4b0a-a117-40db14be3979 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:54.066836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-07494713-80df-4b0a-a117-40db14be3979 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.096363 Aug 28 01:22:54.067403 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1221/2663] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:53 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:54.314658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8435a13e-d5b9-4623-9735-04fab8c99142 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:54.315818 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8435a13e-d5b9-4623-9735-04fab8c99142 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:54.324118 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8435a13e-d5b9-4623-9735-04fab8c99142 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:54.324766 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8435a13e-d5b9-4623-9735-04fab8c99142 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:54.325500 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8435a13e-d5b9-4623-9735-04fab8c99142 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:54.326466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8435a13e-d5b9-4623-9735-04fab8c99142 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:54.327076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8435a13e-d5b9-4623-9735-04fab8c99142 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:54.327658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8435a13e-d5b9-4623-9735-04fab8c99142 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:54.471018 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8435a13e-d5b9-4623-9735-04fab8c99142 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:54.484166 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8435a13e-d5b9-4623-9735-04fab8c99142 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:54.486552 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8435a13e-d5b9-4623-9735-04fab8c99142 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:54.487719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8435a13e-d5b9-4623-9735-04fab8c99142 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.178398 Aug 28 01:22:54.488253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1443/2664] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:54 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:54.550220 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-61b710ae-3f6f-44fb-b83c-4b215a064eb6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:54.550823 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-61b710ae-3f6f-44fb-b83c-4b215a064eb6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:54.557666 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-61b710ae-3f6f-44fb-b83c-4b215a064eb6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:54.558149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-61b710ae-3f6f-44fb-b83c-4b215a064eb6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:54.558582 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-61b710ae-3f6f-44fb-b83c-4b215a064eb6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:54.559270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-61b710ae-3f6f-44fb-b83c-4b215a064eb6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:54.559698 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-61b710ae-3f6f-44fb-b83c-4b215a064eb6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:54.560113 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-61b710ae-3f6f-44fb-b83c-4b215a064eb6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:54.703247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-61b710ae-3f6f-44fb-b83c-4b215a064eb6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:54.711923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-61b710ae-3f6f-44fb-b83c-4b215a064eb6 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.165679 Aug 28 01:22:54.712492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1222/2665] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:54 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:54.966673 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a7d8426b-5999-4e62-b603-8d2992d06800 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:54.967280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a7d8426b-5999-4e62-b603-8d2992d06800 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:54.974329 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7d8426b-5999-4e62-b603-8d2992d06800 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:54.974807 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7d8426b-5999-4e62-b603-8d2992d06800 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:54.975231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7d8426b-5999-4e62-b603-8d2992d06800 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:54.975919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7d8426b-5999-4e62-b603-8d2992d06800 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:54.976360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7d8426b-5999-4e62-b603-8d2992d06800 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:54.976769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a7d8426b-5999-4e62-b603-8d2992d06800 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:55.042903 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a7d8426b-5999-4e62-b603-8d2992d06800 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:55.052494 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a7d8426b-5999-4e62-b603-8d2992d06800 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.090109 Aug 28 01:22:55.053115 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1444/2666] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:54 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:55.088286 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8fa21dc8-7a8a-4960-a108-46dfc25164be tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:55.089157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8fa21dc8-7a8a-4960-a108-46dfc25164be tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:55.097224 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fa21dc8-7a8a-4960-a108-46dfc25164be tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:55.097561 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fa21dc8-7a8a-4960-a108-46dfc25164be tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:55.097990 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fa21dc8-7a8a-4960-a108-46dfc25164be tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:55.098713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fa21dc8-7a8a-4960-a108-46dfc25164be tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:55.099162 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fa21dc8-7a8a-4960-a108-46dfc25164be tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:55.099577 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fa21dc8-7a8a-4960-a108-46dfc25164be tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:55.166798 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8fa21dc8-7a8a-4960-a108-46dfc25164be tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:55.175478 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8fa21dc8-7a8a-4960-a108-46dfc25164be tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.091122 Aug 28 01:22:55.176060 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1223/2667] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:55 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:55.511144 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d2a4a8ea-ff5d-4222-a9ec-c92dbfc34db6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:55.511656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d2a4a8ea-ff5d-4222-a9ec-c92dbfc34db6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:55.518287 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d2a4a8ea-ff5d-4222-a9ec-c92dbfc34db6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:55.518777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d2a4a8ea-ff5d-4222-a9ec-c92dbfc34db6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:55.519220 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d2a4a8ea-ff5d-4222-a9ec-c92dbfc34db6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:55.519973 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d2a4a8ea-ff5d-4222-a9ec-c92dbfc34db6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:55.520446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d2a4a8ea-ff5d-4222-a9ec-c92dbfc34db6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:55.520844 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d2a4a8ea-ff5d-4222-a9ec-c92dbfc34db6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:55.656084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d2a4a8ea-ff5d-4222-a9ec-c92dbfc34db6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:55.665069 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d2a4a8ea-ff5d-4222-a9ec-c92dbfc34db6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:55.666517 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d2a4a8ea-ff5d-4222-a9ec-c92dbfc34db6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:55.667700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d2a4a8ea-ff5d-4222-a9ec-c92dbfc34db6 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.160733 Aug 28 01:22:55.668249 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1445/2668] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:55 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:55.733781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a91b955b-98d2-4f2a-8dba-ebb5f93227b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:55.734394 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a91b955b-98d2-4f2a-8dba-ebb5f93227b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:55.741299 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a91b955b-98d2-4f2a-8dba-ebb5f93227b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:55.741880 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a91b955b-98d2-4f2a-8dba-ebb5f93227b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:55.742501 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a91b955b-98d2-4f2a-8dba-ebb5f93227b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:55.743197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a91b955b-98d2-4f2a-8dba-ebb5f93227b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:55.743649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a91b955b-98d2-4f2a-8dba-ebb5f93227b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:55.744072 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a91b955b-98d2-4f2a-8dba-ebb5f93227b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:55.892575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a91b955b-98d2-4f2a-8dba-ebb5f93227b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:55.901593 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a91b955b-98d2-4f2a-8dba-ebb5f93227b5 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.171873 Aug 28 01:22:55.902385 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1224/2669] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:55 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:56.073775 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-397b431a-a53e-427d-a804-ce5cd3465c93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:56.074401 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-397b431a-a53e-427d-a804-ce5cd3465c93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:56.082389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-397b431a-a53e-427d-a804-ce5cd3465c93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:56.082917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-397b431a-a53e-427d-a804-ce5cd3465c93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:56.083385 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-397b431a-a53e-427d-a804-ce5cd3465c93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:56.084103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-397b431a-a53e-427d-a804-ce5cd3465c93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:56.084590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-397b431a-a53e-427d-a804-ce5cd3465c93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:56.085031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-397b431a-a53e-427d-a804-ce5cd3465c93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:56.156553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-397b431a-a53e-427d-a804-ce5cd3465c93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:56.168049 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-397b431a-a53e-427d-a804-ce5cd3465c93 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.098123 Aug 28 01:22:56.168833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1446/2670] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:56 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:56.213486 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d2253a62-e041-4073-b571-366b1ff9dcf0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:56.214473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d2253a62-e041-4073-b571-366b1ff9dcf0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:56.223751 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2253a62-e041-4073-b571-366b1ff9dcf0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:56.224233 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2253a62-e041-4073-b571-366b1ff9dcf0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:56.224660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2253a62-e041-4073-b571-366b1ff9dcf0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:56.225372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2253a62-e041-4073-b571-366b1ff9dcf0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:56.225812 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2253a62-e041-4073-b571-366b1ff9dcf0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:56.226234 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d2253a62-e041-4073-b571-366b1ff9dcf0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:56.302509 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d2253a62-e041-4073-b571-366b1ff9dcf0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:56.315240 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d2253a62-e041-4073-b571-366b1ff9dcf0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.107757 Aug 28 01:22:56.316376 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1225/2671] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:56 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:56.691778 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ce94dcbe-7adb-48b1-be85-0d2a40d96878 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:56.692478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ce94dcbe-7adb-48b1-be85-0d2a40d96878 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:56.699132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce94dcbe-7adb-48b1-be85-0d2a40d96878 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:56.699613 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce94dcbe-7adb-48b1-be85-0d2a40d96878 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:56.700042 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce94dcbe-7adb-48b1-be85-0d2a40d96878 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:56.700730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce94dcbe-7adb-48b1-be85-0d2a40d96878 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:56.701216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce94dcbe-7adb-48b1-be85-0d2a40d96878 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:56.701631 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce94dcbe-7adb-48b1-be85-0d2a40d96878 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:56.834586 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ce94dcbe-7adb-48b1-be85-0d2a40d96878 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:56.843614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ce94dcbe-7adb-48b1-be85-0d2a40d96878 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:56.845060 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ce94dcbe-7adb-48b1-be85-0d2a40d96878 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:56.846244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ce94dcbe-7adb-48b1-be85-0d2a40d96878 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.158805 Aug 28 01:22:56.846800 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1447/2672] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:56 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:56.926028 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-802d2372-92a6-4e67-8d6d-c36b64f13576 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:56.926666 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-802d2372-92a6-4e67-8d6d-c36b64f13576 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:56.933111 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-802d2372-92a6-4e67-8d6d-c36b64f13576 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:56.933588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-802d2372-92a6-4e67-8d6d-c36b64f13576 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:56.934021 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-802d2372-92a6-4e67-8d6d-c36b64f13576 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:56.935006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-802d2372-92a6-4e67-8d6d-c36b64f13576 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:56.935648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-802d2372-92a6-4e67-8d6d-c36b64f13576 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:56.936181 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-802d2372-92a6-4e67-8d6d-c36b64f13576 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:57.078432 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-802d2372-92a6-4e67-8d6d-c36b64f13576 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:57.093341 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-802d2372-92a6-4e67-8d6d-c36b64f13576 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.172138 Aug 28 01:22:57.093940 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1226/2673] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:56 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:57.191973 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-713acb91-0f23-4791-b412-ef42f7612997 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:57.192626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-713acb91-0f23-4791-b412-ef42f7612997 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:57.200301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-713acb91-0f23-4791-b412-ef42f7612997 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:57.200787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-713acb91-0f23-4791-b412-ef42f7612997 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:57.201257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-713acb91-0f23-4791-b412-ef42f7612997 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:57.202083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-713acb91-0f23-4791-b412-ef42f7612997 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:57.202530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-713acb91-0f23-4791-b412-ef42f7612997 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:57.202948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-713acb91-0f23-4791-b412-ef42f7612997 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:57.269481 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-713acb91-0f23-4791-b412-ef42f7612997 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:57.278380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-713acb91-0f23-4791-b412-ef42f7612997 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.090836 Aug 28 01:22:57.278992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1448/2674] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:57 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:57.340074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0e76a1dc-015a-4510-b100-1c0110f681dd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:57.340680 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0e76a1dc-015a-4510-b100-1c0110f681dd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:57.347866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e76a1dc-015a-4510-b100-1c0110f681dd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:57.348349 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e76a1dc-015a-4510-b100-1c0110f681dd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:57.348774 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e76a1dc-015a-4510-b100-1c0110f681dd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:57.349482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e76a1dc-015a-4510-b100-1c0110f681dd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:57.349927 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e76a1dc-015a-4510-b100-1c0110f681dd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:57.350347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0e76a1dc-015a-4510-b100-1c0110f681dd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:57.366415 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:22:57.421080 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0e76a1dc-015a-4510-b100-1c0110f681dd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:57.429824 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0e76a1dc-015a-4510-b100-1c0110f681dd tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.093577 Aug 28 01:22:57.430452 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1227/2675] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:57 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:57.873572 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f98f88e1-dd39-4c37-9112-e62837bbe38e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:57.874527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f98f88e1-dd39-4c37-9112-e62837bbe38e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:57.882398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f98f88e1-dd39-4c37-9112-e62837bbe38e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:57.882874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f98f88e1-dd39-4c37-9112-e62837bbe38e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:57.883298 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f98f88e1-dd39-4c37-9112-e62837bbe38e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:57.883979 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f98f88e1-dd39-4c37-9112-e62837bbe38e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:57.884471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f98f88e1-dd39-4c37-9112-e62837bbe38e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:57.884947 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f98f88e1-dd39-4c37-9112-e62837bbe38e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:58.030721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f98f88e1-dd39-4c37-9112-e62837bbe38e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:58.044514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f98f88e1-dd39-4c37-9112-e62837bbe38e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:58.046793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f98f88e1-dd39-4c37-9112-e62837bbe38e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:58.048312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f98f88e1-dd39-4c37-9112-e62837bbe38e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.180885 Aug 28 01:22:58.049106 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1449/2676] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:57 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:58.116365 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2b113d6f-1e0a-48b2-8bed-08ac7073e8e0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:58.116998 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2b113d6f-1e0a-48b2-8bed-08ac7073e8e0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:58.123329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b113d6f-1e0a-48b2-8bed-08ac7073e8e0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:58.123904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b113d6f-1e0a-48b2-8bed-08ac7073e8e0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:58.124270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b113d6f-1e0a-48b2-8bed-08ac7073e8e0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:58.124989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b113d6f-1e0a-48b2-8bed-08ac7073e8e0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:58.125450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b113d6f-1e0a-48b2-8bed-08ac7073e8e0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:58.125879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b113d6f-1e0a-48b2-8bed-08ac7073e8e0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:58.278294 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2b113d6f-1e0a-48b2-8bed-08ac7073e8e0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:58.287148 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2b113d6f-1e0a-48b2-8bed-08ac7073e8e0 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.174875 Aug 28 01:22:58.287729 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1228/2677] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:58 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:58.302031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d8a7175b-d9c8-41d8-be98-82b3c9cbea36 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:58.302813 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d8a7175b-d9c8-41d8-be98-82b3c9cbea36 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:58.309484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d8a7175b-d9c8-41d8-be98-82b3c9cbea36 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:58.309954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d8a7175b-d9c8-41d8-be98-82b3c9cbea36 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:58.310377 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d8a7175b-d9c8-41d8-be98-82b3c9cbea36 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:58.311062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d8a7175b-d9c8-41d8-be98-82b3c9cbea36 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:58.311487 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d8a7175b-d9c8-41d8-be98-82b3c9cbea36 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:58.311900 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d8a7175b-d9c8-41d8-be98-82b3c9cbea36 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:58.379286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d8a7175b-d9c8-41d8-be98-82b3c9cbea36 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:58.390559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d8a7175b-d9c8-41d8-be98-82b3c9cbea36 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.093462 Aug 28 01:22:58.391219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1450/2678] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:58 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:58.451157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-400216d1-7ffb-4f9a-baad-06ae00c02ae6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:58.451773 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-400216d1-7ffb-4f9a-baad-06ae00c02ae6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:58.454164 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:22:58.460355 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-400216d1-7ffb-4f9a-baad-06ae00c02ae6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:58.460833 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-400216d1-7ffb-4f9a-baad-06ae00c02ae6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:58.461297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-400216d1-7ffb-4f9a-baad-06ae00c02ae6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:58.461980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-400216d1-7ffb-4f9a-baad-06ae00c02ae6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:58.462437 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-400216d1-7ffb-4f9a-baad-06ae00c02ae6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:58.462858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-400216d1-7ffb-4f9a-baad-06ae00c02ae6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:58.528557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-400216d1-7ffb-4f9a-baad-06ae00c02ae6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:58.537393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-400216d1-7ffb-4f9a-baad-06ae00c02ae6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.090111 Aug 28 01:22:58.537960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1229/2679] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:58 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:59.072551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c2f4f328-22ae-4d0f-9737-04038e5dbc8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:59.073094 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c2f4f328-22ae-4d0f-9737-04038e5dbc8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:59.079444 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2f4f328-22ae-4d0f-9737-04038e5dbc8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:59.079955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2f4f328-22ae-4d0f-9737-04038e5dbc8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:59.080374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2f4f328-22ae-4d0f-9737-04038e5dbc8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:59.081079 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2f4f328-22ae-4d0f-9737-04038e5dbc8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:59.081529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2f4f328-22ae-4d0f-9737-04038e5dbc8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:59.081980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2f4f328-22ae-4d0f-9737-04038e5dbc8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:59.219053 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c2f4f328-22ae-4d0f-9737-04038e5dbc8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:59.228926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c2f4f328-22ae-4d0f-9737-04038e5dbc8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:59.230395 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c2f4f328-22ae-4d0f-9737-04038e5dbc8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:59.231549 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c2f4f328-22ae-4d0f-9737-04038e5dbc8a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.163053 Aug 28 01:22:59.232093 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1451/2680] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:22:59 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:22:59.315060 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c680fe6c-e78a-48a4-a6c1-a0abcf5e6374 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:59.316309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c680fe6c-e78a-48a4-a6c1-a0abcf5e6374 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:59.323173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c680fe6c-e78a-48a4-a6c1-a0abcf5e6374 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:59.323590 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c680fe6c-e78a-48a4-a6c1-a0abcf5e6374 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:59.324019 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c680fe6c-e78a-48a4-a6c1-a0abcf5e6374 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:59.324709 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c680fe6c-e78a-48a4-a6c1-a0abcf5e6374 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:59.325193 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c680fe6c-e78a-48a4-a6c1-a0abcf5e6374 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:59.325610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c680fe6c-e78a-48a4-a6c1-a0abcf5e6374 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:59.411807 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c2c37621-1af6-486a-8ca1-75f060ed3ba2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:59.412416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c2c37621-1af6-486a-8ca1-75f060ed3ba2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:59.419087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2c37621-1af6-486a-8ca1-75f060ed3ba2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:59.419651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2c37621-1af6-486a-8ca1-75f060ed3ba2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:59.420005 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2c37621-1af6-486a-8ca1-75f060ed3ba2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:59.420675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2c37621-1af6-486a-8ca1-75f060ed3ba2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:59.421236 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2c37621-1af6-486a-8ca1-75f060ed3ba2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:59.421983 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c2c37621-1af6-486a-8ca1-75f060ed3ba2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:59.462169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c680fe6c-e78a-48a4-a6c1-a0abcf5e6374 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:59.470752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c680fe6c-e78a-48a4-a6c1-a0abcf5e6374 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.160790 Aug 28 01:22:59.471344 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1230/2681] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:59 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:22:59.489808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c2c37621-1af6-486a-8ca1-75f060ed3ba2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:59.498254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c2c37621-1af6-486a-8ca1-75f060ed3ba2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.090342 Aug 28 01:22:59.498809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1452/2682] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:59 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:22:59.560613 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2d88a9b0-3877-46f3-b4ba-127b469d821c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:22:59.561460 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2d88a9b0-3877-46f3-b4ba-127b469d821c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:22:59.570122 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d88a9b0-3877-46f3-b4ba-127b469d821c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:59.570809 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d88a9b0-3877-46f3-b4ba-127b469d821c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:59.571385 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d88a9b0-3877-46f3-b4ba-127b469d821c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:59.572306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d88a9b0-3877-46f3-b4ba-127b469d821c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:22:59.572918 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d88a9b0-3877-46f3-b4ba-127b469d821c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:22:59.573596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d88a9b0-3877-46f3-b4ba-127b469d821c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:22:59.640898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2d88a9b0-3877-46f3-b4ba-127b469d821c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:22:59.649494 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2d88a9b0-3877-46f3-b4ba-127b469d821c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.093991 Aug 28 01:22:59.650049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1231/2683] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:22:59 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:00.255113 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-07a48893-75b5-41a1-ab0c-709309d28e71 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:00.255742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-07a48893-75b5-41a1-ab0c-709309d28e71 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:00.262165 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07a48893-75b5-41a1-ab0c-709309d28e71 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:00.262643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07a48893-75b5-41a1-ab0c-709309d28e71 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:00.263070 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07a48893-75b5-41a1-ab0c-709309d28e71 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:00.263749 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07a48893-75b5-41a1-ab0c-709309d28e71 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:00.264196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07a48893-75b5-41a1-ab0c-709309d28e71 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:00.264612 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07a48893-75b5-41a1-ab0c-709309d28e71 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:00.397642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-07a48893-75b5-41a1-ab0c-709309d28e71 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:00.406368 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-07a48893-75b5-41a1-ab0c-709309d28e71 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:00.407805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-07a48893-75b5-41a1-ab0c-709309d28e71 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:00.408989 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-07a48893-75b5-41a1-ab0c-709309d28e71 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.157830 Aug 28 01:23:00.409604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1453/2684] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:00 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:00.492542 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-06805ad6-b8a2-4cfb-9c8c-cbf54a2dd3e7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:00.492952 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-06805ad6-b8a2-4cfb-9c8c-cbf54a2dd3e7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:00.499663 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-06805ad6-b8a2-4cfb-9c8c-cbf54a2dd3e7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:00.500148 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-06805ad6-b8a2-4cfb-9c8c-cbf54a2dd3e7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:00.500582 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-06805ad6-b8a2-4cfb-9c8c-cbf54a2dd3e7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:00.501318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-06805ad6-b8a2-4cfb-9c8c-cbf54a2dd3e7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:00.501775 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-06805ad6-b8a2-4cfb-9c8c-cbf54a2dd3e7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:00.502194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-06805ad6-b8a2-4cfb-9c8c-cbf54a2dd3e7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:00.519241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5bd6fb7e-3a9c-4af7-91f7-f0debf3c305a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:00.520409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5bd6fb7e-3a9c-4af7-91f7-f0debf3c305a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:00.527345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5bd6fb7e-3a9c-4af7-91f7-f0debf3c305a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:00.527809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5bd6fb7e-3a9c-4af7-91f7-f0debf3c305a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:00.528264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5bd6fb7e-3a9c-4af7-91f7-f0debf3c305a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:00.528980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5bd6fb7e-3a9c-4af7-91f7-f0debf3c305a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:00.529428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5bd6fb7e-3a9c-4af7-91f7-f0debf3c305a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:00.529864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5bd6fb7e-3a9c-4af7-91f7-f0debf3c305a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:00.601982 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5bd6fb7e-3a9c-4af7-91f7-f0debf3c305a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:00.610342 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5bd6fb7e-3a9c-4af7-91f7-f0debf3c305a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.094937 Aug 28 01:23:00.610891 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1454/2685] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:00 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:00.660350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-06805ad6-b8a2-4cfb-9c8c-cbf54a2dd3e7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:00.668918 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-06805ad6-b8a2-4cfb-9c8c-cbf54a2dd3e7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.180782 Aug 28 01:23:00.669493 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1232/2686] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:00 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:00.670597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5f737382-69d4-4ec1-aa53-8d9c7431b8c0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:00.671256 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5f737382-69d4-4ec1-aa53-8d9c7431b8c0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:00.673128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:23:00.678889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f737382-69d4-4ec1-aa53-8d9c7431b8c0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:00.679377 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f737382-69d4-4ec1-aa53-8d9c7431b8c0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:00.679815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f737382-69d4-4ec1-aa53-8d9c7431b8c0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:00.680505 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f737382-69d4-4ec1-aa53-8d9c7431b8c0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:00.680983 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f737382-69d4-4ec1-aa53-8d9c7431b8c0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:00.681400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f737382-69d4-4ec1-aa53-8d9c7431b8c0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:00.743914 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5f737382-69d4-4ec1-aa53-8d9c7431b8c0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:00.752695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5f737382-69d4-4ec1-aa53-8d9c7431b8c0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.085978 Aug 28 01:23:00.753286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1455/2687] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:00 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:01.434666 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b149f5c4-4295-4db2-bb4d-3429c43fd1ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:01.435272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b149f5c4-4295-4db2-bb4d-3429c43fd1ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:01.442389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b149f5c4-4295-4db2-bb4d-3429c43fd1ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:01.442876 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b149f5c4-4295-4db2-bb4d-3429c43fd1ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:01.443376 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b149f5c4-4295-4db2-bb4d-3429c43fd1ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:01.444088 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b149f5c4-4295-4db2-bb4d-3429c43fd1ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:01.444556 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b149f5c4-4295-4db2-bb4d-3429c43fd1ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:01.445034 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b149f5c4-4295-4db2-bb4d-3429c43fd1ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:01.586895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b149f5c4-4295-4db2-bb4d-3429c43fd1ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:01.599831 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b149f5c4-4295-4db2-bb4d-3429c43fd1ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:01.601289 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b149f5c4-4295-4db2-bb4d-3429c43fd1ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:01.602436 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b149f5c4-4295-4db2-bb4d-3429c43fd1ff tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.173164 Aug 28 01:23:01.602994 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1233/2688] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:01 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:01.631220 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8740bea5-139c-4e3b-9cc7-51c4e5a7e9d8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:01.631828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8740bea5-139c-4e3b-9cc7-51c4e5a7e9d8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:01.640757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8740bea5-139c-4e3b-9cc7-51c4e5a7e9d8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:01.641388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8740bea5-139c-4e3b-9cc7-51c4e5a7e9d8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:01.641973 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8740bea5-139c-4e3b-9cc7-51c4e5a7e9d8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:01.643019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8740bea5-139c-4e3b-9cc7-51c4e5a7e9d8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:01.643685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8740bea5-139c-4e3b-9cc7-51c4e5a7e9d8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:01.644264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8740bea5-139c-4e3b-9cc7-51c4e5a7e9d8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:01.690257 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-beaa0fc5-a207-47e1-b579-604602eb9e06 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:01.691129 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-beaa0fc5-a207-47e1-b579-604602eb9e06 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:01.697756 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-beaa0fc5-a207-47e1-b579-604602eb9e06 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:01.698232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-beaa0fc5-a207-47e1-b579-604602eb9e06 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:01.698665 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-beaa0fc5-a207-47e1-b579-604602eb9e06 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:01.699371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-beaa0fc5-a207-47e1-b579-604602eb9e06 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:01.699825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-beaa0fc5-a207-47e1-b579-604602eb9e06 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:01.700247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-beaa0fc5-a207-47e1-b579-604602eb9e06 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:01.718131 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8740bea5-139c-4e3b-9cc7-51c4e5a7e9d8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:01.723764 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:23:01.728585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8740bea5-139c-4e3b-9cc7-51c4e5a7e9d8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.101162 Aug 28 01:23:01.729206 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1456/2689] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:01 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:01.774338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-85b411d2-b858-466c-ad3c-23acbe04c823 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:01.774941 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-85b411d2-b858-466c-ad3c-23acbe04c823 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:01.781681 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85b411d2-b858-466c-ad3c-23acbe04c823 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:01.782165 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85b411d2-b858-466c-ad3c-23acbe04c823 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:01.782598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85b411d2-b858-466c-ad3c-23acbe04c823 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:01.783288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85b411d2-b858-466c-ad3c-23acbe04c823 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:01.783738 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85b411d2-b858-466c-ad3c-23acbe04c823 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:01.784160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85b411d2-b858-466c-ad3c-23acbe04c823 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:01.850623 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-85b411d2-b858-466c-ad3c-23acbe04c823 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:01.859013 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-85b411d2-b858-466c-ad3c-23acbe04c823 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.088889 Aug 28 01:23:01.859595 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1457/2690] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:01 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:01.872576 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-beaa0fc5-a207-47e1-b579-604602eb9e06 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:01.884930 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-beaa0fc5-a207-47e1-b579-604602eb9e06 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.198323 Aug 28 01:23:01.886017 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1234/2691] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:01 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:02.633602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-30a6496d-5709-45fc-b90d-9fd41d18e24f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:02.634426 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-30a6496d-5709-45fc-b90d-9fd41d18e24f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:02.641977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-30a6496d-5709-45fc-b90d-9fd41d18e24f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:02.642547 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-30a6496d-5709-45fc-b90d-9fd41d18e24f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:02.643055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-30a6496d-5709-45fc-b90d-9fd41d18e24f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:02.643826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-30a6496d-5709-45fc-b90d-9fd41d18e24f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:02.644308 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-30a6496d-5709-45fc-b90d-9fd41d18e24f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:02.644797 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-30a6496d-5709-45fc-b90d-9fd41d18e24f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:02.751801 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-081a8d88-3243-4b2a-81e5-39d3eb689da2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:02.752331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-081a8d88-3243-4b2a-81e5-39d3eb689da2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:02.759224 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-081a8d88-3243-4b2a-81e5-39d3eb689da2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:02.759710 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-081a8d88-3243-4b2a-81e5-39d3eb689da2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:02.760143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-081a8d88-3243-4b2a-81e5-39d3eb689da2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:02.760837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-081a8d88-3243-4b2a-81e5-39d3eb689da2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:02.761297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-081a8d88-3243-4b2a-81e5-39d3eb689da2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:02.761821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-081a8d88-3243-4b2a-81e5-39d3eb689da2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:02.799881 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-30a6496d-5709-45fc-b90d-9fd41d18e24f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:02.809441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-30a6496d-5709-45fc-b90d-9fd41d18e24f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:02.810942 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-30a6496d-5709-45fc-b90d-9fd41d18e24f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:02.812140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-30a6496d-5709-45fc-b90d-9fd41d18e24f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.183067 Aug 28 01:23:02.812679 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1458/2692] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:02 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:02.826666 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-081a8d88-3243-4b2a-81e5-39d3eb689da2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:02.835554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-081a8d88-3243-4b2a-81e5-39d3eb689da2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.087571 Aug 28 01:23:02.836396 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1235/2693] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:02 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:02.880232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cbce7ae1-6d98-4a1d-b6a5-8993dc274e3f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:02.880889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cbce7ae1-6d98-4a1d-b6a5-8993dc274e3f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:02.887278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbce7ae1-6d98-4a1d-b6a5-8993dc274e3f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:02.887747 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbce7ae1-6d98-4a1d-b6a5-8993dc274e3f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:02.888179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbce7ae1-6d98-4a1d-b6a5-8993dc274e3f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:02.888859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbce7ae1-6d98-4a1d-b6a5-8993dc274e3f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:02.889347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbce7ae1-6d98-4a1d-b6a5-8993dc274e3f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:02.889787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbce7ae1-6d98-4a1d-b6a5-8993dc274e3f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:02.906747 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d8c3214d-84d6-4cfa-bef0-1a2db1ad004b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:02.907356 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d8c3214d-84d6-4cfa-bef0-1a2db1ad004b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:02.914020 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d8c3214d-84d6-4cfa-bef0-1a2db1ad004b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:02.914604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d8c3214d-84d6-4cfa-bef0-1a2db1ad004b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:02.914917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d8c3214d-84d6-4cfa-bef0-1a2db1ad004b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:02.915752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d8c3214d-84d6-4cfa-bef0-1a2db1ad004b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:02.916198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d8c3214d-84d6-4cfa-bef0-1a2db1ad004b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:02.916795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d8c3214d-84d6-4cfa-bef0-1a2db1ad004b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:02.956272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-cbce7ae1-6d98-4a1d-b6a5-8993dc274e3f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:02.964824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cbce7ae1-6d98-4a1d-b6a5-8993dc274e3f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.088476 Aug 28 01:23:02.965434 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1459/2694] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:02 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:03.053292 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d8c3214d-84d6-4cfa-bef0-1a2db1ad004b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:03.066317 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d8c3214d-84d6-4cfa-bef0-1a2db1ad004b tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.162773 Aug 28 01:23:03.067218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1236/2695] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:02 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:03.840362 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fc65fdef-117f-47b7-80e4-35860bf8cf25 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:03.841469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fc65fdef-117f-47b7-80e4-35860bf8cf25 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:03.851863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc65fdef-117f-47b7-80e4-35860bf8cf25 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:03.852344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc65fdef-117f-47b7-80e4-35860bf8cf25 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:03.852773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc65fdef-117f-47b7-80e4-35860bf8cf25 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:03.853487 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc65fdef-117f-47b7-80e4-35860bf8cf25 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:03.853935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc65fdef-117f-47b7-80e4-35860bf8cf25 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:03.854352 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc65fdef-117f-47b7-80e4-35860bf8cf25 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:03.863774 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-dea32a72-5021-4749-b50b-930eeaa2e769 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:03.864403 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-dea32a72-5021-4749-b50b-930eeaa2e769 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:03.871165 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dea32a72-5021-4749-b50b-930eeaa2e769 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:03.871654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dea32a72-5021-4749-b50b-930eeaa2e769 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:03.872078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dea32a72-5021-4749-b50b-930eeaa2e769 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:03.872764 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dea32a72-5021-4749-b50b-930eeaa2e769 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:03.873240 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dea32a72-5021-4749-b50b-930eeaa2e769 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:03.873660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dea32a72-5021-4749-b50b-930eeaa2e769 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:03.954248 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-dea32a72-5021-4749-b50b-930eeaa2e769 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:03.963297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-dea32a72-5021-4749-b50b-930eeaa2e769 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.103372 Aug 28 01:23:03.963886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1237/2696] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:03 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:03.986028 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-70c1a12e-6783-4dde-9926-204a1645da91 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:03.986926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-70c1a12e-6783-4dde-9926-204a1645da91 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:03.993293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70c1a12e-6783-4dde-9926-204a1645da91 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:03.993879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70c1a12e-6783-4dde-9926-204a1645da91 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:03.994320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70c1a12e-6783-4dde-9926-204a1645da91 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:03.995019 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70c1a12e-6783-4dde-9926-204a1645da91 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:03.995467 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70c1a12e-6783-4dde-9926-204a1645da91 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:03.995887 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70c1a12e-6783-4dde-9926-204a1645da91 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:04.013325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fc65fdef-117f-47b7-80e4-35860bf8cf25 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:04.022200 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fc65fdef-117f-47b7-80e4-35860bf8cf25 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:04.023638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fc65fdef-117f-47b7-80e4-35860bf8cf25 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:04.024805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fc65fdef-117f-47b7-80e4-35860bf8cf25 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.191512 Aug 28 01:23:04.025512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1460/2697] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:03 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:04.059415 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-70c1a12e-6783-4dde-9926-204a1645da91 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:04.069703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-70c1a12e-6783-4dde-9926-204a1645da91 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.087569 Aug 28 01:23:04.070273 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1238/2698] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:03 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:04.087931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fca6c692-c87d-472c-86dd-f8995335563a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:04.088534 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fca6c692-c87d-472c-86dd-f8995335563a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:04.094963 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fca6c692-c87d-472c-86dd-f8995335563a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:04.095377 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fca6c692-c87d-472c-86dd-f8995335563a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:04.095799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fca6c692-c87d-472c-86dd-f8995335563a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:04.096478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fca6c692-c87d-472c-86dd-f8995335563a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:04.096921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fca6c692-c87d-472c-86dd-f8995335563a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:04.097390 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fca6c692-c87d-472c-86dd-f8995335563a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:04.245546 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fca6c692-c87d-472c-86dd-f8995335563a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:04.258858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fca6c692-c87d-472c-86dd-f8995335563a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.174671 Aug 28 01:23:04.259638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1461/2699] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:04 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:04.985076 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-882e3530-db25-48af-9355-d2d5a95d1146 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:04.985719 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-882e3530-db25-48af-9355-d2d5a95d1146 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:04.993001 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-882e3530-db25-48af-9355-d2d5a95d1146 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:04.993500 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-882e3530-db25-48af-9355-d2d5a95d1146 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:04.993962 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-882e3530-db25-48af-9355-d2d5a95d1146 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:04.994650 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-882e3530-db25-48af-9355-d2d5a95d1146 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:04.995104 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-882e3530-db25-48af-9355-d2d5a95d1146 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:04.995528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-882e3530-db25-48af-9355-d2d5a95d1146 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:05.048225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5f938d4d-be7e-4383-b8cc-e74e868a7566 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:05.048855 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5f938d4d-be7e-4383-b8cc-e74e868a7566 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:05.055926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f938d4d-be7e-4383-b8cc-e74e868a7566 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:05.056301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f938d4d-be7e-4383-b8cc-e74e868a7566 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:05.056699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f938d4d-be7e-4383-b8cc-e74e868a7566 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:05.057455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f938d4d-be7e-4383-b8cc-e74e868a7566 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:05.057827 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-882e3530-db25-48af-9355-d2d5a95d1146 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:05.058017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f938d4d-be7e-4383-b8cc-e74e868a7566 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:05.058289 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5f938d4d-be7e-4383-b8cc-e74e868a7566 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:05.066735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-882e3530-db25-48af-9355-d2d5a95d1146 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.085600 Aug 28 01:23:05.067329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1239/2700] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:04 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:05.090604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3b3fbcb2-049d-49d0-82cc-f220080f7c7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:05.091211 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3b3fbcb2-049d-49d0-82cc-f220080f7c7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:05.097984 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3b3fbcb2-049d-49d0-82cc-f220080f7c7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:05.098460 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3b3fbcb2-049d-49d0-82cc-f220080f7c7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:05.098886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3b3fbcb2-049d-49d0-82cc-f220080f7c7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:05.099572 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3b3fbcb2-049d-49d0-82cc-f220080f7c7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:05.099993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3b3fbcb2-049d-49d0-82cc-f220080f7c7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:05.100469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3b3fbcb2-049d-49d0-82cc-f220080f7c7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:05.175456 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3b3fbcb2-049d-49d0-82cc-f220080f7c7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:05.185384 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3b3fbcb2-049d-49d0-82cc-f220080f7c7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.098460 Aug 28 01:23:05.185981 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1240/2701] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:05 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:05.213018 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5f938d4d-be7e-4383-b8cc-e74e868a7566 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:05.221906 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5f938d4d-be7e-4383-b8cc-e74e868a7566 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:05.223326 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5f938d4d-be7e-4383-b8cc-e74e868a7566 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:05.224501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5f938d4d-be7e-4383-b8cc-e74e868a7566 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.180379 Aug 28 01:23:05.225114 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1462/2702] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:05 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:05.284173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-499e4d41-6f7e-43a5-9ed1-d4d112744eb7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:05.284675 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-499e4d41-6f7e-43a5-9ed1-d4d112744eb7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:05.292304 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-499e4d41-6f7e-43a5-9ed1-d4d112744eb7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:05.292785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-499e4d41-6f7e-43a5-9ed1-d4d112744eb7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:05.293250 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-499e4d41-6f7e-43a5-9ed1-d4d112744eb7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:05.293935 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-499e4d41-6f7e-43a5-9ed1-d4d112744eb7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:05.294377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-499e4d41-6f7e-43a5-9ed1-d4d112744eb7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:05.294808 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-499e4d41-6f7e-43a5-9ed1-d4d112744eb7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:05.438021 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-499e4d41-6f7e-43a5-9ed1-d4d112744eb7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:05.451344 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-499e4d41-6f7e-43a5-9ed1-d4d112744eb7 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.173152 Aug 28 01:23:05.452070 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1241/2703] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:05 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:06.088486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-288235a5-1255-4fef-b180-559bf2e1f278 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:06.089105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-288235a5-1255-4fef-b180-559bf2e1f278 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:06.095938 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-288235a5-1255-4fef-b180-559bf2e1f278 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:06.096412 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-288235a5-1255-4fef-b180-559bf2e1f278 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:06.096870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-288235a5-1255-4fef-b180-559bf2e1f278 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:06.097556 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-288235a5-1255-4fef-b180-559bf2e1f278 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:06.098009 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-288235a5-1255-4fef-b180-559bf2e1f278 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:06.098465 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-288235a5-1255-4fef-b180-559bf2e1f278 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:06.160929 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-288235a5-1255-4fef-b180-559bf2e1f278 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:06.169817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-288235a5-1255-4fef-b180-559bf2e1f278 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.085230 Aug 28 01:23:06.170353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1463/2704] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:06 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:06.206796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-17e7ec1d-a8f0-46bb-bc54-c435c86110c7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:06.207404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-17e7ec1d-a8f0-46bb-bc54-c435c86110c7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:06.214176 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-17e7ec1d-a8f0-46bb-bc54-c435c86110c7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:06.214653 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-17e7ec1d-a8f0-46bb-bc54-c435c86110c7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:06.215073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-17e7ec1d-a8f0-46bb-bc54-c435c86110c7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:06.215750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-17e7ec1d-a8f0-46bb-bc54-c435c86110c7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:06.216193 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-17e7ec1d-a8f0-46bb-bc54-c435c86110c7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:06.216607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-17e7ec1d-a8f0-46bb-bc54-c435c86110c7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:06.247694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-58eac54a-a011-4f86-b05d-21e7aeb3734f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:06.248571 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-58eac54a-a011-4f86-b05d-21e7aeb3734f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:06.258415 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-58eac54a-a011-4f86-b05d-21e7aeb3734f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:06.259064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-58eac54a-a011-4f86-b05d-21e7aeb3734f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:06.259702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-58eac54a-a011-4f86-b05d-21e7aeb3734f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:06.260802 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-58eac54a-a011-4f86-b05d-21e7aeb3734f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:06.261536 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-58eac54a-a011-4f86-b05d-21e7aeb3734f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:06.262239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-58eac54a-a011-4f86-b05d-21e7aeb3734f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:06.286304 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-17e7ec1d-a8f0-46bb-bc54-c435c86110c7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:06.294966 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-17e7ec1d-a8f0-46bb-bc54-c435c86110c7 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.091935 Aug 28 01:23:06.295617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1242/2705] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:06 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:06.399172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-58eac54a-a011-4f86-b05d-21e7aeb3734f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:06.408269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-58eac54a-a011-4f86-b05d-21e7aeb3734f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:06.409732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-58eac54a-a011-4f86-b05d-21e7aeb3734f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:06.410869 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-58eac54a-a011-4f86-b05d-21e7aeb3734f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.167271 Aug 28 01:23:06.411449 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1464/2706] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:06 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:06.475639 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-16b68c1b-cbde-428a-aa20-1b6a951e8e0f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:06.476233 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-16b68c1b-cbde-428a-aa20-1b6a951e8e0f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:06.482785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-16b68c1b-cbde-428a-aa20-1b6a951e8e0f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:06.483260 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-16b68c1b-cbde-428a-aa20-1b6a951e8e0f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:06.483680 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-16b68c1b-cbde-428a-aa20-1b6a951e8e0f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:06.484373 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-16b68c1b-cbde-428a-aa20-1b6a951e8e0f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:06.484827 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-16b68c1b-cbde-428a-aa20-1b6a951e8e0f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:06.485265 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-16b68c1b-cbde-428a-aa20-1b6a951e8e0f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:06.618291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-16b68c1b-cbde-428a-aa20-1b6a951e8e0f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:06.626827 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-16b68c1b-cbde-428a-aa20-1b6a951e8e0f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.155130 Aug 28 01:23:06.627366 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1243/2707] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:06 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:07.195177 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6abd5593-6eda-47c0-a81f-cadc495c45ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:07.195756 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6abd5593-6eda-47c0-a81f-cadc495c45ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:07.202827 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6abd5593-6eda-47c0-a81f-cadc495c45ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:07.203305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6abd5593-6eda-47c0-a81f-cadc495c45ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:07.203727 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6abd5593-6eda-47c0-a81f-cadc495c45ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:07.204405 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6abd5593-6eda-47c0-a81f-cadc495c45ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:07.204847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6abd5593-6eda-47c0-a81f-cadc495c45ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:07.205324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6abd5593-6eda-47c0-a81f-cadc495c45ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:07.267972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6abd5593-6eda-47c0-a81f-cadc495c45ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:07.276455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6abd5593-6eda-47c0-a81f-cadc495c45ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.086698 Aug 28 01:23:07.277017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1465/2708] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:07 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:07.316599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c98f1dd6-6930-4d03-bf7f-048ecf6e1e53 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:07.317207 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c98f1dd6-6930-4d03-bf7f-048ecf6e1e53 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:07.324198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c98f1dd6-6930-4d03-bf7f-048ecf6e1e53 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:07.324686 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c98f1dd6-6930-4d03-bf7f-048ecf6e1e53 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:07.325156 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c98f1dd6-6930-4d03-bf7f-048ecf6e1e53 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:07.325819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c98f1dd6-6930-4d03-bf7f-048ecf6e1e53 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:07.326269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c98f1dd6-6930-4d03-bf7f-048ecf6e1e53 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:07.326689 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c98f1dd6-6930-4d03-bf7f-048ecf6e1e53 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:07.329292 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:23:07.392918 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c98f1dd6-6930-4d03-bf7f-048ecf6e1e53 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:07.401884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c98f1dd6-6930-4d03-bf7f-048ecf6e1e53 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089564 Aug 28 01:23:07.402565 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1244/2709] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:07 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:07.434856 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-84bc192c-6922-4040-949d-2396af5ae366 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:07.435484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-84bc192c-6922-4040-949d-2396af5ae366 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:07.442397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84bc192c-6922-4040-949d-2396af5ae366 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:07.442872 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84bc192c-6922-4040-949d-2396af5ae366 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:07.443299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84bc192c-6922-4040-949d-2396af5ae366 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:07.444096 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84bc192c-6922-4040-949d-2396af5ae366 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:07.444529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84bc192c-6922-4040-949d-2396af5ae366 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:07.444940 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84bc192c-6922-4040-949d-2396af5ae366 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:07.587959 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-84bc192c-6922-4040-949d-2396af5ae366 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:07.597120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-84bc192c-6922-4040-949d-2396af5ae366 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:07.598548 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-84bc192c-6922-4040-949d-2396af5ae366 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:07.599699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-84bc192c-6922-4040-949d-2396af5ae366 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.168819 Aug 28 01:23:07.600295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1466/2710] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:07 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:07.651581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-54a128b8-aa25-4c06-a416-401d73510821 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:07.652342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-54a128b8-aa25-4c06-a416-401d73510821 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:07.662078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54a128b8-aa25-4c06-a416-401d73510821 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:07.662803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54a128b8-aa25-4c06-a416-401d73510821 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:07.663232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54a128b8-aa25-4c06-a416-401d73510821 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:07.663914 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54a128b8-aa25-4c06-a416-401d73510821 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:07.664361 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54a128b8-aa25-4c06-a416-401d73510821 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:07.664796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-54a128b8-aa25-4c06-a416-401d73510821 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:07.807194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-54a128b8-aa25-4c06-a416-401d73510821 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:07.816168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-54a128b8-aa25-4c06-a416-401d73510821 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.170494 Aug 28 01:23:07.816736 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1245/2711] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:07 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:08.298047 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-52cbac4d-13d0-412f-a24a-9c91c3a4edc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:08.298714 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-52cbac4d-13d0-412f-a24a-9c91c3a4edc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:08.305923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52cbac4d-13d0-412f-a24a-9c91c3a4edc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:08.306411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52cbac4d-13d0-412f-a24a-9c91c3a4edc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:08.306839 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52cbac4d-13d0-412f-a24a-9c91c3a4edc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:08.307535 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52cbac4d-13d0-412f-a24a-9c91c3a4edc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:08.307987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52cbac4d-13d0-412f-a24a-9c91c3a4edc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:08.308407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52cbac4d-13d0-412f-a24a-9c91c3a4edc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:08.380381 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-52cbac4d-13d0-412f-a24a-9c91c3a4edc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:08.389253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-52cbac4d-13d0-412f-a24a-9c91c3a4edc4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.095417 Aug 28 01:23:08.389823 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1467/2712] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:08 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:08.423993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ac2aacef-ee4b-406d-9e56-8b909be987f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:08.424622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ac2aacef-ee4b-406d-9e56-8b909be987f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:08.433005 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac2aacef-ee4b-406d-9e56-8b909be987f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:08.433633 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac2aacef-ee4b-406d-9e56-8b909be987f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:08.434204 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac2aacef-ee4b-406d-9e56-8b909be987f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:08.434901 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac2aacef-ee4b-406d-9e56-8b909be987f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:08.435350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac2aacef-ee4b-406d-9e56-8b909be987f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:08.435772 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac2aacef-ee4b-406d-9e56-8b909be987f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:08.500087 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ac2aacef-ee4b-406d-9e56-8b909be987f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:08.512177 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ac2aacef-ee4b-406d-9e56-8b909be987f2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.091775 Aug 28 01:23:08.512919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1246/2713] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:08 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:08.623276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3756ea56-a44f-4501-a48c-91cd5e597309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:08.623870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3756ea56-a44f-4501-a48c-91cd5e597309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:08.630261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3756ea56-a44f-4501-a48c-91cd5e597309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:08.630771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3756ea56-a44f-4501-a48c-91cd5e597309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:08.631164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3756ea56-a44f-4501-a48c-91cd5e597309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:08.631862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3756ea56-a44f-4501-a48c-91cd5e597309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:08.632307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3756ea56-a44f-4501-a48c-91cd5e597309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:08.632723 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3756ea56-a44f-4501-a48c-91cd5e597309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:08.772205 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3756ea56-a44f-4501-a48c-91cd5e597309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:08.780878 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3756ea56-a44f-4501-a48c-91cd5e597309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:08.782320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3756ea56-a44f-4501-a48c-91cd5e597309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:08.783464 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3756ea56-a44f-4501-a48c-91cd5e597309 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.164150 Aug 28 01:23:08.784016 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1468/2714] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:08 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:08.841055 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5a613a5a-38d6-4dbc-ba45-3f114dbb6ceb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:08.842093 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5a613a5a-38d6-4dbc-ba45-3f114dbb6ceb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:08.851530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a613a5a-38d6-4dbc-ba45-3f114dbb6ceb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:08.852009 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a613a5a-38d6-4dbc-ba45-3f114dbb6ceb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:08.852436 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a613a5a-38d6-4dbc-ba45-3f114dbb6ceb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:08.853158 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a613a5a-38d6-4dbc-ba45-3f114dbb6ceb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:08.853607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a613a5a-38d6-4dbc-ba45-3f114dbb6ceb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:08.854021 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a613a5a-38d6-4dbc-ba45-3f114dbb6ceb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:08.998590 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5a613a5a-38d6-4dbc-ba45-3f114dbb6ceb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:09.009781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5a613a5a-38d6-4dbc-ba45-3f114dbb6ceb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.175200 Aug 28 01:23:09.010361 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1247/2715] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:08 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:09.411010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9128d11d-301c-4bd7-ab5b-80129cd859ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:09.411640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9128d11d-301c-4bd7-ab5b-80129cd859ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:09.418832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9128d11d-301c-4bd7-ab5b-80129cd859ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:09.419320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9128d11d-301c-4bd7-ab5b-80129cd859ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:09.419747 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9128d11d-301c-4bd7-ab5b-80129cd859ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:09.420443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9128d11d-301c-4bd7-ab5b-80129cd859ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:09.420886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9128d11d-301c-4bd7-ab5b-80129cd859ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:09.421333 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9128d11d-301c-4bd7-ab5b-80129cd859ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:09.484438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9128d11d-301c-4bd7-ab5b-80129cd859ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:09.493057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9128d11d-301c-4bd7-ab5b-80129cd859ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.085904 Aug 28 01:23:09.493638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1469/2716] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:09 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:09.536787 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f43f122a-2be5-4159-ba0b-b43f3551149c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:09.537415 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f43f122a-2be5-4159-ba0b-b43f3551149c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:09.547195 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f43f122a-2be5-4159-ba0b-b43f3551149c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:09.547842 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f43f122a-2be5-4159-ba0b-b43f3551149c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:09.548406 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f43f122a-2be5-4159-ba0b-b43f3551149c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:09.549529 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f43f122a-2be5-4159-ba0b-b43f3551149c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:09.550126 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f43f122a-2be5-4159-ba0b-b43f3551149c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:09.550687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f43f122a-2be5-4159-ba0b-b43f3551149c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:09.617677 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f43f122a-2be5-4159-ba0b-b43f3551149c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:09.629563 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f43f122a-2be5-4159-ba0b-b43f3551149c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.097133 Aug 28 01:23:09.630455 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1248/2717] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:09 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:09.807872 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5cc97435-3ba7-4c51-9921-4c124ea83e4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:09.808551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5cc97435-3ba7-4c51-9921-4c124ea83e4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:09.816234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5cc97435-3ba7-4c51-9921-4c124ea83e4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:09.816793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5cc97435-3ba7-4c51-9921-4c124ea83e4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:09.817428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5cc97435-3ba7-4c51-9921-4c124ea83e4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:09.818222 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5cc97435-3ba7-4c51-9921-4c124ea83e4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:09.818828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5cc97435-3ba7-4c51-9921-4c124ea83e4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:09.819242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5cc97435-3ba7-4c51-9921-4c124ea83e4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:09.956940 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5cc97435-3ba7-4c51-9921-4c124ea83e4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:09.966116 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5cc97435-3ba7-4c51-9921-4c124ea83e4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:09.967539 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5cc97435-3ba7-4c51-9921-4c124ea83e4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:09.968696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5cc97435-3ba7-4c51-9921-4c124ea83e4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.165593 Aug 28 01:23:09.969281 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1470/2718] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:09 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:10.039681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7bc9f5ea-3936-4424-af11-331df4192eae tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:10.040292 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7bc9f5ea-3936-4424-af11-331df4192eae tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:10.047039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7bc9f5ea-3936-4424-af11-331df4192eae tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:10.047522 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7bc9f5ea-3936-4424-af11-331df4192eae tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:10.047951 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7bc9f5ea-3936-4424-af11-331df4192eae tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:10.048635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7bc9f5ea-3936-4424-af11-331df4192eae tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:10.049112 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7bc9f5ea-3936-4424-af11-331df4192eae tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:10.049533 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7bc9f5ea-3936-4424-af11-331df4192eae tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:10.185216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7bc9f5ea-3936-4424-af11-331df4192eae tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:10.194265 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7bc9f5ea-3936-4424-af11-331df4192eae tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.158504 Aug 28 01:23:10.194847 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1249/2719] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:10 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:10.517719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8286cf77-3cd1-49fc-a91b-04a8c302db4c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:10.518422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8286cf77-3cd1-49fc-a91b-04a8c302db4c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:10.526874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8286cf77-3cd1-49fc-a91b-04a8c302db4c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:10.527445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8286cf77-3cd1-49fc-a91b-04a8c302db4c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:10.527984 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8286cf77-3cd1-49fc-a91b-04a8c302db4c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:10.528839 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8286cf77-3cd1-49fc-a91b-04a8c302db4c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:10.529459 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8286cf77-3cd1-49fc-a91b-04a8c302db4c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:10.529989 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8286cf77-3cd1-49fc-a91b-04a8c302db4c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:10.601507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8286cf77-3cd1-49fc-a91b-04a8c302db4c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:10.611150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8286cf77-3cd1-49fc-a91b-04a8c302db4c tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.098378 Aug 28 01:23:10.611821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1471/2720] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:10 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:10.660556 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ac28af37-88b7-49b1-abd3-085ef5dcc240 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:10.661205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ac28af37-88b7-49b1-abd3-085ef5dcc240 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:10.668883 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac28af37-88b7-49b1-abd3-085ef5dcc240 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:10.669392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac28af37-88b7-49b1-abd3-085ef5dcc240 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:10.669818 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac28af37-88b7-49b1-abd3-085ef5dcc240 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:10.670507 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac28af37-88b7-49b1-abd3-085ef5dcc240 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:10.670950 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac28af37-88b7-49b1-abd3-085ef5dcc240 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:10.671367 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac28af37-88b7-49b1-abd3-085ef5dcc240 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:10.740912 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ac28af37-88b7-49b1-abd3-085ef5dcc240 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:10.749312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ac28af37-88b7-49b1-abd3-085ef5dcc240 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.094892 Aug 28 01:23:10.749865 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1250/2721] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:10 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:10.992633 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4e7a082c-3031-4e02-9955-4c7f2bab537d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:10.993273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4e7a082c-3031-4e02-9955-4c7f2bab537d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:10.999608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4e7a082c-3031-4e02-9955-4c7f2bab537d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:11.000094 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4e7a082c-3031-4e02-9955-4c7f2bab537d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:11.000525 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4e7a082c-3031-4e02-9955-4c7f2bab537d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:11.001320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4e7a082c-3031-4e02-9955-4c7f2bab537d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:11.001772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4e7a082c-3031-4e02-9955-4c7f2bab537d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:11.002191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4e7a082c-3031-4e02-9955-4c7f2bab537d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:11.143171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4e7a082c-3031-4e02-9955-4c7f2bab537d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:11.152443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4e7a082c-3031-4e02-9955-4c7f2bab537d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:11.153907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4e7a082c-3031-4e02-9955-4c7f2bab537d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:11.155019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4e7a082c-3031-4e02-9955-4c7f2bab537d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.166762 Aug 28 01:23:11.155569 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1472/2722] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:10 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:11.222961 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7f53c122-4298-4b69-bea5-c1825b24730f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:11.223648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7f53c122-4298-4b69-bea5-c1825b24730f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:11.230789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7f53c122-4298-4b69-bea5-c1825b24730f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:11.231271 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7f53c122-4298-4b69-bea5-c1825b24730f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:11.231702 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7f53c122-4298-4b69-bea5-c1825b24730f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:11.232392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7f53c122-4298-4b69-bea5-c1825b24730f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:11.232837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7f53c122-4298-4b69-bea5-c1825b24730f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:11.233288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7f53c122-4298-4b69-bea5-c1825b24730f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:11.372543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7f53c122-4298-4b69-bea5-c1825b24730f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:11.381830 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7f53c122-4298-4b69-bea5-c1825b24730f tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.165060 Aug 28 01:23:11.382394 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1251/2723] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:11 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:11.632544 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-75e73862-ddfd-476c-8217-22e6e944ec6d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:11.633174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-75e73862-ddfd-476c-8217-22e6e944ec6d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:11.640073 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75e73862-ddfd-476c-8217-22e6e944ec6d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:11.640552 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75e73862-ddfd-476c-8217-22e6e944ec6d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:11.641007 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75e73862-ddfd-476c-8217-22e6e944ec6d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:11.641751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75e73862-ddfd-476c-8217-22e6e944ec6d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:11.642197 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75e73862-ddfd-476c-8217-22e6e944ec6d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:11.642614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-75e73862-ddfd-476c-8217-22e6e944ec6d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:11.658394 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:23:11.712457 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-75e73862-ddfd-476c-8217-22e6e944ec6d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:11.721440 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-75e73862-ddfd-476c-8217-22e6e944ec6d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.092759 Aug 28 01:23:11.721989 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1473/2724] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:11 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:11.769987 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8a2ac4b6-509b-443b-8cc0-9d33cb494132 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:11.770577 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8a2ac4b6-509b-443b-8cc0-9d33cb494132 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:11.777570 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a2ac4b6-509b-443b-8cc0-9d33cb494132 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:11.778047 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a2ac4b6-509b-443b-8cc0-9d33cb494132 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:11.778492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a2ac4b6-509b-443b-8cc0-9d33cb494132 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:11.779192 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a2ac4b6-509b-443b-8cc0-9d33cb494132 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:11.779648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a2ac4b6-509b-443b-8cc0-9d33cb494132 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:11.780063 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a2ac4b6-509b-443b-8cc0-9d33cb494132 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:11.848059 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8a2ac4b6-509b-443b-8cc0-9d33cb494132 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:11.857188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8a2ac4b6-509b-443b-8cc0-9d33cb494132 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.091028 Aug 28 01:23:11.857764 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1252/2725] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:11 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:12.181176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-89a7b96a-9f9d-4af8-a488-c3f3a02deff0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:12.181777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-89a7b96a-9f9d-4af8-a488-c3f3a02deff0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:12.188307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-89a7b96a-9f9d-4af8-a488-c3f3a02deff0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:12.188788 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-89a7b96a-9f9d-4af8-a488-c3f3a02deff0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:12.189245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-89a7b96a-9f9d-4af8-a488-c3f3a02deff0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:12.189922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-89a7b96a-9f9d-4af8-a488-c3f3a02deff0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:12.190706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-89a7b96a-9f9d-4af8-a488-c3f3a02deff0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:12.191122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-89a7b96a-9f9d-4af8-a488-c3f3a02deff0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:12.330986 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-89a7b96a-9f9d-4af8-a488-c3f3a02deff0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:12.339725 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-89a7b96a-9f9d-4af8-a488-c3f3a02deff0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:12.341163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-89a7b96a-9f9d-4af8-a488-c3f3a02deff0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:12.342282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-89a7b96a-9f9d-4af8-a488-c3f3a02deff0 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.165212 Aug 28 01:23:12.342829 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1474/2726] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:12 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:12.405972 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-29665f46-d4fe-4c9c-9aa5-8a30266352ed tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:12.406579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-29665f46-d4fe-4c9c-9aa5-8a30266352ed tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:12.415624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-29665f46-d4fe-4c9c-9aa5-8a30266352ed tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:12.416272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-29665f46-d4fe-4c9c-9aa5-8a30266352ed tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:12.416926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-29665f46-d4fe-4c9c-9aa5-8a30266352ed tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:12.418031 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-29665f46-d4fe-4c9c-9aa5-8a30266352ed tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:12.418689 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-29665f46-d4fe-4c9c-9aa5-8a30266352ed tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:12.419319 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-29665f46-d4fe-4c9c-9aa5-8a30266352ed tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:12.568234 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-29665f46-d4fe-4c9c-9aa5-8a30266352ed tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:12.576569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-29665f46-d4fe-4c9c-9aa5-8a30266352ed tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.176004 Aug 28 01:23:12.577158 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1253/2727] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:12 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:12.742528 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ead10d84-f1eb-4307-b6a8-1677df76e4b7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:12.743144 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ead10d84-f1eb-4307-b6a8-1677df76e4b7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:12.749888 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ead10d84-f1eb-4307-b6a8-1677df76e4b7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:12.750372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ead10d84-f1eb-4307-b6a8-1677df76e4b7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:12.750800 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ead10d84-f1eb-4307-b6a8-1677df76e4b7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:12.751485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ead10d84-f1eb-4307-b6a8-1677df76e4b7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:12.751934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ead10d84-f1eb-4307-b6a8-1677df76e4b7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:12.752353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ead10d84-f1eb-4307-b6a8-1677df76e4b7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:12.815093 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ead10d84-f1eb-4307-b6a8-1677df76e4b7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:12.824088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ead10d84-f1eb-4307-b6a8-1677df76e4b7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.085589 Aug 28 01:23:12.824653 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1475/2728] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:12 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:12.880401 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1eff3d6f-eb93-41b5-9c51-65b548fbc9fe tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:12.881043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1eff3d6f-eb93-41b5-9c51-65b548fbc9fe tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:12.887982 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1eff3d6f-eb93-41b5-9c51-65b548fbc9fe tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:12.888468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1eff3d6f-eb93-41b5-9c51-65b548fbc9fe tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:12.888901 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1eff3d6f-eb93-41b5-9c51-65b548fbc9fe tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:12.889711 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1eff3d6f-eb93-41b5-9c51-65b548fbc9fe tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:12.890714 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1eff3d6f-eb93-41b5-9c51-65b548fbc9fe tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:12.890714 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1eff3d6f-eb93-41b5-9c51-65b548fbc9fe tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:12.956593 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1eff3d6f-eb93-41b5-9c51-65b548fbc9fe tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:12.965609 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1eff3d6f-eb93-41b5-9c51-65b548fbc9fe tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.090903 Aug 28 01:23:12.966198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1254/2729] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:12 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:13.364944 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f7182cc7-525b-43e0-b30f-77893fabbc4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:13.365578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f7182cc7-525b-43e0-b30f-77893fabbc4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:13.371990 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7182cc7-525b-43e0-b30f-77893fabbc4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:13.372537 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7182cc7-525b-43e0-b30f-77893fabbc4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:13.372895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7182cc7-525b-43e0-b30f-77893fabbc4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:13.373614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7182cc7-525b-43e0-b30f-77893fabbc4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:13.374062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7182cc7-525b-43e0-b30f-77893fabbc4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:13.374469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7182cc7-525b-43e0-b30f-77893fabbc4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:13.503315 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f7182cc7-525b-43e0-b30f-77893fabbc4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:13.512248 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f7182cc7-525b-43e0-b30f-77893fabbc4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:13.513707 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f7182cc7-525b-43e0-b30f-77893fabbc4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:13.514862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f7182cc7-525b-43e0-b30f-77893fabbc4a tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.153867 Aug 28 01:23:13.515429 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1476/2730] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:13 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:13.598307 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1dab273c-53d1-48ae-a146-ce5eca3c918a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:13.598915 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1dab273c-53d1-48ae-a146-ce5eca3c918a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:13.605547 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1dab273c-53d1-48ae-a146-ce5eca3c918a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:13.606050 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1dab273c-53d1-48ae-a146-ce5eca3c918a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:13.606480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1dab273c-53d1-48ae-a146-ce5eca3c918a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:13.607171 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1dab273c-53d1-48ae-a146-ce5eca3c918a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:13.607619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1dab273c-53d1-48ae-a146-ce5eca3c918a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:13.608035 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1dab273c-53d1-48ae-a146-ce5eca3c918a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:13.750377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1dab273c-53d1-48ae-a146-ce5eca3c918a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:13.762907 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1dab273c-53d1-48ae-a146-ce5eca3c918a tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.168131 Aug 28 01:23:13.763844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1255/2731] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:13 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:13.846904 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d1c37696-eab2-4f76-b6e4-717b5768e285 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:13.847657 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d1c37696-eab2-4f76-b6e4-717b5768e285 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:13.856123 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d1c37696-eab2-4f76-b6e4-717b5768e285 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:13.856699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d1c37696-eab2-4f76-b6e4-717b5768e285 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:13.857290 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d1c37696-eab2-4f76-b6e4-717b5768e285 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:13.858137 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d1c37696-eab2-4f76-b6e4-717b5768e285 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:13.858689 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d1c37696-eab2-4f76-b6e4-717b5768e285 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:13.859219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d1c37696-eab2-4f76-b6e4-717b5768e285 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:13.921643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d1c37696-eab2-4f76-b6e4-717b5768e285 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:13.930420 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d1c37696-eab2-4f76-b6e4-717b5768e285 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.088323 Aug 28 01:23:13.931000 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1477/2732] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:13 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:14.006262 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-398a5bd6-c2a8-4440-9648-710d53a8b149 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:14.006834 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-398a5bd6-c2a8-4440-9648-710d53a8b149 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:14.013849 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-398a5bd6-c2a8-4440-9648-710d53a8b149 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:14.014382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-398a5bd6-c2a8-4440-9648-710d53a8b149 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:14.014783 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-398a5bd6-c2a8-4440-9648-710d53a8b149 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:14.015468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-398a5bd6-c2a8-4440-9648-710d53a8b149 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:14.015910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-398a5bd6-c2a8-4440-9648-710d53a8b149 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:14.016327 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-398a5bd6-c2a8-4440-9648-710d53a8b149 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:14.080673 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-398a5bd6-c2a8-4440-9648-710d53a8b149 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:14.093112 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-398a5bd6-c2a8-4440-9648-710d53a8b149 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.090558 Aug 28 01:23:14.093765 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1256/2733] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:14 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:14.539300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f0e11984-6d8f-483f-85cb-1b4420a534c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:14.539918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f0e11984-6d8f-483f-85cb-1b4420a534c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:14.546495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0e11984-6d8f-483f-85cb-1b4420a534c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:14.546979 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0e11984-6d8f-483f-85cb-1b4420a534c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:14.547411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0e11984-6d8f-483f-85cb-1b4420a534c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:14.548122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0e11984-6d8f-483f-85cb-1b4420a534c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:14.548539 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0e11984-6d8f-483f-85cb-1b4420a534c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:14.548928 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0e11984-6d8f-483f-85cb-1b4420a534c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:14.685200 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f0e11984-6d8f-483f-85cb-1b4420a534c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:14.694761 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f0e11984-6d8f-483f-85cb-1b4420a534c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:14.696301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f0e11984-6d8f-483f-85cb-1b4420a534c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:14.697490 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f0e11984-6d8f-483f-85cb-1b4420a534c5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.162537 Aug 28 01:23:14.698071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1478/2734] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:14 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:14.787870 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2ee9b590-367a-4edd-b7e6-913bad1965fb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:14.788413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2ee9b590-367a-4edd-b7e6-913bad1965fb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:14.794890 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ee9b590-367a-4edd-b7e6-913bad1965fb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:14.795375 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ee9b590-367a-4edd-b7e6-913bad1965fb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:14.795804 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ee9b590-367a-4edd-b7e6-913bad1965fb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:14.796491 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ee9b590-367a-4edd-b7e6-913bad1965fb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:14.796934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ee9b590-367a-4edd-b7e6-913bad1965fb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:14.797386 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ee9b590-367a-4edd-b7e6-913bad1965fb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:14.933359 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2ee9b590-367a-4edd-b7e6-913bad1965fb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:14.942617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2ee9b590-367a-4edd-b7e6-913bad1965fb tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.158725 Aug 28 01:23:14.943262 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1257/2735] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:14 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:14.951571 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7fba67d4-3d30-4c9d-8fde-f6cc066898a1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:14.952287 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7fba67d4-3d30-4c9d-8fde-f6cc066898a1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:14.959495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fba67d4-3d30-4c9d-8fde-f6cc066898a1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:14.959978 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fba67d4-3d30-4c9d-8fde-f6cc066898a1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:14.960409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fba67d4-3d30-4c9d-8fde-f6cc066898a1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:14.961140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fba67d4-3d30-4c9d-8fde-f6cc066898a1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:14.961584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fba67d4-3d30-4c9d-8fde-f6cc066898a1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:14.962002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7fba67d4-3d30-4c9d-8fde-f6cc066898a1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:15.025897 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7fba67d4-3d30-4c9d-8fde-f6cc066898a1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:15.034843 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7fba67d4-3d30-4c9d-8fde-f6cc066898a1 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.087111 Aug 28 01:23:15.035426 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1479/2736] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:14 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:15.120224 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cacb18e0-8900-46ea-bdbe-a76d25ff9bf5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:15.121372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cacb18e0-8900-46ea-bdbe-a76d25ff9bf5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:15.130377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cacb18e0-8900-46ea-bdbe-a76d25ff9bf5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:15.130873 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cacb18e0-8900-46ea-bdbe-a76d25ff9bf5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:15.131340 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cacb18e0-8900-46ea-bdbe-a76d25ff9bf5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:15.132036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cacb18e0-8900-46ea-bdbe-a76d25ff9bf5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:15.132484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cacb18e0-8900-46ea-bdbe-a76d25ff9bf5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:15.132895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cacb18e0-8900-46ea-bdbe-a76d25ff9bf5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:15.204638 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-cacb18e0-8900-46ea-bdbe-a76d25ff9bf5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:15.213333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cacb18e0-8900-46ea-bdbe-a76d25ff9bf5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.101716 Aug 28 01:23:15.213919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1258/2737] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:15 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:15.724933 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-56597099-432e-4dfc-a6d0-7a22c06f2ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:15.725565 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-56597099-432e-4dfc-a6d0-7a22c06f2ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:15.732152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56597099-432e-4dfc-a6d0-7a22c06f2ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:15.732627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56597099-432e-4dfc-a6d0-7a22c06f2ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:15.733086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56597099-432e-4dfc-a6d0-7a22c06f2ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:15.733781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56597099-432e-4dfc-a6d0-7a22c06f2ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:15.734210 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56597099-432e-4dfc-a6d0-7a22c06f2ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:15.734647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56597099-432e-4dfc-a6d0-7a22c06f2ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:15.871032 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-56597099-432e-4dfc-a6d0-7a22c06f2ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:15.879632 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-56597099-432e-4dfc-a6d0-7a22c06f2ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:15.881170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-56597099-432e-4dfc-a6d0-7a22c06f2ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:15.882313 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-56597099-432e-4dfc-a6d0-7a22c06f2ba5 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.162618 Aug 28 01:23:15.882878 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1480/2738] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:15 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:15.964016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-96f134e3-2980-4b8e-a80f-a6970a9ba7a4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:15.964622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-96f134e3-2980-4b8e-a80f-a6970a9ba7a4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] [instance: 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:15.970713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96f134e3-2980-4b8e-a80f-a6970a9ba7a4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:15.971187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96f134e3-2980-4b8e-a80f-a6970a9ba7a4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:15.971611 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96f134e3-2980-4b8e-a80f-a6970a9ba7a4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:15.972324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96f134e3-2980-4b8e-a80f-a6970a9ba7a4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:15.972761 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96f134e3-2980-4b8e-a80f-a6970a9ba7a4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:15.973205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96f134e3-2980-4b8e-a80f-a6970a9ba7a4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:16.058077 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-93e55b1d-4414-41f9-bed3-c2075583ad74 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:16.058699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-93e55b1d-4414-41f9-bed3-c2075583ad74 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:16.066019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93e55b1d-4414-41f9-bed3-c2075583ad74 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:16.066502 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93e55b1d-4414-41f9-bed3-c2075583ad74 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:16.066934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93e55b1d-4414-41f9-bed3-c2075583ad74 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:16.067627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93e55b1d-4414-41f9-bed3-c2075583ad74 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:16.068076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93e55b1d-4414-41f9-bed3-c2075583ad74 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:16.068477 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-93e55b1d-4414-41f9-bed3-c2075583ad74 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:16.125479 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-96f134e3-2980-4b8e-a80f-a6970a9ba7a4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '93ee4e8157364093bc9cdbe980bde4bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '89dfee143ed147abb6f2339958412e7e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:16.134629 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-96f134e3-2980-4b8e-a80f-a6970a9ba7a4 tempest-VolumesSnapshotTestJSON-1940777549 tempest-VolumesSnapshotTestJSON-1940777549-project] 10.222.0.42 "GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc" status: 200 len: 1559 microversion: 2.1 time: 0.174412 Aug 28 01:23:16.135296 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1259/2739] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:15 2022] GET /compute/v2.1/servers/2cbc0806-fe4b-4018-93bf-b4cf3c292bdc => generated 1559 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:16.143649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-93e55b1d-4414-41f9-bed3-c2075583ad74 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:16.153938 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-93e55b1d-4414-41f9-bed3-c2075583ad74 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.099651 Aug 28 01:23:16.154511 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1481/2740] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:16 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:23:16.237331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-89f6f188-5a6f-44c7-bdd1-24f016df1901 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:16.238369 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-89f6f188-5a6f-44c7-bdd1-24f016df1901 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:16.245221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-89f6f188-5a6f-44c7-bdd1-24f016df1901 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:16.245850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-89f6f188-5a6f-44c7-bdd1-24f016df1901 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:16.246282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-89f6f188-5a6f-44c7-bdd1-24f016df1901 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:16.247199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-89f6f188-5a6f-44c7-bdd1-24f016df1901 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:16.247835 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-89f6f188-5a6f-44c7-bdd1-24f016df1901 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:16.248255 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-89f6f188-5a6f-44c7-bdd1-24f016df1901 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:16.317994 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-89f6f188-5a6f-44c7-bdd1-24f016df1901 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:16.320292 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:23:16.329380 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-89f6f188-5a6f-44c7-bdd1-24f016df1901 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.095696 Aug 28 01:23:16.337583 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1260/2741] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:16 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:16.906338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a5755100-e458-4791-b472-d43da5c4b15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:16.906936 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a5755100-e458-4791-b472-d43da5c4b15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:16.913363 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5755100-e458-4791-b472-d43da5c4b15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:16.913835 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5755100-e458-4791-b472-d43da5c4b15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:16.914258 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5755100-e458-4791-b472-d43da5c4b15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:16.914936 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5755100-e458-4791-b472-d43da5c4b15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:16.915382 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5755100-e458-4791-b472-d43da5c4b15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:16.915798 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a5755100-e458-4791-b472-d43da5c4b15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:17.060309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a5755100-e458-4791-b472-d43da5c4b15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:17.070458 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a5755100-e458-4791-b472-d43da5c4b15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:17.071911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a5755100-e458-4791-b472-d43da5c4b15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:17.073097 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a5755100-e458-4791-b472-d43da5c4b15c tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1799 microversion: 2.20 time: 0.170726 Aug 28 01:23:17.073655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1482/2742] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:16 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1799 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:17.096268 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-68736449-6f62-49bf-bab0-346f4234eee8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '.version_select at 0x7f0e6e945c10>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:17.119692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-68736449-6f62-49bf-bab0-346f4234eee8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1316453355" status: 204 len: 0 microversion: 2.20 time: 0.027206 Aug 28 01:23:17.120362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1261/2743] 10.222.0.42 () {60 vars in 1262 bytes} [Sun Aug 28 01:23:17 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1316453355 => generated 0 bytes in 28 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Aug 28 01:23:17.179170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ce481a01-b137-4966-a2e7-2ca295b9270f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:17.180196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ce481a01-b137-4966-a2e7-2ca295b9270f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:17.189562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce481a01-b137-4966-a2e7-2ca295b9270f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:17.190735 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce481a01-b137-4966-a2e7-2ca295b9270f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:17.191160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce481a01-b137-4966-a2e7-2ca295b9270f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:17.191930 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce481a01-b137-4966-a2e7-2ca295b9270f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:17.192300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce481a01-b137-4966-a2e7-2ca295b9270f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:17.192719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce481a01-b137-4966-a2e7-2ca295b9270f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:17.272932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ce481a01-b137-4966-a2e7-2ca295b9270f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:17.287553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ce481a01-b137-4966-a2e7-2ca295b9270f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 200 len: 1373 microversion: 2.1 time: 0.113216 Aug 28 01:23:17.288556 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1483/2744] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:17 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 1373 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:17.309670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8a17331f-feec-4d08-9747-b3794baa33cc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '.version_select at 0x7f0e6e9275e0>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:17.348481 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8a17331f-feec-4d08-9747-b3794baa33cc tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1836615983" status: 202 len: 0 microversion: 2.1 time: 0.043122 Aug 28 01:23:17.350797 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1262/2745] 10.222.0.42 () {58 vars in 1221 bytes} [Sun Aug 28 01:23:17 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1836615983 => generated 0 bytes in 45 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 01:23:17.366488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fbacd47f-d68e-4b92-b463-abaf5fec9c36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:17.367386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fbacd47f-d68e-4b92-b463-abaf5fec9c36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:17.382185 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fbacd47f-d68e-4b92-b463-abaf5fec9c36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:17.383074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fbacd47f-d68e-4b92-b463-abaf5fec9c36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:17.396549 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fbacd47f-d68e-4b92-b463-abaf5fec9c36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:17.397387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fbacd47f-d68e-4b92-b463-abaf5fec9c36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:17.397865 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fbacd47f-d68e-4b92-b463-abaf5fec9c36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:17.398305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fbacd47f-d68e-4b92-b463-abaf5fec9c36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:17.456824 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b6cee3ab-d57f-4a07-8944-ddd38d2a6ae1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:17.457786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b6cee3ab-d57f-4a07-8944-ddd38d2a6ae1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:17.471080 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6cee3ab-d57f-4a07-8944-ddd38d2a6ae1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:17.472062 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6cee3ab-d57f-4a07-8944-ddd38d2a6ae1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:17.474364 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6cee3ab-d57f-4a07-8944-ddd38d2a6ae1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:17.477894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6cee3ab-d57f-4a07-8944-ddd38d2a6ae1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:17.478756 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6cee3ab-d57f-4a07-8944-ddd38d2a6ae1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:17.479473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6cee3ab-d57f-4a07-8944-ddd38d2a6ae1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:17.513885 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b6cee3ab-d57f-4a07-8944-ddd38d2a6ae1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Going to try to terminate instance {{(pid=120524) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 28 01:23:17.534423 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fbacd47f-d68e-4b92-b463-abaf5fec9c36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:17.552038 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fbacd47f-d68e-4b92-b463-abaf5fec9c36 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.200334 Aug 28 01:23:17.552950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1484/2746] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:17 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:17.638426 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b6cee3ab-d57f-4a07-8944-ddd38d2a6ae1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 204 len: 0 microversion: 2.1 time: 0.185712 Aug 28 01:23:17.639053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1263/2747] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 01:23:17 2022] DELETE /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Aug 28 01:23:17.647603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b45653e2-6b87-4dc8-9341-2b87d1c500f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:17.648191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b45653e2-6b87-4dc8-9341-2b87d1c500f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:17.655311 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b45653e2-6b87-4dc8-9341-2b87d1c500f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:17.656007 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b45653e2-6b87-4dc8-9341-2b87d1c500f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:17.656617 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b45653e2-6b87-4dc8-9341-2b87d1c500f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:17.657574 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b45653e2-6b87-4dc8-9341-2b87d1c500f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:17.658386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b45653e2-6b87-4dc8-9341-2b87d1c500f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:17.659031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b45653e2-6b87-4dc8-9341-2b87d1c500f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:17.872438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b45653e2-6b87-4dc8-9341-2b87d1c500f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:17.889318 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b45653e2-6b87-4dc8-9341-2b87d1c500f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.245311 Aug 28 01:23:17.890329 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1485/2748] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:17 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:18.576530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-42d5c976-31f9-43a0-9d72-632180b397db tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:18.577285 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-42d5c976-31f9-43a0-9d72-632180b397db tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:18.584498 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42d5c976-31f9-43a0-9d72-632180b397db tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:18.585004 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42d5c976-31f9-43a0-9d72-632180b397db tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:18.585619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42d5c976-31f9-43a0-9d72-632180b397db tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:18.586527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42d5c976-31f9-43a0-9d72-632180b397db tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:18.587169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42d5c976-31f9-43a0-9d72-632180b397db tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:18.587780 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42d5c976-31f9-43a0-9d72-632180b397db tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:18.690494 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-42d5c976-31f9-43a0-9d72-632180b397db tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:18.698347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-42d5c976-31f9-43a0-9d72-632180b397db tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.126164 Aug 28 01:23:18.699433 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1264/2749] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:18 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:18.754150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-b130cfd8-858c-410d-a5a3-3c964aa400d7 req-592e2206-9a38-45af-845f-ec6c1f43e1b4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dd61123b-7a96-4c9e-936c-5794a4fdd7df", "tag": "522d4a1b-32bc-4106-afb0-5995aecc53c0"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:23:18.764999 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-b130cfd8-858c-410d-a5a3-3c964aa400d7 req-592e2206-9a38-45af-845f-ec6c1f43e1b4 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:18.765491 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-b130cfd8-858c-410d-a5a3-3c964aa400d7 req-592e2206-9a38-45af-845f-ec6c1f43e1b4 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:18.765923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-b130cfd8-858c-410d-a5a3-3c964aa400d7 req-592e2206-9a38-45af-845f-ec6c1f43e1b4 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:18.780190 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-b130cfd8-858c-410d-a5a3-3c964aa400d7 req-592e2206-9a38-45af-845f-ec6c1f43e1b4 service nova] Creating event network-changed:522d4a1b-32bc-4106-afb0-5995aecc53c0 for instance dd61123b-7a96-4c9e-936c-5794a4fdd7df on hv-cinder-maste Aug 28 01:23:18.785711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-b130cfd8-858c-410d-a5a3-3c964aa400d7 req-592e2206-9a38-45af-845f-ec6c1f43e1b4 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.122187 Aug 28 01:23:18.786787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1486/2750] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:23:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:23:18.837425 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ca396f5c-921b-4c19-8368-bdec4a3ec7e8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Action: 'create', calling method: .version_select at 0x7f0e6e8dcc10>, body: {"keypair": {"name": "tempest-keypair-515734887", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHoRdouvk7XazUlOMocSnQh2yFk2Y/2w/3oRZgHAYsMHlfQrcFvsoF7nAQwaPXyi8yMWXGPAJp6f1q9x7n0FmKRF1E0Uqyx++TmOoBQSiA4vNHxRaFnl6Elv4iU2m3dEBw=="}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:23:18.865776 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-ca396f5c-921b-4c19-8368-bdec4a3ec7e8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Getting quotas for project 95df81e14a8549fca0851234f4a00b89. Resources: {'key_pairs'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 01:23:18.872249 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-ca396f5c-921b-4c19-8368-bdec4a3ec7e8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Getting quotas for user d209174d0db04b3584556824c6d0a3d4 and project 95df81e14a8549fca0851234f4a00b89. Resources: {'key_pairs'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 01:23:18.918532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ca396f5c-921b-4c19-8368-bdec4a3ec7e8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.079957 Aug 28 01:23:18.918532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1265/2751] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Aug 28 01:23:18 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:23:18.919049 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f98e05ff-1620-44de-b1a6-06c117d8d1f3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:18.923507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f98e05ff-1620-44de-b1a6-06c117d8d1f3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:18.945762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f98e05ff-1620-44de-b1a6-06c117d8d1f3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:18.947494 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f98e05ff-1620-44de-b1a6-06c117d8d1f3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:18.950216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f98e05ff-1620-44de-b1a6-06c117d8d1f3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:18.950791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f98e05ff-1620-44de-b1a6-06c117d8d1f3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:18.951324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f98e05ff-1620-44de-b1a6-06c117d8d1f3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:18.951765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f98e05ff-1620-44de-b1a6-06c117d8d1f3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:19.005455 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cdfb5f80-a797-4394-ad30-bd0beacda665 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Action: 'create', calling method: .version_select at 0x7f0e6e927dc0>, body: {"keypair": {"name": "tempest-keypair-1126382633", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK5B0bMqCMU/AifgT2tYmc6V4ywXWTguHau0w2x43TbYDTZuJerg3BIACyNKbbhF2c3K8fg17DFYb2C3j/73J7UnExXw9YFOdm7vCPTGdqU6im+EEjPEnJqt4kpqamLsxQ=="}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:23:19.025620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-cdfb5f80-a797-4394-ad30-bd0beacda665 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Getting quotas for project 19f069b9ad7a4a5984fa947ad755db0a. Resources: {'key_pairs'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 01:23:19.031903 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-cdfb5f80-a797-4394-ad30-bd0beacda665 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Getting quotas for user 4894ef8b630e4bc084799c26096f7c66 and project 19f069b9ad7a4a5984fa947ad755db0a. Resources: {'key_pairs'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 01:23:19.036055 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=96,nova_api:DELETE=2,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:23:19.074482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cdfb5f80-a797-4394-ad30-bd0beacda665 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.077348 Aug 28 01:23:19.075821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1266/2752] 10.222.0.42 () {62 vars in 1200 bytes} [Sun Aug 28 01:23:18 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 79 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Aug 28 01:23:19.619729 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f98e05ff-1620-44de-b1a6-06c117d8d1f3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:19.632741 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f98e05ff-1620-44de-b1a6-06c117d8d1f3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.721684 Aug 28 01:23:19.645415 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1487/2753] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:18 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 736 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:19.727343 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a9fc7f1b-a0de-4a10-a98e-77993a8b3f6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:19.728315 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a9fc7f1b-a0de-4a10-a98e-77993a8b3f6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:19.736923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a9fc7f1b-a0de-4a10-a98e-77993a8b3f6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:19.738049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a9fc7f1b-a0de-4a10-a98e-77993a8b3f6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:19.738651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a9fc7f1b-a0de-4a10-a98e-77993a8b3f6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:19.739554 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a9fc7f1b-a0de-4a10-a98e-77993a8b3f6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:19.740375 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a9fc7f1b-a0de-4a10-a98e-77993a8b3f6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:19.740983 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a9fc7f1b-a0de-4a10-a98e-77993a8b3f6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:19.833945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a9fc7f1b-a0de-4a10-a98e-77993a8b3f6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:19.843336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a9fc7f1b-a0de-4a10-a98e-77993a8b3f6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.124440 Aug 28 01:23:19.844866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1267/2754] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:19 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:20.543988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-76c2da19-2424-4637-a895-82b5045b0e7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-2027720462", "imageRef": "2c2585b2-bde5-4652-aed7-9f53b1bd5d94", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "e3dfdb02-2ba5-4b22-9a96-a212c53e9d60"}], "security_groups": [{"name": "tempest-securitygroup--912176284"}], "key_name": "tempest-keypair-515734887", "user_data": "}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:23:20.654967 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7c43a684-c049-450b-8f10-8f14829a5154 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:20.655709 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7c43a684-c049-450b-8f10-8f14829a5154 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:20.663680 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7c43a684-c049-450b-8f10-8f14829a5154 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:20.664168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7c43a684-c049-450b-8f10-8f14829a5154 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:20.664595 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7c43a684-c049-450b-8f10-8f14829a5154 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:20.665319 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7c43a684-c049-450b-8f10-8f14829a5154 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:20.665911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7c43a684-c049-450b-8f10-8f14829a5154 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:20.666553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7c43a684-c049-450b-8f10-8f14829a5154 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:20.729093 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.network.neutron [None req-76c2da19-2424-4637-a895-82b5045b0e7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] validate_networks() for [('e3dfdb02-2ba5-4b22-9a96-a212c53e9d60', None, None, None, None, None)] {{(pid=120525) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 28 01:23:20.997236 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7c43a684-c049-450b-8f10-8f14829a5154 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:21.006171 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7c43a684-c049-450b-8f10-8f14829a5154 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.354885 Aug 28 01:23:21.006734 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1268/2755] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:20 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:21.011495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bc03afe9-a4fb-4b78-b90f-6c82af3b7e6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:21.012109 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bc03afe9-a4fb-4b78-b90f-6c82af3b7e6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:21.018913 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc03afe9-a4fb-4b78-b90f-6c82af3b7e6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:21.019394 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc03afe9-a4fb-4b78-b90f-6c82af3b7e6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:21.019814 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc03afe9-a4fb-4b78-b90f-6c82af3b7e6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:21.020506 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc03afe9-a4fb-4b78-b90f-6c82af3b7e6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:21.021169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc03afe9-a4fb-4b78-b90f-6c82af3b7e6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:21.021812 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc03afe9-a4fb-4b78-b90f-6c82af3b7e6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:21.117777 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bc03afe9-a4fb-4b78-b90f-6c82af3b7e6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:21.127670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bc03afe9-a4fb-4b78-b90f-6c82af3b7e6d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.119912 Aug 28 01:23:21.128241 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1269/2756] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:21 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:21.512665 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-404869f7-2d8d-43fd-be01-f87881219932 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1768312146", "imageRef": "2c2585b2-bde5-4652-aed7-9f53b1bd5d94", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "4c439816-86e3-4522-87f6-42fdb101eeb5"}], "security_groups": [{"name": "tempest-securitygroup--1613128392"}], "key_name": "tempest-keypair-1126382633", "user_data": "}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:23:21.521809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-76c2da19-2424-4637-a895-82b5045b0e7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Flavor limits 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 28 01:23:21.522261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-76c2da19-2424-4637-a895-82b5045b0e7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Image limits 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 28 01:23:21.522738 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-76c2da19-2424-4637-a895-82b5045b0e7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Flavor pref 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 28 01:23:21.523144 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-76c2da19-2424-4637-a895-82b5045b0e7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Image pref 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 28 01:23:21.523620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-76c2da19-2424-4637-a895-82b5045b0e7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 28 01:23:21.535033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76c2da19-2424-4637-a895-82b5045b0e7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:21.535502 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76c2da19-2424-4637-a895-82b5045b0e7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:21.535932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76c2da19-2424-4637-a895-82b5045b0e7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:21.536782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76c2da19-2424-4637-a895-82b5045b0e7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:21.537319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76c2da19-2424-4637-a895-82b5045b0e7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:21.537746 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-76c2da19-2424-4637-a895-82b5045b0e7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:21.558273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-76c2da19-2424-4637-a895-82b5045b0e7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Getting quotas for project 95df81e14a8549fca0851234f4a00b89. Resources: {'ram', 'instances', 'cores'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 01:23:21.563048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-76c2da19-2424-4637-a895-82b5045b0e7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Getting quotas for user d209174d0db04b3584556824c6d0a3d4 and project 95df81e14a8549fca0851234f4a00b89. Resources: {'ram', 'instances', 'cores'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 01:23:21.577472 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-76c2da19-2424-4637-a895-82b5045b0e7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Going to run 1 instances... {{(pid=120525) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 28 01:23:21.579775 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-76c2da19-2424-4637-a895-82b5045b0e7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2c2585b2-bde5-4652-aed7-9f53b1bd5d94',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120525) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 28 01:23:21.635716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-76c2da19-2424-4637-a895-82b5045b0e7d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.096623 Aug 28 01:23:21.636359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1488/2757] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Aug 28 01:23:20 2022] POST /compute/v2.1/servers => generated 401 bytes in 1098 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 28 01:23:21.639096 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.network.neutron [None req-404869f7-2d8d-43fd-be01-f87881219932 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] validate_networks() for [('4c439816-86e3-4522-87f6-42fdb101eeb5', None, None, None, None, None)] {{(pid=120524) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 28 01:23:21.647721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-eb10f07e-faf0-4ccb-a7c9-b06525c2b560 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:21.648359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-eb10f07e-faf0-4ccb-a7c9-b06525c2b560 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:21.734044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-eb10f07e-faf0-4ccb-a7c9-b06525c2b560 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:21.742020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-eb10f07e-faf0-4ccb-a7c9-b06525c2b560 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc" status: 200 len: 1241 microversion: 2.1 time: 0.098352 Aug 28 01:23:21.755355 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1489/2758] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:21 2022] GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc => generated 1241 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:23:22.031666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c4c19557-c602-4873-88fb-50a2b3b818e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:22.032751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c4c19557-c602-4873-88fb-50a2b3b818e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:22.041711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c4c19557-c602-4873-88fb-50a2b3b818e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:22.042440 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c4c19557-c602-4873-88fb-50a2b3b818e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:22.043225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c4c19557-c602-4873-88fb-50a2b3b818e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:22.044704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c4c19557-c602-4873-88fb-50a2b3b818e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:22.045593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c4c19557-c602-4873-88fb-50a2b3b818e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:22.046626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c4c19557-c602-4873-88fb-50a2b3b818e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:22.241776 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-404869f7-2d8d-43fd-be01-f87881219932 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Flavor limits 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 28 01:23:22.242275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-404869f7-2d8d-43fd-be01-f87881219932 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Image limits 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 28 01:23:22.242706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-404869f7-2d8d-43fd-be01-f87881219932 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Flavor pref 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 28 01:23:22.243145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-404869f7-2d8d-43fd-be01-f87881219932 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Image pref 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 28 01:23:22.243543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-404869f7-2d8d-43fd-be01-f87881219932 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 28 01:23:22.253655 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-404869f7-2d8d-43fd-be01-f87881219932 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:22.254282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-404869f7-2d8d-43fd-be01-f87881219932 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:22.255898 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-404869f7-2d8d-43fd-be01-f87881219932 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:22.257377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-404869f7-2d8d-43fd-be01-f87881219932 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:22.257911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-404869f7-2d8d-43fd-be01-f87881219932 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:22.258467 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-404869f7-2d8d-43fd-be01-f87881219932 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:22.274253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c4c19557-c602-4873-88fb-50a2b3b818e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:22.277822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-404869f7-2d8d-43fd-be01-f87881219932 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Getting quotas for project 19f069b9ad7a4a5984fa947ad755db0a. Resources: {'ram', 'instances', 'cores'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 01:23:22.281808 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-404869f7-2d8d-43fd-be01-f87881219932 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Getting quotas for user 4894ef8b630e4bc084799c26096f7c66 and project 19f069b9ad7a4a5984fa947ad755db0a. Resources: {'ram', 'instances', 'cores'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 01:23:22.283851 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c4c19557-c602-4873-88fb-50a2b3b818e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.259310 Aug 28 01:23:22.284902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1490/2759] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:22 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:23:22.289957 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b9875cdd-c6ca-428e-a80c-cd975b03d5ca tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:22.290553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b9875cdd-c6ca-428e-a80c-cd975b03d5ca tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:22.293411 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-404869f7-2d8d-43fd-be01-f87881219932 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Going to run 1 instances... {{(pid=120524) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 28 01:23:22.296080 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-404869f7-2d8d-43fd-be01-f87881219932 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2c2585b2-bde5-4652-aed7-9f53b1bd5d94',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120524) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 28 01:23:22.296558 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9875cdd-c6ca-428e-a80c-cd975b03d5ca tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:22.297117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9875cdd-c6ca-428e-a80c-cd975b03d5ca tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:22.297627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9875cdd-c6ca-428e-a80c-cd975b03d5ca tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:22.298313 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9875cdd-c6ca-428e-a80c-cd975b03d5ca tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:22.298880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9875cdd-c6ca-428e-a80c-cd975b03d5ca tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:22.299306 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9875cdd-c6ca-428e-a80c-cd975b03d5ca tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:22.351220 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-404869f7-2d8d-43fd-be01-f87881219932 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 0.843487 Aug 28 01:23:22.352051 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1270/2760] 10.222.0.42 () {62 vars in 1188 bytes} [Sun Aug 28 01:23:21 2022] POST /compute/v2.1/servers => generated 402 bytes in 845 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Aug 28 01:23:22.364146 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f234b12a-5b13-442c-9a4f-2df47e740fbc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:22.364773 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f234b12a-5b13-442c-9a4f-2df47e740fbc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:22.385905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b9875cdd-c6ca-428e-a80c-cd975b03d5ca tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:22.396712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b9875cdd-c6ca-428e-a80c-cd975b03d5ca tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.110473 Aug 28 01:23:22.397345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1491/2761] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:22 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:22.432858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f234b12a-5b13-442c-9a4f-2df47e740fbc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:22.442827 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f234b12a-5b13-442c-9a4f-2df47e740fbc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:22.444622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f234b12a-5b13-442c-9a4f-2df47e740fbc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:22.446033 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f234b12a-5b13-442c-9a4f-2df47e740fbc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce" status: 200 len: 1282 microversion: 2.20 time: 0.086906 Aug 28 01:23:22.446984 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1271/2762] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:22 2022] GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce => generated 1282 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:22.765650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b9df9ee8-f71d-412b-8316-707401011bcd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:22.766387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b9df9ee8-f71d-412b-8316-707401011bcd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:22.775913 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9df9ee8-f71d-412b-8316-707401011bcd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:22.776646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9df9ee8-f71d-412b-8316-707401011bcd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:22.777298 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9df9ee8-f71d-412b-8316-707401011bcd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:22.778302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9df9ee8-f71d-412b-8316-707401011bcd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:22.778929 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9df9ee8-f71d-412b-8316-707401011bcd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:22.779524 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b9df9ee8-f71d-412b-8316-707401011bcd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:22.795766 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:23:22.843948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b9df9ee8-f71d-412b-8316-707401011bcd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:22.851904 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b9df9ee8-f71d-412b-8316-707401011bcd tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc" status: 200 len: 1293 microversion: 2.1 time: 0.091849 Aug 28 01:23:22.852493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1492/2763] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:22 2022] GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc => generated 1293 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:23.305426 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b0fb87af-312d-4f2d-8bd6-a9697437c601 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:23.306043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b0fb87af-312d-4f2d-8bd6-a9697437c601 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:23.313293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0fb87af-312d-4f2d-8bd6-a9697437c601 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:23.313786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0fb87af-312d-4f2d-8bd6-a9697437c601 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:23.314221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0fb87af-312d-4f2d-8bd6-a9697437c601 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:23.315495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0fb87af-312d-4f2d-8bd6-a9697437c601 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:23.316125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0fb87af-312d-4f2d-8bd6-a9697437c601 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:23.316725 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0fb87af-312d-4f2d-8bd6-a9697437c601 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:23.417434 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a3ed0c5c-2436-4724-a915-4751325372c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:23.418057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a3ed0c5c-2436-4724-a915-4751325372c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:23.424408 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a3ed0c5c-2436-4724-a915-4751325372c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:23.424887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a3ed0c5c-2436-4724-a915-4751325372c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:23.425557 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a3ed0c5c-2436-4724-a915-4751325372c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:23.426256 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a3ed0c5c-2436-4724-a915-4751325372c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:23.426704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a3ed0c5c-2436-4724-a915-4751325372c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:23.427126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a3ed0c5c-2436-4724-a915-4751325372c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:23.509948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a3ed0c5c-2436-4724-a915-4751325372c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:23.518619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a3ed0c5c-2436-4724-a915-4751325372c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.105136 Aug 28 01:23:23.519166 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1493/2764] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:23 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:23:23.524104 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4887a4cc-b911-42b4-80e4-b1506d7e7e31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:23.524916 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4887a4cc-b911-42b4-80e4-b1506d7e7e31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:23.533404 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4887a4cc-b911-42b4-80e4-b1506d7e7e31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:23.533899 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4887a4cc-b911-42b4-80e4-b1506d7e7e31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:23.534330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4887a4cc-b911-42b4-80e4-b1506d7e7e31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:23.535044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4887a4cc-b911-42b4-80e4-b1506d7e7e31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:23.535510 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4887a4cc-b911-42b4-80e4-b1506d7e7e31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:23.535906 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4887a4cc-b911-42b4-80e4-b1506d7e7e31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:23.589651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b0fb87af-312d-4f2d-8bd6-a9697437c601 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:23.600657 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b0fb87af-312d-4f2d-8bd6-a9697437c601 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.299285 Aug 28 01:23:23.601348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1272/2765] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:23 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:23.616756 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4887a4cc-b911-42b4-80e4-b1506d7e7e31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:23.626294 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4887a4cc-b911-42b4-80e4-b1506d7e7e31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:23.627958 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4887a4cc-b911-42b4-80e4-b1506d7e7e31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:23.629181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4887a4cc-b911-42b4-80e4-b1506d7e7e31 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce" status: 200 len: 1334 microversion: 2.20 time: 0.108609 Aug 28 01:23:23.629889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1494/2766] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:23 2022] GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce => generated 1334 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:23.873407 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fd53f64a-1cc2-4950-831e-3712a206e881 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:23.873983 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fd53f64a-1cc2-4950-831e-3712a206e881 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:23.880513 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd53f64a-1cc2-4950-831e-3712a206e881 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:23.881101 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd53f64a-1cc2-4950-831e-3712a206e881 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:23.881537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd53f64a-1cc2-4950-831e-3712a206e881 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:23.882242 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd53f64a-1cc2-4950-831e-3712a206e881 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:23.882673 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd53f64a-1cc2-4950-831e-3712a206e881 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:23.883111 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd53f64a-1cc2-4950-831e-3712a206e881 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:23.952538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fd53f64a-1cc2-4950-831e-3712a206e881 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:23.960387 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fd53f64a-1cc2-4950-831e-3712a206e881 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc" status: 200 len: 1299 microversion: 2.1 time: 0.090828 Aug 28 01:23:23.960985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1273/2767] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:23 2022] GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc => generated 1299 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:24.539394 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-02319913-d8c6-41da-b866-687e8234602a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:24.540000 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-02319913-d8c6-41da-b866-687e8234602a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:24.547231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-02319913-d8c6-41da-b866-687e8234602a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:24.547712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-02319913-d8c6-41da-b866-687e8234602a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:24.548142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-02319913-d8c6-41da-b866-687e8234602a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:24.548828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-02319913-d8c6-41da-b866-687e8234602a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:24.549295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-02319913-d8c6-41da-b866-687e8234602a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:24.549715 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-02319913-d8c6-41da-b866-687e8234602a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:24.624443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-baf77841-a683-4316-9820-47628743695c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:24.625498 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-baf77841-a683-4316-9820-47628743695c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:24.633279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-baf77841-a683-4316-9820-47628743695c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:24.633928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-baf77841-a683-4316-9820-47628743695c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:24.634702 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-baf77841-a683-4316-9820-47628743695c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:24.635534 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-baf77841-a683-4316-9820-47628743695c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:24.636213 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-baf77841-a683-4316-9820-47628743695c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:24.636875 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-baf77841-a683-4316-9820-47628743695c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:24.819776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-02319913-d8c6-41da-b866-687e8234602a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:24.828493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-02319913-d8c6-41da-b866-687e8234602a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.292940 Aug 28 01:23:24.829092 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1495/2768] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:24 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:24.833593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a289ca1e-4269-449d-b2e9-100bf6ec44a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:24.834217 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a289ca1e-4269-449d-b2e9-100bf6ec44a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:24.835955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:23:24.840785 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a289ca1e-4269-449d-b2e9-100bf6ec44a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:24.841376 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a289ca1e-4269-449d-b2e9-100bf6ec44a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:24.841812 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a289ca1e-4269-449d-b2e9-100bf6ec44a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:24.842501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a289ca1e-4269-449d-b2e9-100bf6ec44a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:24.843015 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a289ca1e-4269-449d-b2e9-100bf6ec44a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:24.843330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a289ca1e-4269-449d-b2e9-100bf6ec44a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:24.986583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a289ca1e-4269-449d-b2e9-100bf6ec44a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:24.995324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a289ca1e-4269-449d-b2e9-100bf6ec44a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:24.996752 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a289ca1e-4269-449d-b2e9-100bf6ec44a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:24.997898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a289ca1e-4269-449d-b2e9-100bf6ec44a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce" status: 200 len: 1340 microversion: 2.20 time: 0.167881 Aug 28 01:23:24.998434 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1496/2769] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:24 2022] GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce => generated 1340 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:25.002897 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2f9006f9-f28d-4aca-87af-d8a091ff2361 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:25.003526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2f9006f9-f28d-4aca-87af-d8a091ff2361 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:25.010075 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f9006f9-f28d-4aca-87af-d8a091ff2361 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:25.010565 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f9006f9-f28d-4aca-87af-d8a091ff2361 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:25.010977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f9006f9-f28d-4aca-87af-d8a091ff2361 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:25.012877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f9006f9-f28d-4aca-87af-d8a091ff2361 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:25.013367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f9006f9-f28d-4aca-87af-d8a091ff2361 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:25.013797 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2f9006f9-f28d-4aca-87af-d8a091ff2361 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:25.049917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-baf77841-a683-4316-9820-47628743695c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:25.058435 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-baf77841-a683-4316-9820-47628743695c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.439201 Aug 28 01:23:25.059993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1274/2770] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:24 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:23:25.321048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2f9006f9-f28d-4aca-87af-d8a091ff2361 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:25.328956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2f9006f9-f28d-4aca-87af-d8a091ff2361 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc" status: 200 len: 1366 microversion: 2.1 time: 0.329481 Aug 28 01:23:25.329930 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1497/2771] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:24 2022] GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc => generated 1366 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:25.718501 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-0a71b1ca-2d08-4696-993e-a01165cdcc88 req-a335b37b-dd02-4951-a0e7-083534f88ac3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2d365998-141e-4842-a129-31469afdb5cc", "tag": "eef83555-5015-45c5-95ba-dac334fe4757"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:23:25.737829 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-0a71b1ca-2d08-4696-993e-a01165cdcc88 req-a335b37b-dd02-4951-a0e7-083534f88ac3 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:25.741588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-0a71b1ca-2d08-4696-993e-a01165cdcc88 req-a335b37b-dd02-4951-a0e7-083534f88ac3 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:25.742018 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-0a71b1ca-2d08-4696-993e-a01165cdcc88 req-a335b37b-dd02-4951-a0e7-083534f88ac3 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:25.755090 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-0a71b1ca-2d08-4696-993e-a01165cdcc88 req-a335b37b-dd02-4951-a0e7-083534f88ac3 service nova] Creating event network-changed:eef83555-5015-45c5-95ba-dac334fe4757 for instance 2d365998-141e-4842-a129-31469afdb5cc on hv-cinder-maste Aug 28 01:23:25.760376 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-0a71b1ca-2d08-4696-993e-a01165cdcc88 req-a335b37b-dd02-4951-a0e7-083534f88ac3 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045485 Aug 28 01:23:25.760943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1275/2772] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:23:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:23:25.852872 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3d627126-91dc-4b1c-a06d-838cdeea57af tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:25.853593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3d627126-91dc-4b1c-a06d-838cdeea57af tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:25.860950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d627126-91dc-4b1c-a06d-838cdeea57af tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:25.861880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d627126-91dc-4b1c-a06d-838cdeea57af tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:25.862529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d627126-91dc-4b1c-a06d-838cdeea57af tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:25.863331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d627126-91dc-4b1c-a06d-838cdeea57af tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:25.863993 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d627126-91dc-4b1c-a06d-838cdeea57af tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:25.864513 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d627126-91dc-4b1c-a06d-838cdeea57af tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:26.004148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3d627126-91dc-4b1c-a06d-838cdeea57af tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:26.021590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3d627126-91dc-4b1c-a06d-838cdeea57af tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.174422 Aug 28 01:23:26.022916 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bfbcb96f-bac7-4af7-b1f0-e016ebfe3a73 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:26.023818 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bfbcb96f-bac7-4af7-b1f0-e016ebfe3a73 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:26.029300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1498/2773] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:25 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:26.036769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bfbcb96f-bac7-4af7-b1f0-e016ebfe3a73 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:26.037597 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bfbcb96f-bac7-4af7-b1f0-e016ebfe3a73 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:26.038211 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bfbcb96f-bac7-4af7-b1f0-e016ebfe3a73 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:26.038927 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bfbcb96f-bac7-4af7-b1f0-e016ebfe3a73 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:26.039552 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bfbcb96f-bac7-4af7-b1f0-e016ebfe3a73 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:26.040177 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bfbcb96f-bac7-4af7-b1f0-e016ebfe3a73 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:26.080771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-09477d66-c50f-4ca8-a20d-98cb327f3dfa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:26.081523 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-09477d66-c50f-4ca8-a20d-98cb327f3dfa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:26.087685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09477d66-c50f-4ca8-a20d-98cb327f3dfa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:26.088267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09477d66-c50f-4ca8-a20d-98cb327f3dfa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:26.088775 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09477d66-c50f-4ca8-a20d-98cb327f3dfa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:26.089591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09477d66-c50f-4ca8-a20d-98cb327f3dfa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:26.090110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09477d66-c50f-4ca8-a20d-98cb327f3dfa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:26.090602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09477d66-c50f-4ca8-a20d-98cb327f3dfa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:26.463226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bfbcb96f-bac7-4af7-b1f0-e016ebfe3a73 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:26.470487 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-09477d66-c50f-4ca8-a20d-98cb327f3dfa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:26.475412 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bfbcb96f-bac7-4af7-b1f0-e016ebfe3a73 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:26.477911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bfbcb96f-bac7-4af7-b1f0-e016ebfe3a73 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:26.479087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-09477d66-c50f-4ca8-a20d-98cb327f3dfa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.402112 Aug 28 01:23:26.479682 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bfbcb96f-bac7-4af7-b1f0-e016ebfe3a73 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce" status: 200 len: 1408 microversion: 2.20 time: 0.461838 Aug 28 01:23:26.479885 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1499/2774] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:26 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:23:26.481147 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1276/2775] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:26 2022] GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce => generated 1408 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 01:23:26.499486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1dd9699c-0d13-4b15-821c-c1b2d77e96c2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:26.499486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1dd9699c-0d13-4b15-821c-c1b2d77e96c2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:26.501629 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1dd9699c-0d13-4b15-821c-c1b2d77e96c2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:26.502130 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1dd9699c-0d13-4b15-821c-c1b2d77e96c2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:26.502558 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1dd9699c-0d13-4b15-821c-c1b2d77e96c2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:26.503263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1dd9699c-0d13-4b15-821c-c1b2d77e96c2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:26.503709 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1dd9699c-0d13-4b15-821c-c1b2d77e96c2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:26.504156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1dd9699c-0d13-4b15-821c-c1b2d77e96c2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:26.816099 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1dd9699c-0d13-4b15-821c-c1b2d77e96c2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:26.824342 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1dd9699c-0d13-4b15-821c-c1b2d77e96c2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc" status: 200 len: 1366 microversion: 2.1 time: 0.343594 Aug 28 01:23:26.825409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1500/2776] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:26 2022] GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc => generated 1366 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:27.044210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1590e0d8-3934-4a9a-90ea-4ca890b2d65e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:27.044928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1590e0d8-3934-4a9a-90ea-4ca890b2d65e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:27.052370 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1590e0d8-3934-4a9a-90ea-4ca890b2d65e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:27.053009 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1590e0d8-3934-4a9a-90ea-4ca890b2d65e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:27.053512 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1590e0d8-3934-4a9a-90ea-4ca890b2d65e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:27.054280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1590e0d8-3934-4a9a-90ea-4ca890b2d65e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:27.054812 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1590e0d8-3934-4a9a-90ea-4ca890b2d65e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:27.055295 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1590e0d8-3934-4a9a-90ea-4ca890b2d65e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:27.130670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1590e0d8-3934-4a9a-90ea-4ca890b2d65e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:27.139322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1590e0d8-3934-4a9a-90ea-4ca890b2d65e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.099238 Aug 28 01:23:27.139913 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1277/2777] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:27 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:23:27.501153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-aff0f0e6-99f7-43cb-94b1-a213606c2f79 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:27.501862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-aff0f0e6-99f7-43cb-94b1-a213606c2f79 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:27.506858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-96e7e8df-0a0d-4f2f-9859-56fd9a407d91 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:27.507466 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-96e7e8df-0a0d-4f2f-9859-56fd9a407d91 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:27.509343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aff0f0e6-99f7-43cb-94b1-a213606c2f79 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:27.509834 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aff0f0e6-99f7-43cb-94b1-a213606c2f79 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:27.510280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aff0f0e6-99f7-43cb-94b1-a213606c2f79 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:27.510989 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aff0f0e6-99f7-43cb-94b1-a213606c2f79 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:27.511434 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aff0f0e6-99f7-43cb-94b1-a213606c2f79 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:27.511852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-aff0f0e6-99f7-43cb-94b1-a213606c2f79 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:27.514288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96e7e8df-0a0d-4f2f-9859-56fd9a407d91 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:27.514774 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96e7e8df-0a0d-4f2f-9859-56fd9a407d91 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:27.515205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96e7e8df-0a0d-4f2f-9859-56fd9a407d91 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:27.515911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96e7e8df-0a0d-4f2f-9859-56fd9a407d91 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:27.516356 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96e7e8df-0a0d-4f2f-9859-56fd9a407d91 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:27.516777 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-96e7e8df-0a0d-4f2f-9859-56fd9a407d91 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:27.712930 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-aff0f0e6-99f7-43cb-94b1-a213606c2f79 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:27.717564 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-96e7e8df-0a0d-4f2f-9859-56fd9a407d91 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:27.724831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-aff0f0e6-99f7-43cb-94b1-a213606c2f79 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.227549 Aug 28 01:23:27.725561 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1501/2778] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:27 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:27.728327 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-96e7e8df-0a0d-4f2f-9859-56fd9a407d91 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:27.729845 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-96e7e8df-0a0d-4f2f-9859-56fd9a407d91 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:27.730933 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-96e7e8df-0a0d-4f2f-9859-56fd9a407d91 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce" status: 200 len: 1408 microversion: 2.20 time: 0.228240 Aug 28 01:23:27.731457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1278/2779] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:27 2022] GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce => generated 1408 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 01:23:27.786334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-7644ef8e-eba2-4429-beb0-488fcf373603 req-21565e90-f070-4253-97a5-ccecf588dfcc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "81d7ea67-14fe-4038-8519-dd1f56d18cce", "tag": "84a74b5e-4dfc-48a0-ba25-a1e19ee5ff83"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:23:27.797204 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-7644ef8e-eba2-4429-beb0-488fcf373603 req-21565e90-f070-4253-97a5-ccecf588dfcc service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:27.797859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-7644ef8e-eba2-4429-beb0-488fcf373603 req-21565e90-f070-4253-97a5-ccecf588dfcc service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:27.798420 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-7644ef8e-eba2-4429-beb0-488fcf373603 req-21565e90-f070-4253-97a5-ccecf588dfcc service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:27.814990 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-7644ef8e-eba2-4429-beb0-488fcf373603 req-21565e90-f070-4253-97a5-ccecf588dfcc service nova] Creating event network-changed:84a74b5e-4dfc-48a0-ba25-a1e19ee5ff83 for instance 81d7ea67-14fe-4038-8519-dd1f56d18cce on hv-cinder-maste Aug 28 01:23:27.821640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-7644ef8e-eba2-4429-beb0-488fcf373603 req-21565e90-f070-4253-97a5-ccecf588dfcc service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040913 Aug 28 01:23:27.822215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1502/2780] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:23:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:23:27.849042 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-068fa02a-f032-406a-ae5f-065612dfa495 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:27.849793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-068fa02a-f032-406a-ae5f-065612dfa495 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:27.856565 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-068fa02a-f032-406a-ae5f-065612dfa495 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:27.857062 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-068fa02a-f032-406a-ae5f-065612dfa495 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:27.857535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-068fa02a-f032-406a-ae5f-065612dfa495 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:27.858241 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-068fa02a-f032-406a-ae5f-065612dfa495 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:27.858692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-068fa02a-f032-406a-ae5f-065612dfa495 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:27.859107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-068fa02a-f032-406a-ae5f-065612dfa495 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:27.988425 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-068fa02a-f032-406a-ae5f-065612dfa495 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:27.996277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-068fa02a-f032-406a-ae5f-065612dfa495 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc" status: 200 len: 1366 microversion: 2.1 time: 0.152830 Aug 28 01:23:27.996830 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1279/2781] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:27 2022] GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc => generated 1366 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:28.161170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cbaac538-cb53-4e30-a136-4d4f40dd6559 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:28.162016 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cbaac538-cb53-4e30-a136-4d4f40dd6559 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:28.169548 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbaac538-cb53-4e30-a136-4d4f40dd6559 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:28.170020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbaac538-cb53-4e30-a136-4d4f40dd6559 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:28.170459 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbaac538-cb53-4e30-a136-4d4f40dd6559 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:28.171399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbaac538-cb53-4e30-a136-4d4f40dd6559 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:28.172018 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbaac538-cb53-4e30-a136-4d4f40dd6559 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:28.172461 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbaac538-cb53-4e30-a136-4d4f40dd6559 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:28.239115 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-cbaac538-cb53-4e30-a136-4d4f40dd6559 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:28.247724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cbaac538-cb53-4e30-a136-4d4f40dd6559 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.091151 Aug 28 01:23:28.248281 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1503/2782] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:28 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:28.749936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6f56d441-660b-45fa-b0db-a42a26355da2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:28.750620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6f56d441-660b-45fa-b0db-a42a26355da2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:28.753569 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5bc51813-16f1-42e6-857b-ced60d62a348 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:28.754296 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5bc51813-16f1-42e6-857b-ced60d62a348 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:28.758346 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f56d441-660b-45fa-b0db-a42a26355da2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:28.758819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f56d441-660b-45fa-b0db-a42a26355da2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:28.759231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f56d441-660b-45fa-b0db-a42a26355da2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:28.759924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f56d441-660b-45fa-b0db-a42a26355da2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:28.760365 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f56d441-660b-45fa-b0db-a42a26355da2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:28.760776 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6f56d441-660b-45fa-b0db-a42a26355da2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:28.760935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5bc51813-16f1-42e6-857b-ced60d62a348 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:28.761325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5bc51813-16f1-42e6-857b-ced60d62a348 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:28.761761 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5bc51813-16f1-42e6-857b-ced60d62a348 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:28.762456 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5bc51813-16f1-42e6-857b-ced60d62a348 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:28.762906 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5bc51813-16f1-42e6-857b-ced60d62a348 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:28.763323 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5bc51813-16f1-42e6-857b-ced60d62a348 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:28.957428 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6f56d441-660b-45fa-b0db-a42a26355da2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:28.967309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5bc51813-16f1-42e6-857b-ced60d62a348 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:28.967971 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6f56d441-660b-45fa-b0db-a42a26355da2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.221585 Aug 28 01:23:28.968896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1280/2783] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:28 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:28.979115 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5bc51813-16f1-42e6-857b-ced60d62a348 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:28.980631 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5bc51813-16f1-42e6-857b-ced60d62a348 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:28.981817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5bc51813-16f1-42e6-857b-ced60d62a348 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce" status: 200 len: 1408 microversion: 2.20 time: 0.232273 Aug 28 01:23:28.982359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1504/2784] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:28 2022] GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce => generated 1408 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 01:23:29.017223 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f9e67a18-4f4b-4a02-b2c8-ee662dabb9c6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:29.017744 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f9e67a18-4f4b-4a02-b2c8-ee662dabb9c6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:29.024584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9e67a18-4f4b-4a02-b2c8-ee662dabb9c6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:29.025107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9e67a18-4f4b-4a02-b2c8-ee662dabb9c6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:29.025536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9e67a18-4f4b-4a02-b2c8-ee662dabb9c6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:29.026248 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9e67a18-4f4b-4a02-b2c8-ee662dabb9c6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:29.026668 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9e67a18-4f4b-4a02-b2c8-ee662dabb9c6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:29.027211 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9e67a18-4f4b-4a02-b2c8-ee662dabb9c6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:29.179664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f9e67a18-4f4b-4a02-b2c8-ee662dabb9c6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:29.188933 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f9e67a18-4f4b-4a02-b2c8-ee662dabb9c6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc" status: 200 len: 1366 microversion: 2.1 time: 0.175887 Aug 28 01:23:29.189553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1281/2785] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:29 2022] GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc => generated 1366 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:29.270389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b46f5a34-ed84-47b5-acf3-65d9a90bfe92 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:29.271103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b46f5a34-ed84-47b5-acf3-65d9a90bfe92 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:29.279495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b46f5a34-ed84-47b5-acf3-65d9a90bfe92 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:29.280653 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b46f5a34-ed84-47b5-acf3-65d9a90bfe92 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:29.281342 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b46f5a34-ed84-47b5-acf3-65d9a90bfe92 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:29.282274 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b46f5a34-ed84-47b5-acf3-65d9a90bfe92 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:29.282979 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b46f5a34-ed84-47b5-acf3-65d9a90bfe92 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:29.283450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b46f5a34-ed84-47b5-acf3-65d9a90bfe92 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:29.432251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b46f5a34-ed84-47b5-acf3-65d9a90bfe92 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:29.448346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b46f5a34-ed84-47b5-acf3-65d9a90bfe92 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.182425 Aug 28 01:23:29.449425 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1505/2786] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:29 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:29.995324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b5ad6387-3266-4b44-93f8-1c9802b9eec4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:29.996459 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b5ad6387-3266-4b44-93f8-1c9802b9eec4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:30.004767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-62674d50-4782-4c32-a7d6-b6e23f5586f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:30.006154 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-62674d50-4782-4c32-a7d6-b6e23f5586f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:30.012622 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5ad6387-3266-4b44-93f8-1c9802b9eec4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:30.013377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5ad6387-3266-4b44-93f8-1c9802b9eec4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:30.014374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5ad6387-3266-4b44-93f8-1c9802b9eec4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:30.015312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62674d50-4782-4c32-a7d6-b6e23f5586f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:30.015870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62674d50-4782-4c32-a7d6-b6e23f5586f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:30.016418 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5ad6387-3266-4b44-93f8-1c9802b9eec4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:30.016609 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62674d50-4782-4c32-a7d6-b6e23f5586f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:30.017676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5ad6387-3266-4b44-93f8-1c9802b9eec4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:30.017973 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62674d50-4782-4c32-a7d6-b6e23f5586f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:30.018769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62674d50-4782-4c32-a7d6-b6e23f5586f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:30.019263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-62674d50-4782-4c32-a7d6-b6e23f5586f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:30.019593 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b5ad6387-3266-4b44-93f8-1c9802b9eec4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:30.080575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:23:30.439783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-62674d50-4782-4c32-a7d6-b6e23f5586f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:30.449786 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-62674d50-4782-4c32-a7d6-b6e23f5586f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:30.451419 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-62674d50-4782-4c32-a7d6-b6e23f5586f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:30.452667 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-62674d50-4782-4c32-a7d6-b6e23f5586f2 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce" status: 200 len: 1408 microversion: 2.20 time: 0.452415 Aug 28 01:23:30.461708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1506/2787] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:29 2022] GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce => generated 1408 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Aug 28 01:23:30.461990 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b5ad6387-3266-4b44-93f8-1c9802b9eec4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:30.462116 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-70594e2e-bc9a-4f64-b5d8-e294a0e9ed5d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:30.462116 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-70594e2e-bc9a-4f64-b5d8-e294a0e9ed5d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:30.470283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70594e2e-bc9a-4f64-b5d8-e294a0e9ed5d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:30.471312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70594e2e-bc9a-4f64-b5d8-e294a0e9ed5d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:30.471833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70594e2e-bc9a-4f64-b5d8-e294a0e9ed5d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:30.472696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70594e2e-bc9a-4f64-b5d8-e294a0e9ed5d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:30.473681 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70594e2e-bc9a-4f64-b5d8-e294a0e9ed5d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:30.473975 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b5ad6387-3266-4b44-93f8-1c9802b9eec4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.483424 Aug 28 01:23:30.474252 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-70594e2e-bc9a-4f64-b5d8-e294a0e9ed5d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:30.475012 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1282/2788] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:29 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:30.485076 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cc77aacc-5a1b-4645-bf34-9006b8b453c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:30.486016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cc77aacc-5a1b-4645-bf34-9006b8b453c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:30.505655 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc77aacc-5a1b-4645-bf34-9006b8b453c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:30.507084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc77aacc-5a1b-4645-bf34-9006b8b453c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:30.507714 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc77aacc-5a1b-4645-bf34-9006b8b453c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:30.508770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc77aacc-5a1b-4645-bf34-9006b8b453c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:30.509414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc77aacc-5a1b-4645-bf34-9006b8b453c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:30.510011 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cc77aacc-5a1b-4645-bf34-9006b8b453c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:30.661157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-cc77aacc-5a1b-4645-bf34-9006b8b453c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:30.679640 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cc77aacc-5a1b-4645-bf34-9006b8b453c4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.200025 Aug 28 01:23:30.693490 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1283/2789] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:30 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:30.746007 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-70594e2e-bc9a-4f64-b5d8-e294a0e9ed5d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:30.754036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-70594e2e-bc9a-4f64-b5d8-e294a0e9ed5d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc" status: 200 len: 1366 microversion: 2.1 time: 0.299661 Aug 28 01:23:30.754613 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1507/2790] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:30 2022] GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc => generated 1366 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:31.476244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-56cb08f8-8ece-417b-8a84-4b4561f50835 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:31.476910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-56cb08f8-8ece-417b-8a84-4b4561f50835 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:31.483759 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-56cb08f8-8ece-417b-8a84-4b4561f50835 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:31.484221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-56cb08f8-8ece-417b-8a84-4b4561f50835 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:31.484666 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-56cb08f8-8ece-417b-8a84-4b4561f50835 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:31.485391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-56cb08f8-8ece-417b-8a84-4b4561f50835 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:31.485838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-56cb08f8-8ece-417b-8a84-4b4561f50835 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:31.486254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-56cb08f8-8ece-417b-8a84-4b4561f50835 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:31.501454 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-38e83869-6b64-4c30-ab49-903710e6b7ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:31.502110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-38e83869-6b64-4c30-ab49-903710e6b7ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:31.508808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38e83869-6b64-4c30-ab49-903710e6b7ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:31.509302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38e83869-6b64-4c30-ab49-903710e6b7ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:31.509759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38e83869-6b64-4c30-ab49-903710e6b7ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:31.510466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38e83869-6b64-4c30-ab49-903710e6b7ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:31.510919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38e83869-6b64-4c30-ab49-903710e6b7ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:31.511346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38e83869-6b64-4c30-ab49-903710e6b7ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:31.546842 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=2 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:23:31.691541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-56cb08f8-8ece-417b-8a84-4b4561f50835 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:31.699990 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-38e83869-6b64-4c30-ab49-903710e6b7ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:31.703319 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-56cb08f8-8ece-417b-8a84-4b4561f50835 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:31.704739 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-56cb08f8-8ece-417b-8a84-4b4561f50835 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:31.705982 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-56cb08f8-8ece-417b-8a84-4b4561f50835 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce" status: 200 len: 1408 microversion: 2.20 time: 0.233832 Aug 28 01:23:31.706846 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1284/2791] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:31 2022] GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce => generated 1408 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:31.711464 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0fff6008-a4db-4fbc-af6e-90b72ae6b455 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:31.712069 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0fff6008-a4db-4fbc-af6e-90b72ae6b455 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:31.718927 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0fff6008-a4db-4fbc-af6e-90b72ae6b455 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:31.719411 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0fff6008-a4db-4fbc-af6e-90b72ae6b455 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:31.719836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0fff6008-a4db-4fbc-af6e-90b72ae6b455 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:31.720535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0fff6008-a4db-4fbc-af6e-90b72ae6b455 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:31.720996 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0fff6008-a4db-4fbc-af6e-90b72ae6b455 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:31.721429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0fff6008-a4db-4fbc-af6e-90b72ae6b455 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:31.723953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-38e83869-6b64-4c30-ab49-903710e6b7ca tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.226300 Aug 28 01:23:31.725085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1508/2792] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:31 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:23:31.775262 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-10f02f4d-6113-47e8-89b6-99da3ddc760d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:31.775866 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-10f02f4d-6113-47e8-89b6-99da3ddc760d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:31.777785 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=8 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:23:31.783263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10f02f4d-6113-47e8-89b6-99da3ddc760d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:31.783745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10f02f4d-6113-47e8-89b6-99da3ddc760d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:31.784167 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10f02f4d-6113-47e8-89b6-99da3ddc760d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:31.784954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10f02f4d-6113-47e8-89b6-99da3ddc760d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:31.785355 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10f02f4d-6113-47e8-89b6-99da3ddc760d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:31.785732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-10f02f4d-6113-47e8-89b6-99da3ddc760d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:31.788162 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0fff6008-a4db-4fbc-af6e-90b72ae6b455 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:31.796822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0fff6008-a4db-4fbc-af6e-90b72ae6b455 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089135 Aug 28 01:23:31.797621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1285/2793] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:31 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:31.929749 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-10f02f4d-6113-47e8-89b6-99da3ddc760d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:31.932008 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:23:31.938932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-10f02f4d-6113-47e8-89b6-99da3ddc760d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc" status: 200 len: 1366 microversion: 2.1 time: 0.167638 Aug 28 01:23:31.939517 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1509/2794] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:31 2022] GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc => generated 1366 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:23:32.048540 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-f6a85f8f-7a74-417c-995f-e2c35754ce19 req-e952d86e-40bf-4c0a-9a0a-6416c9ccb76b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2d365998-141e-4842-a129-31469afdb5cc", "name": "network-vif-plugged", "status": "completed", "tag": "eef83555-5015-45c5-95ba-dac334fe4757"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:23:32.059042 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-f6a85f8f-7a74-417c-995f-e2c35754ce19 req-e952d86e-40bf-4c0a-9a0a-6416c9ccb76b service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:32.059559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-f6a85f8f-7a74-417c-995f-e2c35754ce19 req-e952d86e-40bf-4c0a-9a0a-6416c9ccb76b service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:32.059956 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-f6a85f8f-7a74-417c-995f-e2c35754ce19 req-e952d86e-40bf-4c0a-9a0a-6416c9ccb76b service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:32.072097 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-f6a85f8f-7a74-417c-995f-e2c35754ce19 req-e952d86e-40bf-4c0a-9a0a-6416c9ccb76b service nova] Creating event network-vif-plugged:eef83555-5015-45c5-95ba-dac334fe4757 for instance 2d365998-141e-4842-a129-31469afdb5cc on hv-cinder-maste Aug 28 01:23:32.077797 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-f6a85f8f-7a74-417c-995f-e2c35754ce19 req-e952d86e-40bf-4c0a-9a0a-6416c9ccb76b service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.095462 Aug 28 01:23:32.078377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1286/2795] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:23:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:23:32.742420 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d01a2b4a-8fe0-4077-9431-9525b8ee9c78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:32.743243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=2 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:23:32.743885 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d01a2b4a-8fe0-4077-9431-9525b8ee9c78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:32.744064 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=16 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:23:32.748700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6cf40e14-2c7d-479e-8547-5f97a1fe220f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:32.749383 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6cf40e14-2c7d-479e-8547-5f97a1fe220f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:32.751686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d01a2b4a-8fe0-4077-9431-9525b8ee9c78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:32.752181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d01a2b4a-8fe0-4077-9431-9525b8ee9c78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:32.753008 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d01a2b4a-8fe0-4077-9431-9525b8ee9c78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:32.753316 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d01a2b4a-8fe0-4077-9431-9525b8ee9c78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:32.753815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d01a2b4a-8fe0-4077-9431-9525b8ee9c78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:32.754334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d01a2b4a-8fe0-4077-9431-9525b8ee9c78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:32.757160 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6cf40e14-2c7d-479e-8547-5f97a1fe220f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:32.757654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6cf40e14-2c7d-479e-8547-5f97a1fe220f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:32.758085 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6cf40e14-2c7d-479e-8547-5f97a1fe220f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:32.758802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6cf40e14-2c7d-479e-8547-5f97a1fe220f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:32.759296 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6cf40e14-2c7d-479e-8547-5f97a1fe220f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:32.759730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6cf40e14-2c7d-479e-8547-5f97a1fe220f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:32.971536 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d01a2b4a-8fe0-4077-9431-9525b8ee9c78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:32.974856 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6cf40e14-2c7d-479e-8547-5f97a1fe220f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:32.980265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d01a2b4a-8fe0-4077-9431-9525b8ee9c78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:32.981714 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d01a2b4a-8fe0-4077-9431-9525b8ee9c78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:32.982817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d01a2b4a-8fe0-4077-9431-9525b8ee9c78 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce" status: 200 len: 1408 microversion: 2.20 time: 0.246886 Aug 28 01:23:32.983371 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1510/2796] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:32 2022] GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce => generated 1408 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:32.991242 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6cf40e14-2c7d-479e-8547-5f97a1fe220f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.242083 Aug 28 01:23:32.991242 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1287/2797] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:32 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:32.991570 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-19b347e0-4d33-49a6-bd06-9c2cc7b41e6b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:32.991570 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-19b347e0-4d33-49a6-bd06-9c2cc7b41e6b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:32.995619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-19b347e0-4d33-49a6-bd06-9c2cc7b41e6b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:32.996336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-19b347e0-4d33-49a6-bd06-9c2cc7b41e6b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:32.996948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-19b347e0-4d33-49a6-bd06-9c2cc7b41e6b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:32.997866 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-19b347e0-4d33-49a6-bd06-9c2cc7b41e6b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:32.998189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-064efef0-fcf1-4079-8410-3b099d280806 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:32.998512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-19b347e0-4d33-49a6-bd06-9c2cc7b41e6b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:32.999131 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-19b347e0-4d33-49a6-bd06-9c2cc7b41e6b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:32.999265 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-064efef0-fcf1-4079-8410-3b099d280806 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:33.005596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-064efef0-fcf1-4079-8410-3b099d280806 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:33.006084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-064efef0-fcf1-4079-8410-3b099d280806 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:33.006563 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-064efef0-fcf1-4079-8410-3b099d280806 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:33.007344 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-064efef0-fcf1-4079-8410-3b099d280806 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:33.007793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-064efef0-fcf1-4079-8410-3b099d280806 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:33.008339 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-064efef0-fcf1-4079-8410-3b099d280806 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:33.069243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-19b347e0-4d33-49a6-bd06-9c2cc7b41e6b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:33.078060 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-19b347e0-4d33-49a6-bd06-9c2cc7b41e6b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.093656 Aug 28 01:23:33.078678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1511/2798] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:32 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:33.159075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-064efef0-fcf1-4079-8410-3b099d280806 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:33.167892 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-064efef0-fcf1-4079-8410-3b099d280806 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc" status: 200 len: 1366 microversion: 2.1 time: 0.181858 Aug 28 01:23:33.168597 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1288/2799] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:32 2022] GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc => generated 1366 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:34.006343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e1ae54af-9d2d-4a07-a22b-52c71788c099 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:34.007019 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e1ae54af-9d2d-4a07-a22b-52c71788c099 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:34.009894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bbc98ab3-84a4-4e28-b32f-1d09f39f9d07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:34.011545 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bbc98ab3-84a4-4e28-b32f-1d09f39f9d07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:34.013892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1ae54af-9d2d-4a07-a22b-52c71788c099 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:34.014388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1ae54af-9d2d-4a07-a22b-52c71788c099 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:34.014826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1ae54af-9d2d-4a07-a22b-52c71788c099 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:34.015541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1ae54af-9d2d-4a07-a22b-52c71788c099 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:34.016002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1ae54af-9d2d-4a07-a22b-52c71788c099 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:34.016415 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e1ae54af-9d2d-4a07-a22b-52c71788c099 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:34.022075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bbc98ab3-84a4-4e28-b32f-1d09f39f9d07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:34.023363 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bbc98ab3-84a4-4e28-b32f-1d09f39f9d07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:34.023927 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bbc98ab3-84a4-4e28-b32f-1d09f39f9d07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:34.025292 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bbc98ab3-84a4-4e28-b32f-1d09f39f9d07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:34.025292 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bbc98ab3-84a4-4e28-b32f-1d09f39f9d07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:34.025939 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bbc98ab3-84a4-4e28-b32f-1d09f39f9d07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:34.224408 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e1ae54af-9d2d-4a07-a22b-52c71788c099 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:34.233411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e1ae54af-9d2d-4a07-a22b-52c71788c099 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.231050 Aug 28 01:23:34.238583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1512/2800] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:34 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:34.238583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f91f4363-47de-4865-a2a7-040c63f9fbcc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:34.238807 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bbc98ab3-84a4-4e28-b32f-1d09f39f9d07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:34.239059 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f91f4363-47de-4865-a2a7-040c63f9fbcc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:34.245579 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f91f4363-47de-4865-a2a7-040c63f9fbcc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:34.246065 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f91f4363-47de-4865-a2a7-040c63f9fbcc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:34.246498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f91f4363-47de-4865-a2a7-040c63f9fbcc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:34.247202 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f91f4363-47de-4865-a2a7-040c63f9fbcc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:34.247664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f91f4363-47de-4865-a2a7-040c63f9fbcc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:34.247884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bbc98ab3-84a4-4e28-b32f-1d09f39f9d07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:34.248076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f91f4363-47de-4865-a2a7-040c63f9fbcc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:34.249429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bbc98ab3-84a4-4e28-b32f-1d09f39f9d07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:34.250783 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bbc98ab3-84a4-4e28-b32f-1d09f39f9d07 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce" status: 200 len: 1408 microversion: 2.20 time: 0.247658 Aug 28 01:23:34.251308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1289/2801] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:34 2022] GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce => generated 1408 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:34.255990 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b45ddab6-3832-40ed-a30f-c6e8658c6ae0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:34.256607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b45ddab6-3832-40ed-a30f-c6e8658c6ae0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:34.263113 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b45ddab6-3832-40ed-a30f-c6e8658c6ae0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:34.263603 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b45ddab6-3832-40ed-a30f-c6e8658c6ae0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:34.264039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b45ddab6-3832-40ed-a30f-c6e8658c6ae0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:34.264943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b45ddab6-3832-40ed-a30f-c6e8658c6ae0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:34.265426 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b45ddab6-3832-40ed-a30f-c6e8658c6ae0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:34.265840 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b45ddab6-3832-40ed-a30f-c6e8658c6ae0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:34.326476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f91f4363-47de-4865-a2a7-040c63f9fbcc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:34.336013 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f91f4363-47de-4865-a2a7-040c63f9fbcc tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.100960 Aug 28 01:23:34.336614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1513/2802] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:34 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:34.419503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b45ddab6-3832-40ed-a30f-c6e8658c6ae0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:34.427781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b45ddab6-3832-40ed-a30f-c6e8658c6ae0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc" status: 200 len: 1366 microversion: 2.1 time: 0.175340 Aug 28 01:23:34.428370 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1290/2803] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:34 2022] GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc => generated 1366 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:35.255590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4b15982f-d954-4e23-a866-172911b2e7a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:35.256192 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4b15982f-d954-4e23-a866-172911b2e7a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:35.263599 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b15982f-d954-4e23-a866-172911b2e7a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:35.264100 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b15982f-d954-4e23-a866-172911b2e7a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:35.264523 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b15982f-d954-4e23-a866-172911b2e7a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:35.265256 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b15982f-d954-4e23-a866-172911b2e7a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:35.265700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b15982f-d954-4e23-a866-172911b2e7a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:35.266137 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4b15982f-d954-4e23-a866-172911b2e7a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:35.273592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c20980c1-9a0c-4bc9-aa0e-bf43f2589924 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:35.274206 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c20980c1-9a0c-4bc9-aa0e-bf43f2589924 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:35.280943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c20980c1-9a0c-4bc9-aa0e-bf43f2589924 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:35.281441 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c20980c1-9a0c-4bc9-aa0e-bf43f2589924 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:35.281888 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c20980c1-9a0c-4bc9-aa0e-bf43f2589924 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:35.282568 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c20980c1-9a0c-4bc9-aa0e-bf43f2589924 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:35.283012 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c20980c1-9a0c-4bc9-aa0e-bf43f2589924 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:35.283456 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c20980c1-9a0c-4bc9-aa0e-bf43f2589924 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:35.463387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4b15982f-d954-4e23-a866-172911b2e7a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:35.469991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c20980c1-9a0c-4bc9-aa0e-bf43f2589924 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:35.472057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4b15982f-d954-4e23-a866-172911b2e7a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.220319 Aug 28 01:23:35.472617 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1514/2804] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:35 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:35.477164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bc3390dc-8bd8-486d-99d3-06ab8fd2e9e6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:35.477763 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bc3390dc-8bd8-486d-99d3-06ab8fd2e9e6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:35.478461 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c20980c1-9a0c-4bc9-aa0e-bf43f2589924 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:35.479889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c20980c1-9a0c-4bc9-aa0e-bf43f2589924 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:35.481010 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c20980c1-9a0c-4bc9-aa0e-bf43f2589924 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce" status: 200 len: 1408 microversion: 2.20 time: 0.211425 Aug 28 01:23:35.489037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1291/2805] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:35 2022] GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce => generated 1408 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:35.489234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc3390dc-8bd8-486d-99d3-06ab8fd2e9e6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:35.489234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc3390dc-8bd8-486d-99d3-06ab8fd2e9e6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:35.489234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc3390dc-8bd8-486d-99d3-06ab8fd2e9e6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:35.489234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc3390dc-8bd8-486d-99d3-06ab8fd2e9e6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:35.489589 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc3390dc-8bd8-486d-99d3-06ab8fd2e9e6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:35.489677 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9e974a00-8c42-4ff6-a87a-374d3f48c2f7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:35.489773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc3390dc-8bd8-486d-99d3-06ab8fd2e9e6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:35.489861 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9e974a00-8c42-4ff6-a87a-374d3f48c2f7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:35.492754 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e974a00-8c42-4ff6-a87a-374d3f48c2f7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:35.493389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e974a00-8c42-4ff6-a87a-374d3f48c2f7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:35.493813 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e974a00-8c42-4ff6-a87a-374d3f48c2f7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:35.494558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e974a00-8c42-4ff6-a87a-374d3f48c2f7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:35.495003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e974a00-8c42-4ff6-a87a-374d3f48c2f7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:35.495414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e974a00-8c42-4ff6-a87a-374d3f48c2f7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:35.555022 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bc3390dc-8bd8-486d-99d3-06ab8fd2e9e6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:35.563244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bc3390dc-8bd8-486d-99d3-06ab8fd2e9e6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089632 Aug 28 01:23:35.563842 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1515/2806] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:35 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:35.645332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9e974a00-8c42-4ff6-a87a-374d3f48c2f7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:35.653575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9e974a00-8c42-4ff6-a87a-374d3f48c2f7 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc" status: 200 len: 1556 microversion: 2.1 time: 0.171080 Aug 28 01:23:35.654155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1292/2807] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:35 2022] GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc => generated 1556 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:35.673707 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-957679ec-1bde-4201-91f8-1630cc0fe5e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:35.674191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-957679ec-1bde-4201-91f8-1630cc0fe5e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:35.680627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-957679ec-1bde-4201-91f8-1630cc0fe5e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:35.681137 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-957679ec-1bde-4201-91f8-1630cc0fe5e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:35.681566 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-957679ec-1bde-4201-91f8-1630cc0fe5e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:35.682282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-957679ec-1bde-4201-91f8-1630cc0fe5e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:35.682726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-957679ec-1bde-4201-91f8-1630cc0fe5e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:35.683144 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-957679ec-1bde-4201-91f8-1630cc0fe5e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:35.766325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-957679ec-1bde-4201-91f8-1630cc0fe5e5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.096395 Aug 28 01:23:35.767180 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1516/2808] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Aug 28 01:23:35 2022] GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc/os-interface => generated 284 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:23:36.023676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-029bbb7e-90e3-4761-ba62-a14676c15932 req-39911d6c-3a35-4214-a45b-f4243297b7b4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "81d7ea67-14fe-4038-8519-dd1f56d18cce", "name": "network-vif-plugged", "status": "completed", "tag": "84a74b5e-4dfc-48a0-ba25-a1e19ee5ff83"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:23:36.035623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-029bbb7e-90e3-4761-ba62-a14676c15932 req-39911d6c-3a35-4214-a45b-f4243297b7b4 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:36.036198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-029bbb7e-90e3-4761-ba62-a14676c15932 req-39911d6c-3a35-4214-a45b-f4243297b7b4 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:36.036705 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-029bbb7e-90e3-4761-ba62-a14676c15932 req-39911d6c-3a35-4214-a45b-f4243297b7b4 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:36.050549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-029bbb7e-90e3-4761-ba62-a14676c15932 req-39911d6c-3a35-4214-a45b-f4243297b7b4 service nova] Creating event network-vif-plugged:84a74b5e-4dfc-48a0-ba25-a1e19ee5ff83 for instance 81d7ea67-14fe-4038-8519-dd1f56d18cce on hv-cinder-maste Aug 28 01:23:36.056794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-029bbb7e-90e3-4761-ba62-a14676c15932 req-39911d6c-3a35-4214-a45b-f4243297b7b4 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036523 Aug 28 01:23:36.057301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1293/2809] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:23:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:23:36.494036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-302fb0a8-3353-47a8-a9e6-249b91f6af9f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:36.494638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-302fb0a8-3353-47a8-a9e6-249b91f6af9f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:36.501825 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-302fb0a8-3353-47a8-a9e6-249b91f6af9f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:36.502329 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-302fb0a8-3353-47a8-a9e6-249b91f6af9f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:36.502757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-302fb0a8-3353-47a8-a9e6-249b91f6af9f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:36.503445 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-302fb0a8-3353-47a8-a9e6-249b91f6af9f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:36.504179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-302fb0a8-3353-47a8-a9e6-249b91f6af9f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:36.504603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-302fb0a8-3353-47a8-a9e6-249b91f6af9f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:36.505093 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2a015ee8-98c5-40c1-879e-248d085d8b7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:36.505940 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2a015ee8-98c5-40c1-879e-248d085d8b7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:36.512667 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2a015ee8-98c5-40c1-879e-248d085d8b7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:36.513176 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2a015ee8-98c5-40c1-879e-248d085d8b7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:36.513600 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2a015ee8-98c5-40c1-879e-248d085d8b7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:36.514293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2a015ee8-98c5-40c1-879e-248d085d8b7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:36.514732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2a015ee8-98c5-40c1-879e-248d085d8b7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:36.515130 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2a015ee8-98c5-40c1-879e-248d085d8b7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:36.782566 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-302fb0a8-3353-47a8-a9e6-249b91f6af9f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:36.791331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-302fb0a8-3353-47a8-a9e6-249b91f6af9f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.301492 Aug 28 01:23:36.804986 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1517/2810] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:36 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:36.804986 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d5cd4ac4-ab83-4e49-8447-9f7934d5d141 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:36.804986 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d5cd4ac4-ab83-4e49-8447-9f7934d5d141 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:36.805589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2a015ee8-98c5-40c1-879e-248d085d8b7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:36.811104 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5cd4ac4-ab83-4e49-8447-9f7934d5d141 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:36.811683 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5cd4ac4-ab83-4e49-8447-9f7934d5d141 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:36.812186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5cd4ac4-ab83-4e49-8447-9f7934d5d141 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:36.813075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2a015ee8-98c5-40c1-879e-248d085d8b7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:36.813246 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5cd4ac4-ab83-4e49-8447-9f7934d5d141 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:36.813726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5cd4ac4-ab83-4e49-8447-9f7934d5d141 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:36.814204 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5cd4ac4-ab83-4e49-8447-9f7934d5d141 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:36.814643 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2a015ee8-98c5-40c1-879e-248d085d8b7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:36.816340 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2a015ee8-98c5-40c1-879e-248d085d8b7b tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce" status: 200 len: 1408 microversion: 2.20 time: 0.316580 Aug 28 01:23:36.817272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1294/2811] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:36 2022] GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce => generated 1408 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:36.891874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d5cd4ac4-ab83-4e49-8447-9f7934d5d141 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:36.900914 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d5cd4ac4-ab83-4e49-8447-9f7934d5d141 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.107292 Aug 28 01:23:36.901755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1518/2812] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:36 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:37.239301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-ef0bd4d1-ce27-42e4-8eb6-3b771c865c95 req-1560deec-e79d-4e73-bce8-b25604d6b015 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2d365998-141e-4842-a129-31469afdb5cc", "tag": "eef83555-5015-45c5-95ba-dac334fe4757"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:23:37.249085 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-ef0bd4d1-ce27-42e4-8eb6-3b771c865c95 req-1560deec-e79d-4e73-bce8-b25604d6b015 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:37.249566 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-ef0bd4d1-ce27-42e4-8eb6-3b771c865c95 req-1560deec-e79d-4e73-bce8-b25604d6b015 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:37.250026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-ef0bd4d1-ce27-42e4-8eb6-3b771c865c95 req-1560deec-e79d-4e73-bce8-b25604d6b015 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:37.262362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-ef0bd4d1-ce27-42e4-8eb6-3b771c865c95 req-1560deec-e79d-4e73-bce8-b25604d6b015 service nova] Creating event network-changed:eef83555-5015-45c5-95ba-dac334fe4757 for instance 2d365998-141e-4842-a129-31469afdb5cc on hv-cinder-maste Aug 28 01:23:37.267467 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-ef0bd4d1-ce27-42e4-8eb6-3b771c865c95 req-1560deec-e79d-4e73-bce8-b25604d6b015 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031821 Aug 28 01:23:37.268297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1295/2813] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:23:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:23:37.814084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-babc4652-f797-4ab6-8005-5b122eb172ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:37.814760 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-babc4652-f797-4ab6-8005-5b122eb172ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:37.822166 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-babc4652-f797-4ab6-8005-5b122eb172ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:37.822655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-babc4652-f797-4ab6-8005-5b122eb172ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:37.823491 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-babc4652-f797-4ab6-8005-5b122eb172ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:37.824200 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-babc4652-f797-4ab6-8005-5b122eb172ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:37.824651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-babc4652-f797-4ab6-8005-5b122eb172ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:37.825113 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-babc4652-f797-4ab6-8005-5b122eb172ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:37.840450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8f0e23a1-23f2-4aab-9d49-d1d07770ebc9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:37.841908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8f0e23a1-23f2-4aab-9d49-d1d07770ebc9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:37.850166 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8f0e23a1-23f2-4aab-9d49-d1d07770ebc9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:37.851043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8f0e23a1-23f2-4aab-9d49-d1d07770ebc9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:37.851711 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8f0e23a1-23f2-4aab-9d49-d1d07770ebc9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:37.852613 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8f0e23a1-23f2-4aab-9d49-d1d07770ebc9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:37.853252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8f0e23a1-23f2-4aab-9d49-d1d07770ebc9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:37.853836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8f0e23a1-23f2-4aab-9d49-d1d07770ebc9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:38.057745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-babc4652-f797-4ab6-8005-5b122eb172ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:38.068145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8f0e23a1-23f2-4aab-9d49-d1d07770ebc9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:38.073499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-babc4652-f797-4ab6-8005-5b122eb172ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.263283 Aug 28 01:23:38.074095 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1519/2814] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:37 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:38.078866 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-38e8a46f-bf5b-48a9-8190-29220461b583 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:38.079470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-38e8a46f-bf5b-48a9-8190-29220461b583 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:38.086021 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38e8a46f-bf5b-48a9-8190-29220461b583 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:38.086284 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8f0e23a1-23f2-4aab-9d49-d1d07770ebc9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:38.086889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38e8a46f-bf5b-48a9-8190-29220461b583 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:38.087323 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38e8a46f-bf5b-48a9-8190-29220461b583 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:38.087722 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8f0e23a1-23f2-4aab-9d49-d1d07770ebc9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:38.088219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38e8a46f-bf5b-48a9-8190-29220461b583 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:38.088665 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38e8a46f-bf5b-48a9-8190-29220461b583 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:38.088800 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8f0e23a1-23f2-4aab-9d49-d1d07770ebc9 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce" status: 200 len: 1408 microversion: 2.20 time: 0.253192 Aug 28 01:23:38.089121 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38e8a46f-bf5b-48a9-8190-29220461b583 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:38.089678 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1296/2815] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:37 2022] GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce => generated 1408 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:38.158307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-38e8a46f-bf5b-48a9-8190-29220461b583 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:38.167174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-38e8a46f-bf5b-48a9-8190-29220461b583 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.091826 Aug 28 01:23:38.167847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1520/2816] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:38 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:39.096172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a65b7650-ba8e-4dbe-89b9-bf5de53c0018 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:39.096838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a65b7650-ba8e-4dbe-89b9-bf5de53c0018 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:39.104220 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a65b7650-ba8e-4dbe-89b9-bf5de53c0018 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:39.104699 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a65b7650-ba8e-4dbe-89b9-bf5de53c0018 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:39.105168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a65b7650-ba8e-4dbe-89b9-bf5de53c0018 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:39.105851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a65b7650-ba8e-4dbe-89b9-bf5de53c0018 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:39.106296 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a65b7650-ba8e-4dbe-89b9-bf5de53c0018 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:39.106712 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a65b7650-ba8e-4dbe-89b9-bf5de53c0018 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:39.110107 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-964f9585-0534-4cec-a077-cf430eff6028 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:39.110693 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-964f9585-0534-4cec-a077-cf430eff6028 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:39.117640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-964f9585-0534-4cec-a077-cf430eff6028 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:39.118122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-964f9585-0534-4cec-a077-cf430eff6028 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:39.118524 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-964f9585-0534-4cec-a077-cf430eff6028 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:39.119220 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-964f9585-0534-4cec-a077-cf430eff6028 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:39.119669 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-964f9585-0534-4cec-a077-cf430eff6028 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:39.120089 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-964f9585-0534-4cec-a077-cf430eff6028 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:39.317204 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a65b7650-ba8e-4dbe-89b9-bf5de53c0018 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:39.322003 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-964f9585-0534-4cec-a077-cf430eff6028 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:39.330064 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a65b7650-ba8e-4dbe-89b9-bf5de53c0018 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.237800 Aug 28 01:23:39.331068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1297/2817] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:39 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:39.335405 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-964f9585-0534-4cec-a077-cf430eff6028 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:39.336829 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-964f9585-0534-4cec-a077-cf430eff6028 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:39.337063 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5f23132b-0a95-4cb5-89d6-dd87bd994089 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:39.337612 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5f23132b-0a95-4cb5-89d6-dd87bd994089 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:39.338005 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-964f9585-0534-4cec-a077-cf430eff6028 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce" status: 200 len: 1600 microversion: 2.20 time: 0.231749 Aug 28 01:23:39.338715 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1521/2818] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:23:39 2022] GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce => generated 1600 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:23:39.343609 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5f23132b-0a95-4cb5-89d6-dd87bd994089 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:39.344205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5f23132b-0a95-4cb5-89d6-dd87bd994089 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:39.344605 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5f23132b-0a95-4cb5-89d6-dd87bd994089 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:39.345321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5f23132b-0a95-4cb5-89d6-dd87bd994089 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:39.345773 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5f23132b-0a95-4cb5-89d6-dd87bd994089 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:39.346242 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5f23132b-0a95-4cb5-89d6-dd87bd994089 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:39.359947 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c5c11a39-dafd-47d0-bab7-14ee38d01740 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:39.360436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c5c11a39-dafd-47d0-bab7-14ee38d01740 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:39.367300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5c11a39-dafd-47d0-bab7-14ee38d01740 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:39.367781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5c11a39-dafd-47d0-bab7-14ee38d01740 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:39.368208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5c11a39-dafd-47d0-bab7-14ee38d01740 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:39.368895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5c11a39-dafd-47d0-bab7-14ee38d01740 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:39.369363 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5c11a39-dafd-47d0-bab7-14ee38d01740 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:39.369777 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5c11a39-dafd-47d0-bab7-14ee38d01740 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:39.410527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5f23132b-0a95-4cb5-89d6-dd87bd994089 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:39.420179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5f23132b-0a95-4cb5-89d6-dd87bd994089 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.087929 Aug 28 01:23:39.420748 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1298/2819] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:39 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:39.462571 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c5c11a39-dafd-47d0-bab7-14ee38d01740 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.106392 Aug 28 01:23:39.463419 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1522/2820] 10.222.0.42 () {60 vars in 1316 bytes} [Sun Aug 28 01:23:39 2022] GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce/os-interface => generated 284 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:23:40.358624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2ed3bd4d-c1f6-4825-9bfe-4b28d6091dc7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:40.359226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2ed3bd4d-c1f6-4825-9bfe-4b28d6091dc7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:40.366329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ed3bd4d-c1f6-4825-9bfe-4b28d6091dc7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:40.366827 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ed3bd4d-c1f6-4825-9bfe-4b28d6091dc7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:40.367250 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ed3bd4d-c1f6-4825-9bfe-4b28d6091dc7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:40.367933 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ed3bd4d-c1f6-4825-9bfe-4b28d6091dc7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:40.368372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ed3bd4d-c1f6-4825-9bfe-4b28d6091dc7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:40.368780 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ed3bd4d-c1f6-4825-9bfe-4b28d6091dc7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:40.441456 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-42ab6db0-f40a-434f-bfce-de0b10c1c7ae tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:40.442110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-42ab6db0-f40a-434f-bfce-de0b10c1c7ae tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:40.449141 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42ab6db0-f40a-434f-bfce-de0b10c1c7ae tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:40.449615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42ab6db0-f40a-434f-bfce-de0b10c1c7ae tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:40.450046 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42ab6db0-f40a-434f-bfce-de0b10c1c7ae tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:40.450724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42ab6db0-f40a-434f-bfce-de0b10c1c7ae tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:40.451176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42ab6db0-f40a-434f-bfce-de0b10c1c7ae tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:40.451596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-42ab6db0-f40a-434f-bfce-de0b10c1c7ae tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:40.535086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-42ab6db0-f40a-434f-bfce-de0b10c1c7ae tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:40.544360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-42ab6db0-f40a-434f-bfce-de0b10c1c7ae tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.107288 Aug 28 01:23:40.556677 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2ed3bd4d-c1f6-4825-9bfe-4b28d6091dc7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:40.558520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1523/2821] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:40 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:40.563249 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-68531461-c07e-4e54-b344-f8c9ec9bed16 req-55d26466-9020-4f94-8845-b24280a89317 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "81d7ea67-14fe-4038-8519-dd1f56d18cce", "tag": "84a74b5e-4dfc-48a0-ba25-a1e19ee5ff83"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:23:40.565608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2ed3bd4d-c1f6-4825-9bfe-4b28d6091dc7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.210535 Aug 28 01:23:40.572222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1299/2822] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:40 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:40.573799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-68531461-c07e-4e54-b344-f8c9ec9bed16 req-55d26466-9020-4f94-8845-b24280a89317 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:40.574272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-68531461-c07e-4e54-b344-f8c9ec9bed16 req-55d26466-9020-4f94-8845-b24280a89317 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:40.574901 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-68531461-c07e-4e54-b344-f8c9ec9bed16 req-55d26466-9020-4f94-8845-b24280a89317 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:40.586277 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-68531461-c07e-4e54-b344-f8c9ec9bed16 req-55d26466-9020-4f94-8845-b24280a89317 service nova] Creating event network-changed:84a74b5e-4dfc-48a0-ba25-a1e19ee5ff83 for instance 81d7ea67-14fe-4038-8519-dd1f56d18cce on hv-cinder-maste Aug 28 01:23:40.591821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-68531461-c07e-4e54-b344-f8c9ec9bed16 req-55d26466-9020-4f94-8845-b24280a89317 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032037 Aug 28 01:23:40.592472 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1524/2823] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:23:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:23:41.566089 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9f2c44f6-c78f-4445-a9ce-3d532c30f699 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:41.566660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9f2c44f6-c78f-4445-a9ce-3d532c30f699 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:41.573551 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f2c44f6-c78f-4445-a9ce-3d532c30f699 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:41.574036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f2c44f6-c78f-4445-a9ce-3d532c30f699 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:41.574471 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f2c44f6-c78f-4445-a9ce-3d532c30f699 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:41.575171 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f2c44f6-c78f-4445-a9ce-3d532c30f699 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:41.575625 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f2c44f6-c78f-4445-a9ce-3d532c30f699 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:41.576049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f2c44f6-c78f-4445-a9ce-3d532c30f699 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:41.588876 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f407bac0-0920-441f-812d-98bea0ab4b3a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:41.589609 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f407bac0-0920-441f-812d-98bea0ab4b3a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:41.596685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f407bac0-0920-441f-812d-98bea0ab4b3a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:41.597306 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f407bac0-0920-441f-812d-98bea0ab4b3a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:41.597763 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f407bac0-0920-441f-812d-98bea0ab4b3a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:41.598458 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f407bac0-0920-441f-812d-98bea0ab4b3a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:41.598998 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f407bac0-0920-441f-812d-98bea0ab4b3a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:41.599417 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f407bac0-0920-441f-812d-98bea0ab4b3a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:41.651114 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9f2c44f6-c78f-4445-a9ce-3d532c30f699 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:41.654107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:23:41.662651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9f2c44f6-c78f-4445-a9ce-3d532c30f699 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.100744 Aug 28 01:23:41.663619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1300/2824] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:41 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:41.748285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f407bac0-0920-441f-812d-98bea0ab4b3a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:41.759715 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f407bac0-0920-441f-812d-98bea0ab4b3a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.175075 Aug 28 01:23:41.760746 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1525/2825] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:41 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:23:42.692816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3793903b-5554-414e-bb8a-f1b1ca855742 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:42.693435 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3793903b-5554-414e-bb8a-f1b1ca855742 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:42.700380 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3793903b-5554-414e-bb8a-f1b1ca855742 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:42.700862 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3793903b-5554-414e-bb8a-f1b1ca855742 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:42.701313 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3793903b-5554-414e-bb8a-f1b1ca855742 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:42.702002 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3793903b-5554-414e-bb8a-f1b1ca855742 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:42.702445 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3793903b-5554-414e-bb8a-f1b1ca855742 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:42.702858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3793903b-5554-414e-bb8a-f1b1ca855742 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:42.778275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3793903b-5554-414e-bb8a-f1b1ca855742 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:42.782001 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-24e61379-3c00-42d1-949a-fada9e1b0c34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:42.783098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-24e61379-3c00-42d1-949a-fada9e1b0c34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:42.791193 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3793903b-5554-414e-bb8a-f1b1ca855742 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.102531 Aug 28 01:23:42.792165 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1301/2826] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:42 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:42.794153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24e61379-3c00-42d1-949a-fada9e1b0c34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:42.795008 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24e61379-3c00-42d1-949a-fada9e1b0c34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:42.795760 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24e61379-3c00-42d1-949a-fada9e1b0c34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:42.797014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24e61379-3c00-42d1-949a-fada9e1b0c34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:42.797843 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24e61379-3c00-42d1-949a-fada9e1b0c34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:42.798627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-24e61379-3c00-42d1-949a-fada9e1b0c34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:42.946619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-24e61379-3c00-42d1-949a-fada9e1b0c34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:42.955380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-24e61379-3c00-42d1-949a-fada9e1b0c34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.178119 Aug 28 01:23:42.955959 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1526/2827] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:42 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:23:43.820269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a09e471c-345f-4ddd-982b-607b451a20d0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:43.820920 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a09e471c-345f-4ddd-982b-607b451a20d0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:43.828933 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a09e471c-345f-4ddd-982b-607b451a20d0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:43.829807 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a09e471c-345f-4ddd-982b-607b451a20d0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:43.830233 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a09e471c-345f-4ddd-982b-607b451a20d0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:43.830926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a09e471c-345f-4ddd-982b-607b451a20d0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:43.831368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a09e471c-345f-4ddd-982b-607b451a20d0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:43.831781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a09e471c-345f-4ddd-982b-607b451a20d0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:43.918528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a09e471c-345f-4ddd-982b-607b451a20d0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:43.929216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a09e471c-345f-4ddd-982b-607b451a20d0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.112769 Aug 28 01:23:43.929810 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1302/2828] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:43 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:43.978866 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e4ae8cf8-6685-458d-ac44-df08fec58e4b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:43.979733 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e4ae8cf8-6685-458d-ac44-df08fec58e4b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:43.989857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e4ae8cf8-6685-458d-ac44-df08fec58e4b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:43.990649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e4ae8cf8-6685-458d-ac44-df08fec58e4b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:43.991275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e4ae8cf8-6685-458d-ac44-df08fec58e4b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:43.992343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e4ae8cf8-6685-458d-ac44-df08fec58e4b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:43.992791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e4ae8cf8-6685-458d-ac44-df08fec58e4b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:43.993240 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e4ae8cf8-6685-458d-ac44-df08fec58e4b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:44.129410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e4ae8cf8-6685-458d-ac44-df08fec58e4b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:44.131737 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:23:44.139278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e4ae8cf8-6685-458d-ac44-df08fec58e4b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.166210 Aug 28 01:23:44.139852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1527/2829] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:43 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:44.954665 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e2c3f8d7-0244-4765-acac-35bc5467a145 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:44.955325 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e2c3f8d7-0244-4765-acac-35bc5467a145 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:44.962699 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e2c3f8d7-0244-4765-acac-35bc5467a145 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:44.963232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e2c3f8d7-0244-4765-acac-35bc5467a145 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:44.963676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e2c3f8d7-0244-4765-acac-35bc5467a145 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:44.964407 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e2c3f8d7-0244-4765-acac-35bc5467a145 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:44.964881 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e2c3f8d7-0244-4765-acac-35bc5467a145 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:44.965363 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e2c3f8d7-0244-4765-acac-35bc5467a145 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:45.033559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e2c3f8d7-0244-4765-acac-35bc5467a145 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:45.044204 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e2c3f8d7-0244-4765-acac-35bc5467a145 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.093654 Aug 28 01:23:45.044741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1303/2830] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:44 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:45.164296 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8cdb0a04-9c89-466b-9371-151cdd877825 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:45.164917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8cdb0a04-9c89-466b-9371-151cdd877825 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:45.173384 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8cdb0a04-9c89-466b-9371-151cdd877825 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:45.173996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8cdb0a04-9c89-466b-9371-151cdd877825 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:45.174576 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8cdb0a04-9c89-466b-9371-151cdd877825 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:45.175528 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8cdb0a04-9c89-466b-9371-151cdd877825 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:45.176032 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8cdb0a04-9c89-466b-9371-151cdd877825 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:45.176624 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8cdb0a04-9c89-466b-9371-151cdd877825 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:45.310416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8cdb0a04-9c89-466b-9371-151cdd877825 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:45.319671 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8cdb0a04-9c89-466b-9371-151cdd877825 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.162058 Aug 28 01:23:45.320367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1528/2831] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:45 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:46.065889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6e6a0376-25cc-48bf-9bd3-08f1ab152a3f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:46.066623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6e6a0376-25cc-48bf-9bd3-08f1ab152a3f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:46.074386 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6e6a0376-25cc-48bf-9bd3-08f1ab152a3f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:46.074766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6e6a0376-25cc-48bf-9bd3-08f1ab152a3f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:46.075237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6e6a0376-25cc-48bf-9bd3-08f1ab152a3f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:46.075967 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6e6a0376-25cc-48bf-9bd3-08f1ab152a3f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:46.076437 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6e6a0376-25cc-48bf-9bd3-08f1ab152a3f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:46.077006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6e6a0376-25cc-48bf-9bd3-08f1ab152a3f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:46.145350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6e6a0376-25cc-48bf-9bd3-08f1ab152a3f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:46.153909 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6e6a0376-25cc-48bf-9bd3-08f1ab152a3f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.092260 Aug 28 01:23:46.154453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1304/2832] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:46 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:46.342934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-534e55c0-a8b4-4585-b717-ffd230c02b4a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:46.343634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-534e55c0-a8b4-4585-b717-ffd230c02b4a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:46.350693 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-534e55c0-a8b4-4585-b717-ffd230c02b4a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:46.351242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-534e55c0-a8b4-4585-b717-ffd230c02b4a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:46.351814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-534e55c0-a8b4-4585-b717-ffd230c02b4a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:46.352597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-534e55c0-a8b4-4585-b717-ffd230c02b4a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:46.353129 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-534e55c0-a8b4-4585-b717-ffd230c02b4a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:46.353592 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-534e55c0-a8b4-4585-b717-ffd230c02b4a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:46.489645 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-534e55c0-a8b4-4585-b717-ffd230c02b4a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:46.498762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-534e55c0-a8b4-4585-b717-ffd230c02b4a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.161031 Aug 28 01:23:46.499349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1529/2833] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:46 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:47.183568 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d06e186e-2d74-4599-bf1f-b3d533d31394 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:47.184213 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d06e186e-2d74-4599-bf1f-b3d533d31394 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:47.187596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:23:47.194623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d06e186e-2d74-4599-bf1f-b3d533d31394 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:47.195362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d06e186e-2d74-4599-bf1f-b3d533d31394 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:47.195803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d06e186e-2d74-4599-bf1f-b3d533d31394 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:47.196494 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d06e186e-2d74-4599-bf1f-b3d533d31394 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:47.196942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d06e186e-2d74-4599-bf1f-b3d533d31394 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:47.197395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d06e186e-2d74-4599-bf1f-b3d533d31394 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:47.269192 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d06e186e-2d74-4599-bf1f-b3d533d31394 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:47.281004 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d06e186e-2d74-4599-bf1f-b3d533d31394 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.102929 Aug 28 01:23:47.281645 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1305/2834] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:47 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:47.524384 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-65173d61-4f7c-404a-9096-dbdb39fa01ed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:47.524993 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-65173d61-4f7c-404a-9096-dbdb39fa01ed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:47.532280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65173d61-4f7c-404a-9096-dbdb39fa01ed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:47.532757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65173d61-4f7c-404a-9096-dbdb39fa01ed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:47.533226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65173d61-4f7c-404a-9096-dbdb39fa01ed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:47.533910 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65173d61-4f7c-404a-9096-dbdb39fa01ed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:47.534353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65173d61-4f7c-404a-9096-dbdb39fa01ed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:47.534764 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-65173d61-4f7c-404a-9096-dbdb39fa01ed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:47.678714 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-65173d61-4f7c-404a-9096-dbdb39fa01ed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:47.687429 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-65173d61-4f7c-404a-9096-dbdb39fa01ed tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.168963 Aug 28 01:23:47.687998 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1530/2835] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:47 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:48.306847 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bef3143d-3e31-48f2-a3e0-3995f9be5e95 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:48.307447 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bef3143d-3e31-48f2-a3e0-3995f9be5e95 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:48.314248 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bef3143d-3e31-48f2-a3e0-3995f9be5e95 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:48.314732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bef3143d-3e31-48f2-a3e0-3995f9be5e95 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:48.315518 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bef3143d-3e31-48f2-a3e0-3995f9be5e95 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:48.316209 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bef3143d-3e31-48f2-a3e0-3995f9be5e95 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:48.316652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bef3143d-3e31-48f2-a3e0-3995f9be5e95 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:48.317111 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bef3143d-3e31-48f2-a3e0-3995f9be5e95 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:48.383226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bef3143d-3e31-48f2-a3e0-3995f9be5e95 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:48.391690 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bef3143d-3e31-48f2-a3e0-3995f9be5e95 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089773 Aug 28 01:23:48.392259 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1306/2836] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:48 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:48.709151 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bcf53ddd-b7cd-4226-bd4d-5eb781ff5576 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:48.710125 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bcf53ddd-b7cd-4226-bd4d-5eb781ff5576 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:48.717851 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bcf53ddd-b7cd-4226-bd4d-5eb781ff5576 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:48.718330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bcf53ddd-b7cd-4226-bd4d-5eb781ff5576 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:48.718767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bcf53ddd-b7cd-4226-bd4d-5eb781ff5576 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:48.719455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bcf53ddd-b7cd-4226-bd4d-5eb781ff5576 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:48.719907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bcf53ddd-b7cd-4226-bd4d-5eb781ff5576 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:48.720413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bcf53ddd-b7cd-4226-bd4d-5eb781ff5576 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:48.863795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bcf53ddd-b7cd-4226-bd4d-5eb781ff5576 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:48.876248 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bcf53ddd-b7cd-4226-bd4d-5eb781ff5576 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.171210 Aug 28 01:23:48.876999 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1531/2837] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:48 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:49.415726 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f399a2e4-4394-4a36-8b03-48882fe95468 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:49.416308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f399a2e4-4394-4a36-8b03-48882fe95468 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:49.423477 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f399a2e4-4394-4a36-8b03-48882fe95468 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:49.423955 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f399a2e4-4394-4a36-8b03-48882fe95468 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:49.424383 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f399a2e4-4394-4a36-8b03-48882fe95468 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:49.425093 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f399a2e4-4394-4a36-8b03-48882fe95468 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:49.425531 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f399a2e4-4394-4a36-8b03-48882fe95468 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:49.425943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f399a2e4-4394-4a36-8b03-48882fe95468 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:49.494769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f399a2e4-4394-4a36-8b03-48882fe95468 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:49.503360 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f399a2e4-4394-4a36-8b03-48882fe95468 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.093024 Aug 28 01:23:49.503921 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1307/2838] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:49 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:49.899921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-88c62363-adb7-4922-967f-6b0fed94374b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:49.900566 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-88c62363-adb7-4922-967f-6b0fed94374b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:49.907464 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88c62363-adb7-4922-967f-6b0fed94374b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:49.907946 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88c62363-adb7-4922-967f-6b0fed94374b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:49.908379 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88c62363-adb7-4922-967f-6b0fed94374b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:49.909101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88c62363-adb7-4922-967f-6b0fed94374b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:49.909555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88c62363-adb7-4922-967f-6b0fed94374b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:49.909978 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-88c62363-adb7-4922-967f-6b0fed94374b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:50.051510 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-88c62363-adb7-4922-967f-6b0fed94374b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:50.060215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-88c62363-adb7-4922-967f-6b0fed94374b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.165505 Aug 28 01:23:50.060684 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1532/2839] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:49 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:50.527569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1bae8f31-41b1-40dc-8e5f-d7e54ab5f5b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:50.528235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1bae8f31-41b1-40dc-8e5f-d7e54ab5f5b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:50.535005 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bae8f31-41b1-40dc-8e5f-d7e54ab5f5b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:50.535479 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bae8f31-41b1-40dc-8e5f-d7e54ab5f5b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:50.535908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bae8f31-41b1-40dc-8e5f-d7e54ab5f5b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:50.536592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bae8f31-41b1-40dc-8e5f-d7e54ab5f5b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:50.537057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bae8f31-41b1-40dc-8e5f-d7e54ab5f5b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:50.537478 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1bae8f31-41b1-40dc-8e5f-d7e54ab5f5b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:50.601174 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1bae8f31-41b1-40dc-8e5f-d7e54ab5f5b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:50.609953 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1bae8f31-41b1-40dc-8e5f-d7e54ab5f5b0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.086304 Aug 28 01:23:50.610522 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1308/2840] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:50 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:51.082704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f6954e76-8432-4186-9469-73829cf50e27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:51.083338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f6954e76-8432-4186-9469-73829cf50e27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:51.090254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6954e76-8432-4186-9469-73829cf50e27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:51.090717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6954e76-8432-4186-9469-73829cf50e27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:51.091174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6954e76-8432-4186-9469-73829cf50e27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:51.091867 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6954e76-8432-4186-9469-73829cf50e27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:51.092316 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6954e76-8432-4186-9469-73829cf50e27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:51.092713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6954e76-8432-4186-9469-73829cf50e27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:51.228132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f6954e76-8432-4186-9469-73829cf50e27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:51.240389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f6954e76-8432-4186-9469-73829cf50e27 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.161401 Aug 28 01:23:51.241215 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1533/2841] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:51 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:51.633350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-dae66c79-536f-44f3-8478-aef8145e4476 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:51.633980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-dae66c79-536f-44f3-8478-aef8145e4476 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:51.640730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dae66c79-536f-44f3-8478-aef8145e4476 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:51.641272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dae66c79-536f-44f3-8478-aef8145e4476 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:51.641705 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dae66c79-536f-44f3-8478-aef8145e4476 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:51.642397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dae66c79-536f-44f3-8478-aef8145e4476 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:51.642845 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dae66c79-536f-44f3-8478-aef8145e4476 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:51.643265 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dae66c79-536f-44f3-8478-aef8145e4476 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:51.713841 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-dae66c79-536f-44f3-8478-aef8145e4476 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:51.722538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-dae66c79-536f-44f3-8478-aef8145e4476 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.093164 Aug 28 01:23:51.723120 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1309/2842] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:51 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:52.263129 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0bc0096a-3bdc-4aad-ab36-105032c85d04 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:52.263724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0bc0096a-3bdc-4aad-ab36-105032c85d04 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:52.270539 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0bc0096a-3bdc-4aad-ab36-105032c85d04 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:52.271035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0bc0096a-3bdc-4aad-ab36-105032c85d04 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:52.271467 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0bc0096a-3bdc-4aad-ab36-105032c85d04 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:52.272160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0bc0096a-3bdc-4aad-ab36-105032c85d04 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:52.272608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0bc0096a-3bdc-4aad-ab36-105032c85d04 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:52.273063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0bc0096a-3bdc-4aad-ab36-105032c85d04 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:52.413830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0bc0096a-3bdc-4aad-ab36-105032c85d04 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:52.422329 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0bc0096a-3bdc-4aad-ab36-105032c85d04 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.163243 Aug 28 01:23:52.422929 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1534/2843] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:52 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:52.743929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4a2418d6-29cc-4319-9bae-bf512133bf63 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:52.744576 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4a2418d6-29cc-4319-9bae-bf512133bf63 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:52.750955 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a2418d6-29cc-4319-9bae-bf512133bf63 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:52.751455 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a2418d6-29cc-4319-9bae-bf512133bf63 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:52.751855 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a2418d6-29cc-4319-9bae-bf512133bf63 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:52.752570 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a2418d6-29cc-4319-9bae-bf512133bf63 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:52.753004 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a2418d6-29cc-4319-9bae-bf512133bf63 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:52.753461 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4a2418d6-29cc-4319-9bae-bf512133bf63 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:52.831016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4a2418d6-29cc-4319-9bae-bf512133bf63 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:52.839671 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4a2418d6-29cc-4319-9bae-bf512133bf63 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.099772 Aug 28 01:23:52.840216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1310/2844] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:52 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:53.444670 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f8864242-e3e3-4288-90d8-4462f0b4b2c6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:53.445311 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f8864242-e3e3-4288-90d8-4462f0b4b2c6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:53.452408 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f8864242-e3e3-4288-90d8-4462f0b4b2c6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:53.452884 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f8864242-e3e3-4288-90d8-4462f0b4b2c6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:53.453338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f8864242-e3e3-4288-90d8-4462f0b4b2c6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:53.454000 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f8864242-e3e3-4288-90d8-4462f0b4b2c6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:53.454469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f8864242-e3e3-4288-90d8-4462f0b4b2c6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:53.454869 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f8864242-e3e3-4288-90d8-4462f0b4b2c6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:53.596206 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f8864242-e3e3-4288-90d8-4462f0b4b2c6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:53.604567 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f8864242-e3e3-4288-90d8-4462f0b4b2c6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.163819 Aug 28 01:23:53.605168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1535/2845] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:53 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:53.863235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-691053f8-547a-4193-b47e-1d24911759f8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:53.864083 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-691053f8-547a-4193-b47e-1d24911759f8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:53.873437 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-691053f8-547a-4193-b47e-1d24911759f8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:53.874118 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-691053f8-547a-4193-b47e-1d24911759f8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:53.874722 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-691053f8-547a-4193-b47e-1d24911759f8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:53.875722 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-691053f8-547a-4193-b47e-1d24911759f8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:53.876320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-691053f8-547a-4193-b47e-1d24911759f8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:53.877337 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-691053f8-547a-4193-b47e-1d24911759f8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:53.942027 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-691053f8-547a-4193-b47e-1d24911759f8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:53.950771 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-691053f8-547a-4193-b47e-1d24911759f8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.093011 Aug 28 01:23:53.951350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1311/2846] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:53 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:54.626915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ac9ae9e3-c318-49d3-854d-7859c3e3a978 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:54.627544 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ac9ae9e3-c318-49d3-854d-7859c3e3a978 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:54.634248 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac9ae9e3-c318-49d3-854d-7859c3e3a978 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:54.634720 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac9ae9e3-c318-49d3-854d-7859c3e3a978 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:54.635143 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac9ae9e3-c318-49d3-854d-7859c3e3a978 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:54.635820 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac9ae9e3-c318-49d3-854d-7859c3e3a978 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:54.636302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac9ae9e3-c318-49d3-854d-7859c3e3a978 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:54.636719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac9ae9e3-c318-49d3-854d-7859c3e3a978 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:54.787156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ac9ae9e3-c318-49d3-854d-7859c3e3a978 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:54.797649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ac9ae9e3-c318-49d3-854d-7859c3e3a978 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.174648 Aug 28 01:23:54.798224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1536/2847] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:54 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:54.972152 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-62809369-cb04-44b1-8d7d-432cf9d27fc1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:54.972730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-62809369-cb04-44b1-8d7d-432cf9d27fc1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:54.979669 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-62809369-cb04-44b1-8d7d-432cf9d27fc1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:54.980213 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-62809369-cb04-44b1-8d7d-432cf9d27fc1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:54.980648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-62809369-cb04-44b1-8d7d-432cf9d27fc1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:54.981360 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-62809369-cb04-44b1-8d7d-432cf9d27fc1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:54.981804 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-62809369-cb04-44b1-8d7d-432cf9d27fc1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:54.982214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-62809369-cb04-44b1-8d7d-432cf9d27fc1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:55.057604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-62809369-cb04-44b1-8d7d-432cf9d27fc1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:55.066916 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-62809369-cb04-44b1-8d7d-432cf9d27fc1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.098801 Aug 28 01:23:55.067458 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1312/2848] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:54 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:55.819643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0c10ddce-f171-419f-b375-3e84aec8fbf5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:55.820299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0c10ddce-f171-419f-b375-3e84aec8fbf5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:55.823080 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=82 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:23:55.828559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c10ddce-f171-419f-b375-3e84aec8fbf5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:55.829064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c10ddce-f171-419f-b375-3e84aec8fbf5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:55.829495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c10ddce-f171-419f-b375-3e84aec8fbf5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:55.830194 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c10ddce-f171-419f-b375-3e84aec8fbf5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:55.830648 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c10ddce-f171-419f-b375-3e84aec8fbf5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:55.831060 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c10ddce-f171-419f-b375-3e84aec8fbf5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:55.973090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0c10ddce-f171-419f-b375-3e84aec8fbf5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:55.981999 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0c10ddce-f171-419f-b375-3e84aec8fbf5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.166250 Aug 28 01:23:55.982543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1537/2849] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:55 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:56.087438 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2dc3b6ec-83bf-4e2c-806c-9d6645a93b6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:56.088044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2dc3b6ec-83bf-4e2c-806c-9d6645a93b6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:56.095405 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2dc3b6ec-83bf-4e2c-806c-9d6645a93b6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:56.095888 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2dc3b6ec-83bf-4e2c-806c-9d6645a93b6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:56.096321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2dc3b6ec-83bf-4e2c-806c-9d6645a93b6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:56.096993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2dc3b6ec-83bf-4e2c-806c-9d6645a93b6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:56.097478 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2dc3b6ec-83bf-4e2c-806c-9d6645a93b6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:56.097903 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2dc3b6ec-83bf-4e2c-806c-9d6645a93b6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:56.166692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2dc3b6ec-83bf-4e2c-806c-9d6645a93b6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:56.175584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2dc3b6ec-83bf-4e2c-806c-9d6645a93b6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.092130 Aug 28 01:23:56.176148 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1313/2850] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:56 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:57.010562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-66cab8b7-c2e6-4616-89eb-bd144b8822b0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:57.011256 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-66cab8b7-c2e6-4616-89eb-bd144b8822b0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:57.018157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66cab8b7-c2e6-4616-89eb-bd144b8822b0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:57.018634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66cab8b7-c2e6-4616-89eb-bd144b8822b0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:57.019063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66cab8b7-c2e6-4616-89eb-bd144b8822b0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:57.019761 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66cab8b7-c2e6-4616-89eb-bd144b8822b0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:57.020170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66cab8b7-c2e6-4616-89eb-bd144b8822b0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:57.020605 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-66cab8b7-c2e6-4616-89eb-bd144b8822b0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:57.169772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-66cab8b7-c2e6-4616-89eb-bd144b8822b0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:57.181549 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-66cab8b7-c2e6-4616-89eb-bd144b8822b0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.174610 Aug 28 01:23:57.182329 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1538/2851] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:57 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:57.196857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c8b876d6-5aa8-47ff-b288-9b3c9838bfd8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:57.197517 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c8b876d6-5aa8-47ff-b288-9b3c9838bfd8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:57.204387 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8b876d6-5aa8-47ff-b288-9b3c9838bfd8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:57.204872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8b876d6-5aa8-47ff-b288-9b3c9838bfd8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:57.205377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8b876d6-5aa8-47ff-b288-9b3c9838bfd8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:57.206053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8b876d6-5aa8-47ff-b288-9b3c9838bfd8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:57.206501 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8b876d6-5aa8-47ff-b288-9b3c9838bfd8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:57.206923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c8b876d6-5aa8-47ff-b288-9b3c9838bfd8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:57.273865 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c8b876d6-5aa8-47ff-b288-9b3c9838bfd8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:57.282407 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c8b876d6-5aa8-47ff-b288-9b3c9838bfd8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089417 Aug 28 01:23:57.282983 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1314/2852] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:57 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:58.203705 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4fb3a3ab-0449-4ea3-9368-3c2e6e1109e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:58.204301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4fb3a3ab-0449-4ea3-9368-3c2e6e1109e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:58.211254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fb3a3ab-0449-4ea3-9368-3c2e6e1109e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:58.211734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fb3a3ab-0449-4ea3-9368-3c2e6e1109e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:58.212174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fb3a3ab-0449-4ea3-9368-3c2e6e1109e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:58.212864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fb3a3ab-0449-4ea3-9368-3c2e6e1109e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:58.213339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fb3a3ab-0449-4ea3-9368-3c2e6e1109e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:58.213760 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fb3a3ab-0449-4ea3-9368-3c2e6e1109e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:58.303527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-450550d4-8f1f-4608-8100-79e9177aae66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:58.304135 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-450550d4-8f1f-4608-8100-79e9177aae66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:58.311523 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-450550d4-8f1f-4608-8100-79e9177aae66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:58.312005 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-450550d4-8f1f-4608-8100-79e9177aae66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:58.312434 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-450550d4-8f1f-4608-8100-79e9177aae66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:58.313163 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-450550d4-8f1f-4608-8100-79e9177aae66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:58.313612 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-450550d4-8f1f-4608-8100-79e9177aae66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:58.314026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-450550d4-8f1f-4608-8100-79e9177aae66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:58.362255 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4fb3a3ab-0449-4ea3-9368-3c2e6e1109e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:58.371447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4fb3a3ab-0449-4ea3-9368-3c2e6e1109e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.171468 Aug 28 01:23:58.372020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1539/2853] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:58 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:58.386700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-450550d4-8f1f-4608-8100-79e9177aae66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:58.394943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-450550d4-8f1f-4608-8100-79e9177aae66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.095228 Aug 28 01:23:58.395529 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1315/2854] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:58 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:59.395478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0112c223-8f46-423a-bc8a-1737b1a06111 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:59.396237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0112c223-8f46-423a-bc8a-1737b1a06111 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:59.404265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0112c223-8f46-423a-bc8a-1737b1a06111 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:59.404753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0112c223-8f46-423a-bc8a-1737b1a06111 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:59.405230 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0112c223-8f46-423a-bc8a-1737b1a06111 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:59.405925 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0112c223-8f46-423a-bc8a-1737b1a06111 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:59.406384 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0112c223-8f46-423a-bc8a-1737b1a06111 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:59.406804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0112c223-8f46-423a-bc8a-1737b1a06111 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:59.415861 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-85c27eba-4b05-462c-b3a5-929617d74a05 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:23:59.416464 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-85c27eba-4b05-462c-b3a5-929617d74a05 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:23:59.423042 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85c27eba-4b05-462c-b3a5-929617d74a05 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:59.423519 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85c27eba-4b05-462c-b3a5-929617d74a05 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:59.423947 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85c27eba-4b05-462c-b3a5-929617d74a05 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:59.424640 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85c27eba-4b05-462c-b3a5-929617d74a05 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:23:59.425114 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85c27eba-4b05-462c-b3a5-929617d74a05 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:23:59.425638 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85c27eba-4b05-462c-b3a5-929617d74a05 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:23:59.505123 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-85c27eba-4b05-462c-b3a5-929617d74a05 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:59.515974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-85c27eba-4b05-462c-b3a5-929617d74a05 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.103613 Aug 28 01:23:59.517603 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1316/2855] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:59 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:23:59.562676 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0112c223-8f46-423a-bc8a-1737b1a06111 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:23:59.571317 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0112c223-8f46-423a-bc8a-1737b1a06111 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.180981 Aug 28 01:23:59.571939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1540/2856] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:23:59 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:00.540010 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a283eb17-16ef-4616-bedb-a3c55f5b565f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:00.540617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a283eb17-16ef-4616-bedb-a3c55f5b565f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:00.547992 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a283eb17-16ef-4616-bedb-a3c55f5b565f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:00.548466 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a283eb17-16ef-4616-bedb-a3c55f5b565f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:00.548895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a283eb17-16ef-4616-bedb-a3c55f5b565f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:00.549609 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a283eb17-16ef-4616-bedb-a3c55f5b565f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:00.550049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a283eb17-16ef-4616-bedb-a3c55f5b565f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:00.550466 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a283eb17-16ef-4616-bedb-a3c55f5b565f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:00.565430 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:24:00.591958 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-df62613d-a0a2-437c-bb8a-91c1128bbc42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:00.592587 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-df62613d-a0a2-437c-bb8a-91c1128bbc42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:00.600071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df62613d-a0a2-437c-bb8a-91c1128bbc42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:00.600558 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df62613d-a0a2-437c-bb8a-91c1128bbc42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:00.600997 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df62613d-a0a2-437c-bb8a-91c1128bbc42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:00.601728 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df62613d-a0a2-437c-bb8a-91c1128bbc42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:00.602182 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df62613d-a0a2-437c-bb8a-91c1128bbc42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:00.602611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-df62613d-a0a2-437c-bb8a-91c1128bbc42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:00.616270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a283eb17-16ef-4616-bedb-a3c55f5b565f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:00.624725 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a283eb17-16ef-4616-bedb-a3c55f5b565f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.090842 Aug 28 01:24:00.625334 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1317/2857] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:00 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:00.734650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-df62613d-a0a2-437c-bb8a-91c1128bbc42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:00.743171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-df62613d-a0a2-437c-bb8a-91c1128bbc42 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.155161 Aug 28 01:24:00.743739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1541/2858] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:00 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:01.646676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-375cc1a8-5d9b-4a5d-abea-65ee75d4d66c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:01.647263 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-375cc1a8-5d9b-4a5d-abea-65ee75d4d66c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:01.654249 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-375cc1a8-5d9b-4a5d-abea-65ee75d4d66c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:01.654734 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-375cc1a8-5d9b-4a5d-abea-65ee75d4d66c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:01.655169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-375cc1a8-5d9b-4a5d-abea-65ee75d4d66c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:01.655862 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-375cc1a8-5d9b-4a5d-abea-65ee75d4d66c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:01.656309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-375cc1a8-5d9b-4a5d-abea-65ee75d4d66c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:01.656733 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-375cc1a8-5d9b-4a5d-abea-65ee75d4d66c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:01.729030 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-375cc1a8-5d9b-4a5d-abea-65ee75d4d66c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:01.737585 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-375cc1a8-5d9b-4a5d-abea-65ee75d4d66c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.094863 Aug 28 01:24:01.738157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1318/2859] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:01 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:01.764881 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b09995da-9bee-4525-b1f9-6132b96c0e64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:01.765521 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b09995da-9bee-4525-b1f9-6132b96c0e64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:01.772309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b09995da-9bee-4525-b1f9-6132b96c0e64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:01.772787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b09995da-9bee-4525-b1f9-6132b96c0e64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:01.773249 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b09995da-9bee-4525-b1f9-6132b96c0e64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:01.773975 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b09995da-9bee-4525-b1f9-6132b96c0e64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:01.774421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b09995da-9bee-4525-b1f9-6132b96c0e64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:01.774839 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b09995da-9bee-4525-b1f9-6132b96c0e64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:01.910413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b09995da-9bee-4525-b1f9-6132b96c0e64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:01.921042 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b09995da-9bee-4525-b1f9-6132b96c0e64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.159986 Aug 28 01:24:01.921655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1542/2860] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:01 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:02.762700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bcc88d20-c342-4988-9b50-8334b6fba428 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:02.763338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bcc88d20-c342-4988-9b50-8334b6fba428 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:02.771613 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcc88d20-c342-4988-9b50-8334b6fba428 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:02.772079 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcc88d20-c342-4988-9b50-8334b6fba428 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:02.772509 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcc88d20-c342-4988-9b50-8334b6fba428 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:02.773244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcc88d20-c342-4988-9b50-8334b6fba428 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:02.773696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcc88d20-c342-4988-9b50-8334b6fba428 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:02.774124 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcc88d20-c342-4988-9b50-8334b6fba428 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:02.843397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bcc88d20-c342-4988-9b50-8334b6fba428 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:02.855546 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bcc88d20-c342-4988-9b50-8334b6fba428 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.098331 Aug 28 01:24:02.856110 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1319/2861] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:02 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:02.946982 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-acd88aa5-d933-4338-8dd1-17bc149d5ca7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:02.947698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-acd88aa5-d933-4338-8dd1-17bc149d5ca7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:02.954952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-acd88aa5-d933-4338-8dd1-17bc149d5ca7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:02.955441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-acd88aa5-d933-4338-8dd1-17bc149d5ca7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:02.955878 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-acd88aa5-d933-4338-8dd1-17bc149d5ca7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:02.956576 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-acd88aa5-d933-4338-8dd1-17bc149d5ca7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:02.957029 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-acd88aa5-d933-4338-8dd1-17bc149d5ca7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:02.957541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-acd88aa5-d933-4338-8dd1-17bc149d5ca7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:03.090383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-acd88aa5-d933-4338-8dd1-17bc149d5ca7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:03.102185 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-acd88aa5-d933-4338-8dd1-17bc149d5ca7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.161836 Aug 28 01:24:03.102995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1543/2862] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:02 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:03.878866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-352b0de7-a4a1-4e08-9d70-24771f7947b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:03.879716 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-352b0de7-a4a1-4e08-9d70-24771f7947b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:03.888820 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-352b0de7-a4a1-4e08-9d70-24771f7947b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:03.889494 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-352b0de7-a4a1-4e08-9d70-24771f7947b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:03.890061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-352b0de7-a4a1-4e08-9d70-24771f7947b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:03.890965 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-352b0de7-a4a1-4e08-9d70-24771f7947b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:03.891568 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-352b0de7-a4a1-4e08-9d70-24771f7947b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:03.892121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-352b0de7-a4a1-4e08-9d70-24771f7947b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:03.960354 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-352b0de7-a4a1-4e08-9d70-24771f7947b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:03.968818 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-352b0de7-a4a1-4e08-9d70-24771f7947b5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.095089 Aug 28 01:24:03.969403 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1320/2863] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:03 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:04.124301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0ea82af1-1d6a-4662-8cd2-3822afff9647 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:04.124941 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0ea82af1-1d6a-4662-8cd2-3822afff9647 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:04.132124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ea82af1-1d6a-4662-8cd2-3822afff9647 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:04.132540 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ea82af1-1d6a-4662-8cd2-3822afff9647 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:04.132992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ea82af1-1d6a-4662-8cd2-3822afff9647 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:04.133707 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ea82af1-1d6a-4662-8cd2-3822afff9647 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:04.134159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ea82af1-1d6a-4662-8cd2-3822afff9647 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:04.134563 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ea82af1-1d6a-4662-8cd2-3822afff9647 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:04.149750 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:24:04.282051 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0ea82af1-1d6a-4662-8cd2-3822afff9647 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:04.290776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0ea82af1-1d6a-4662-8cd2-3822afff9647 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.170670 Aug 28 01:24:04.291386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1544/2864] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:04 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:04.990508 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ee433a58-edb9-4ec0-94ac-908a3d361336 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:04.991200 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ee433a58-edb9-4ec0-94ac-908a3d361336 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:04.999143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ee433a58-edb9-4ec0-94ac-908a3d361336 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:04.999672 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ee433a58-edb9-4ec0-94ac-908a3d361336 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:05.000145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ee433a58-edb9-4ec0-94ac-908a3d361336 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:05.000900 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ee433a58-edb9-4ec0-94ac-908a3d361336 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:05.001423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ee433a58-edb9-4ec0-94ac-908a3d361336 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:05.001879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ee433a58-edb9-4ec0-94ac-908a3d361336 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:05.083724 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ee433a58-edb9-4ec0-94ac-908a3d361336 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:05.096899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ee433a58-edb9-4ec0-94ac-908a3d361336 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.110398 Aug 28 01:24:05.097821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1321/2865] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:04 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:05.313669 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-831e6dd6-9b61-4c4d-a477-b7dd1dc705c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:05.314341 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-831e6dd6-9b61-4c4d-a477-b7dd1dc705c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:05.322418 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-831e6dd6-9b61-4c4d-a477-b7dd1dc705c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:05.322973 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-831e6dd6-9b61-4c4d-a477-b7dd1dc705c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:05.323473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-831e6dd6-9b61-4c4d-a477-b7dd1dc705c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:05.324275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-831e6dd6-9b61-4c4d-a477-b7dd1dc705c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:05.324790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-831e6dd6-9b61-4c4d-a477-b7dd1dc705c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:05.325306 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-831e6dd6-9b61-4c4d-a477-b7dd1dc705c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:05.719438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-831e6dd6-9b61-4c4d-a477-b7dd1dc705c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:05.732325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-831e6dd6-9b61-4c4d-a477-b7dd1dc705c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.423186 Aug 28 01:24:05.732943 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1545/2866] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:05 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:06.121670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f856e796-9df2-45ab-8903-127b34b76fa5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:06.122321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f856e796-9df2-45ab-8903-127b34b76fa5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:06.129688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f856e796-9df2-45ab-8903-127b34b76fa5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:06.130173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f856e796-9df2-45ab-8903-127b34b76fa5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:06.130615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f856e796-9df2-45ab-8903-127b34b76fa5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:06.131330 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f856e796-9df2-45ab-8903-127b34b76fa5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:06.131785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f856e796-9df2-45ab-8903-127b34b76fa5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:06.132198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f856e796-9df2-45ab-8903-127b34b76fa5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:06.193878 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f856e796-9df2-45ab-8903-127b34b76fa5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:06.202224 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f856e796-9df2-45ab-8903-127b34b76fa5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.084437 Aug 28 01:24:06.202766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1322/2867] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:06 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:06.754254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9259f568-9052-4860-91c4-14017499b9b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:06.754912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9259f568-9052-4860-91c4-14017499b9b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:06.762637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9259f568-9052-4860-91c4-14017499b9b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:06.763118 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9259f568-9052-4860-91c4-14017499b9b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:06.763542 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9259f568-9052-4860-91c4-14017499b9b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:06.764234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9259f568-9052-4860-91c4-14017499b9b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:06.764678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9259f568-9052-4860-91c4-14017499b9b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:06.765140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9259f568-9052-4860-91c4-14017499b9b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:06.905366 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9259f568-9052-4860-91c4-14017499b9b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:06.913645 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9259f568-9052-4860-91c4-14017499b9b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.163383 Aug 28 01:24:06.914212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1546/2868] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:06 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:07.222926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b6d5087a-aacc-4707-b094-ae84bbfc8727 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:07.223526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b6d5087a-aacc-4707-b094-ae84bbfc8727 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:07.231590 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6d5087a-aacc-4707-b094-ae84bbfc8727 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:07.232077 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6d5087a-aacc-4707-b094-ae84bbfc8727 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:07.232512 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6d5087a-aacc-4707-b094-ae84bbfc8727 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:07.233252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6d5087a-aacc-4707-b094-ae84bbfc8727 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:07.233703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6d5087a-aacc-4707-b094-ae84bbfc8727 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:07.234117 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6d5087a-aacc-4707-b094-ae84bbfc8727 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:07.301482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b6d5087a-aacc-4707-b094-ae84bbfc8727 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:07.315090 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b6d5087a-aacc-4707-b094-ae84bbfc8727 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.095541 Aug 28 01:24:07.315999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1323/2869] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:07 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:07.938937 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7377db27-13d0-48e4-b304-78e123ad62d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:07.940363 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7377db27-13d0-48e4-b304-78e123ad62d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:07.950448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7377db27-13d0-48e4-b304-78e123ad62d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:07.951270 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7377db27-13d0-48e4-b304-78e123ad62d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:07.951701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7377db27-13d0-48e4-b304-78e123ad62d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:07.952423 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7377db27-13d0-48e4-b304-78e123ad62d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:07.952864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7377db27-13d0-48e4-b304-78e123ad62d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:07.953324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7377db27-13d0-48e4-b304-78e123ad62d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:08.115037 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7377db27-13d0-48e4-b304-78e123ad62d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:08.124274 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7377db27-13d0-48e4-b304-78e123ad62d2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.191908 Aug 28 01:24:08.124836 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1547/2870] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:07 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:08.337491 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4dc826f9-0f15-4ebd-8da2-b022af85a035 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:08.338106 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4dc826f9-0f15-4ebd-8da2-b022af85a035 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:08.345700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4dc826f9-0f15-4ebd-8da2-b022af85a035 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:08.346186 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4dc826f9-0f15-4ebd-8da2-b022af85a035 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:08.346616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4dc826f9-0f15-4ebd-8da2-b022af85a035 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:08.347305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4dc826f9-0f15-4ebd-8da2-b022af85a035 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:08.347752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4dc826f9-0f15-4ebd-8da2-b022af85a035 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:08.348172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4dc826f9-0f15-4ebd-8da2-b022af85a035 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:08.415699 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4dc826f9-0f15-4ebd-8da2-b022af85a035 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:08.428132 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4dc826f9-0f15-4ebd-8da2-b022af85a035 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.095310 Aug 28 01:24:08.429048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1324/2871] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:08 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:09.151481 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4a0b2ab7-6199-4d9a-ad89-b90959024742 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:09.152196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4a0b2ab7-6199-4d9a-ad89-b90959024742 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:09.159076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a0b2ab7-6199-4d9a-ad89-b90959024742 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:09.159588 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a0b2ab7-6199-4d9a-ad89-b90959024742 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:09.160044 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a0b2ab7-6199-4d9a-ad89-b90959024742 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:09.160784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a0b2ab7-6199-4d9a-ad89-b90959024742 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:09.161237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a0b2ab7-6199-4d9a-ad89-b90959024742 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:09.161658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a0b2ab7-6199-4d9a-ad89-b90959024742 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:09.294649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4a0b2ab7-6199-4d9a-ad89-b90959024742 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:09.303050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4a0b2ab7-6199-4d9a-ad89-b90959024742 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.156682 Aug 28 01:24:09.303619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1548/2872] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:09 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:09.452039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4d01dbf7-fdb3-48c8-8c03-cabd280f3070 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:09.452649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4d01dbf7-fdb3-48c8-8c03-cabd280f3070 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:09.459781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d01dbf7-fdb3-48c8-8c03-cabd280f3070 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:09.460249 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d01dbf7-fdb3-48c8-8c03-cabd280f3070 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:09.460687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d01dbf7-fdb3-48c8-8c03-cabd280f3070 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:09.461432 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d01dbf7-fdb3-48c8-8c03-cabd280f3070 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:09.461884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d01dbf7-fdb3-48c8-8c03-cabd280f3070 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:09.462307 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d01dbf7-fdb3-48c8-8c03-cabd280f3070 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:09.523289 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4d01dbf7-fdb3-48c8-8c03-cabd280f3070 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:09.532204 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4d01dbf7-fdb3-48c8-8c03-cabd280f3070 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.085726 Aug 28 01:24:09.532791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1325/2873] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:09 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:10.327985 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c54389fc-a35f-41a5-b634-0cc74c0b5ee6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:10.328608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c54389fc-a35f-41a5-b634-0cc74c0b5ee6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:10.335582 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c54389fc-a35f-41a5-b634-0cc74c0b5ee6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:10.336072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c54389fc-a35f-41a5-b634-0cc74c0b5ee6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:10.336506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c54389fc-a35f-41a5-b634-0cc74c0b5ee6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:10.337256 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c54389fc-a35f-41a5-b634-0cc74c0b5ee6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:10.337706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c54389fc-a35f-41a5-b634-0cc74c0b5ee6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:10.338135 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c54389fc-a35f-41a5-b634-0cc74c0b5ee6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:10.475345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c54389fc-a35f-41a5-b634-0cc74c0b5ee6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:10.483975 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c54389fc-a35f-41a5-b634-0cc74c0b5ee6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.161690 Aug 28 01:24:10.484551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1549/2874] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:10 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:10.556488 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7383043a-ba9a-43a4-b7ad-5e260835f731 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:10.557368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7383043a-ba9a-43a4-b7ad-5e260835f731 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:10.565603 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7383043a-ba9a-43a4-b7ad-5e260835f731 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:10.566080 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7383043a-ba9a-43a4-b7ad-5e260835f731 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:10.566511 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7383043a-ba9a-43a4-b7ad-5e260835f731 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:10.567207 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7383043a-ba9a-43a4-b7ad-5e260835f731 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:10.567649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7383043a-ba9a-43a4-b7ad-5e260835f731 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:10.568064 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7383043a-ba9a-43a4-b7ad-5e260835f731 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:10.638981 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7383043a-ba9a-43a4-b7ad-5e260835f731 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:10.647580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7383043a-ba9a-43a4-b7ad-5e260835f731 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.097082 Aug 28 01:24:10.648157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1326/2875] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:10 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:11.509460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a2a3b304-6ca2-400b-b49a-a279002d98e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:11.510088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a2a3b304-6ca2-400b-b49a-a279002d98e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:11.516696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2a3b304-6ca2-400b-b49a-a279002d98e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:11.517186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2a3b304-6ca2-400b-b49a-a279002d98e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:11.517615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2a3b304-6ca2-400b-b49a-a279002d98e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:11.518283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2a3b304-6ca2-400b-b49a-a279002d98e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:11.518731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2a3b304-6ca2-400b-b49a-a279002d98e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:11.519142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2a3b304-6ca2-400b-b49a-a279002d98e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:11.651617 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a2a3b304-6ca2-400b-b49a-a279002d98e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:11.663237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a2a3b304-6ca2-400b-b49a-a279002d98e6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.157209 Aug 28 01:24:11.663979 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1550/2876] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:11 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:11.670571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8a32c99a-e803-4ac4-bbd2-81a097c1048d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:11.671273 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8a32c99a-e803-4ac4-bbd2-81a097c1048d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:11.677987 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a32c99a-e803-4ac4-bbd2-81a097c1048d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:11.678548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a32c99a-e803-4ac4-bbd2-81a097c1048d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:11.679049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a32c99a-e803-4ac4-bbd2-81a097c1048d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:11.679752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a32c99a-e803-4ac4-bbd2-81a097c1048d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:11.680187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a32c99a-e803-4ac4-bbd2-81a097c1048d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:11.680611 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8a32c99a-e803-4ac4-bbd2-81a097c1048d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:11.742513 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8a32c99a-e803-4ac4-bbd2-81a097c1048d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:11.751441 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8a32c99a-e803-4ac4-bbd2-81a097c1048d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.086992 Aug 28 01:24:11.751994 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1327/2877] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:11 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:12.689346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e6a557d8-8317-4319-9323-15d08036afbf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:12.690298 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e6a557d8-8317-4319-9323-15d08036afbf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:12.700136 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6a557d8-8317-4319-9323-15d08036afbf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:12.700881 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6a557d8-8317-4319-9323-15d08036afbf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:12.701421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6a557d8-8317-4319-9323-15d08036afbf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:12.702110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6a557d8-8317-4319-9323-15d08036afbf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:12.702571 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6a557d8-8317-4319-9323-15d08036afbf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:12.703028 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6a557d8-8317-4319-9323-15d08036afbf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:12.773368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a5037dd1-0946-4423-826a-9bedf4719f18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:12.774045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a5037dd1-0946-4423-826a-9bedf4719f18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:12.781115 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5037dd1-0946-4423-826a-9bedf4719f18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:12.781844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5037dd1-0946-4423-826a-9bedf4719f18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:12.782235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5037dd1-0946-4423-826a-9bedf4719f18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:12.782942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5037dd1-0946-4423-826a-9bedf4719f18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:12.783480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5037dd1-0946-4423-826a-9bedf4719f18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:12.783807 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5037dd1-0946-4423-826a-9bedf4719f18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:12.850764 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a5037dd1-0946-4423-826a-9bedf4719f18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:12.859331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a5037dd1-0946-4423-826a-9bedf4719f18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089948 Aug 28 01:24:12.860137 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1328/2878] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:12 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:12.871704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e6a557d8-8317-4319-9323-15d08036afbf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:12.880048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e6a557d8-8317-4319-9323-15d08036afbf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.196636 Aug 28 01:24:12.880651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1551/2879] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:12 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:13.881865 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-243c4183-30cd-48ef-a7c7-41f1b49950ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:13.882616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-243c4183-30cd-48ef-a7c7-41f1b49950ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:13.890432 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-243c4183-30cd-48ef-a7c7-41f1b49950ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:13.890911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-243c4183-30cd-48ef-a7c7-41f1b49950ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:13.891346 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-243c4183-30cd-48ef-a7c7-41f1b49950ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:13.892040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-243c4183-30cd-48ef-a7c7-41f1b49950ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:13.892493 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-243c4183-30cd-48ef-a7c7-41f1b49950ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:13.892917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-243c4183-30cd-48ef-a7c7-41f1b49950ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:13.902602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-84a67ff1-a3ac-4e8b-95cf-c153e44585e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:13.903206 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-84a67ff1-a3ac-4e8b-95cf-c153e44585e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:13.909827 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84a67ff1-a3ac-4e8b-95cf-c153e44585e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:13.910405 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84a67ff1-a3ac-4e8b-95cf-c153e44585e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:13.910917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84a67ff1-a3ac-4e8b-95cf-c153e44585e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:13.911655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84a67ff1-a3ac-4e8b-95cf-c153e44585e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:13.912180 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84a67ff1-a3ac-4e8b-95cf-c153e44585e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:13.912588 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-84a67ff1-a3ac-4e8b-95cf-c153e44585e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:13.958274 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-243c4183-30cd-48ef-a7c7-41f1b49950ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:13.967000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-243c4183-30cd-48ef-a7c7-41f1b49950ff tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.090053 Aug 28 01:24:13.967549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1329/2880] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:13 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:14.053024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-84a67ff1-a3ac-4e8b-95cf-c153e44585e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:14.061532 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-84a67ff1-a3ac-4e8b-95cf-c153e44585e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.163411 Aug 28 01:24:14.062084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1552/2881] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:13 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:14.989003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e397f74f-8857-4f8c-b68d-51ece4c9ca9e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:14.989693 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e397f74f-8857-4f8c-b68d-51ece4c9ca9e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:14.997379 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e397f74f-8857-4f8c-b68d-51ece4c9ca9e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:14.997925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e397f74f-8857-4f8c-b68d-51ece4c9ca9e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:14.998351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e397f74f-8857-4f8c-b68d-51ece4c9ca9e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:14.999036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e397f74f-8857-4f8c-b68d-51ece4c9ca9e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:14.999480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e397f74f-8857-4f8c-b68d-51ece4c9ca9e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:14.999894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e397f74f-8857-4f8c-b68d-51ece4c9ca9e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:15.075455 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e397f74f-8857-4f8c-b68d-51ece4c9ca9e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:15.083466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-eb4afc8c-3eac-4710-85e3-c7a65cf6bafb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:15.084237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-eb4afc8c-3eac-4710-85e3-c7a65cf6bafb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:15.084533 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e397f74f-8857-4f8c-b68d-51ece4c9ca9e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.099907 Aug 28 01:24:15.085137 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1330/2882] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:14 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:15.091491 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb4afc8c-3eac-4710-85e3-c7a65cf6bafb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:15.092062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb4afc8c-3eac-4710-85e3-c7a65cf6bafb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:15.092556 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb4afc8c-3eac-4710-85e3-c7a65cf6bafb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:15.093337 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb4afc8c-3eac-4710-85e3-c7a65cf6bafb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:15.093783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb4afc8c-3eac-4710-85e3-c7a65cf6bafb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:15.094226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb4afc8c-3eac-4710-85e3-c7a65cf6bafb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:15.226585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-eb4afc8c-3eac-4710-85e3-c7a65cf6bafb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:15.239379 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-eb4afc8c-3eac-4710-85e3-c7a65cf6bafb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.159460 Aug 28 01:24:15.240135 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1553/2883] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:15 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:16.107705 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ecaab2eb-6b6d-4f3e-9290-1589e68d3fa5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:16.108770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ecaab2eb-6b6d-4f3e-9290-1589e68d3fa5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:16.116084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ecaab2eb-6b6d-4f3e-9290-1589e68d3fa5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:16.116572 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ecaab2eb-6b6d-4f3e-9290-1589e68d3fa5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:16.117013 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ecaab2eb-6b6d-4f3e-9290-1589e68d3fa5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:16.118020 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ecaab2eb-6b6d-4f3e-9290-1589e68d3fa5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:16.118485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ecaab2eb-6b6d-4f3e-9290-1589e68d3fa5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:16.118875 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ecaab2eb-6b6d-4f3e-9290-1589e68d3fa5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:16.183321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ecaab2eb-6b6d-4f3e-9290-1589e68d3fa5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:16.192154 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ecaab2eb-6b6d-4f3e-9290-1589e68d3fa5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089966 Aug 28 01:24:16.192722 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1331/2884] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:16 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:16.264250 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-67f6d134-0729-43fd-bb24-3886c8a60868 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:16.264879 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-67f6d134-0729-43fd-bb24-3886c8a60868 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:16.272029 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67f6d134-0729-43fd-bb24-3886c8a60868 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:16.272510 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67f6d134-0729-43fd-bb24-3886c8a60868 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:16.272937 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67f6d134-0729-43fd-bb24-3886c8a60868 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:16.273663 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67f6d134-0729-43fd-bb24-3886c8a60868 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:16.274107 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67f6d134-0729-43fd-bb24-3886c8a60868 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:16.274511 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67f6d134-0729-43fd-bb24-3886c8a60868 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:16.410499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-67f6d134-0729-43fd-bb24-3886c8a60868 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:16.420282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-67f6d134-0729-43fd-bb24-3886c8a60868 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.159840 Aug 28 01:24:16.420876 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1554/2885] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:16 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:17.216188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-465ec60c-e00e-4d3c-9408-05a0dd56f956 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:17.217117 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-465ec60c-e00e-4d3c-9408-05a0dd56f956 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:17.219346 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=54 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:24:17.225193 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-465ec60c-e00e-4d3c-9408-05a0dd56f956 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:17.225727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-465ec60c-e00e-4d3c-9408-05a0dd56f956 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:17.226294 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-465ec60c-e00e-4d3c-9408-05a0dd56f956 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:17.227005 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-465ec60c-e00e-4d3c-9408-05a0dd56f956 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:17.227453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-465ec60c-e00e-4d3c-9408-05a0dd56f956 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:17.227872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-465ec60c-e00e-4d3c-9408-05a0dd56f956 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:17.297317 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-465ec60c-e00e-4d3c-9408-05a0dd56f956 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:17.308060 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-465ec60c-e00e-4d3c-9408-05a0dd56f956 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.098185 Aug 28 01:24:17.308659 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1332/2886] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:17 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:17.442526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a4f31198-795a-437d-8c8b-fd47745f3abb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:17.443190 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a4f31198-795a-437d-8c8b-fd47745f3abb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:17.451360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4f31198-795a-437d-8c8b-fd47745f3abb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:17.451871 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4f31198-795a-437d-8c8b-fd47745f3abb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:17.452264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4f31198-795a-437d-8c8b-fd47745f3abb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:17.453086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4f31198-795a-437d-8c8b-fd47745f3abb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:17.453536 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4f31198-795a-437d-8c8b-fd47745f3abb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:17.453944 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a4f31198-795a-437d-8c8b-fd47745f3abb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:17.716994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a4f31198-795a-437d-8c8b-fd47745f3abb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:17.725660 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a4f31198-795a-437d-8c8b-fd47745f3abb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.287333 Aug 28 01:24:17.726271 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1555/2887] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:17 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:18.330014 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-416a53df-0e9c-4971-82a9-401e26ea205e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:18.330624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-416a53df-0e9c-4971-82a9-401e26ea205e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:18.337219 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-416a53df-0e9c-4971-82a9-401e26ea205e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:18.337712 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-416a53df-0e9c-4971-82a9-401e26ea205e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:18.338143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-416a53df-0e9c-4971-82a9-401e26ea205e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:18.338837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-416a53df-0e9c-4971-82a9-401e26ea205e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:18.339293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-416a53df-0e9c-4971-82a9-401e26ea205e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:18.339752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-416a53df-0e9c-4971-82a9-401e26ea205e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:18.410587 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-416a53df-0e9c-4971-82a9-401e26ea205e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:18.419832 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-416a53df-0e9c-4971-82a9-401e26ea205e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.093597 Aug 28 01:24:18.421620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1333/2888] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:18 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:18.749701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b2300ebd-d247-443d-a7e7-229189fb08ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:18.750466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b2300ebd-d247-443d-a7e7-229189fb08ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:18.757591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2300ebd-d247-443d-a7e7-229189fb08ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:18.758014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2300ebd-d247-443d-a7e7-229189fb08ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:18.758365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2300ebd-d247-443d-a7e7-229189fb08ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:18.759107 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2300ebd-d247-443d-a7e7-229189fb08ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:18.759550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2300ebd-d247-443d-a7e7-229189fb08ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:18.759964 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b2300ebd-d247-443d-a7e7-229189fb08ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:18.893727 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b2300ebd-d247-443d-a7e7-229189fb08ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:18.902076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b2300ebd-d247-443d-a7e7-229189fb08ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.158305 Aug 28 01:24:18.902650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1556/2889] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:18 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:19.446396 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b7e13523-0c90-480f-9e73-ffda0a1b5ad5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:19.447023 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b7e13523-0c90-480f-9e73-ffda0a1b5ad5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:19.453710 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b7e13523-0c90-480f-9e73-ffda0a1b5ad5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:19.454187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b7e13523-0c90-480f-9e73-ffda0a1b5ad5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:19.454619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b7e13523-0c90-480f-9e73-ffda0a1b5ad5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:19.455319 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b7e13523-0c90-480f-9e73-ffda0a1b5ad5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:19.455760 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b7e13523-0c90-480f-9e73-ffda0a1b5ad5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:19.456172 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b7e13523-0c90-480f-9e73-ffda0a1b5ad5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:19.458806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:24:19.518957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b7e13523-0c90-480f-9e73-ffda0a1b5ad5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:19.527686 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b7e13523-0c90-480f-9e73-ffda0a1b5ad5 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.085104 Aug 28 01:24:19.528275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1334/2890] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:19 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:19.923906 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1ff742b3-9156-401b-b40f-b42115c26b83 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:19.924507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1ff742b3-9156-401b-b40f-b42115c26b83 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:19.934635 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ff742b3-9156-401b-b40f-b42115c26b83 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:19.935411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ff742b3-9156-401b-b40f-b42115c26b83 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:19.935892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ff742b3-9156-401b-b40f-b42115c26b83 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:19.936624 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ff742b3-9156-401b-b40f-b42115c26b83 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:19.937318 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ff742b3-9156-401b-b40f-b42115c26b83 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:19.937881 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ff742b3-9156-401b-b40f-b42115c26b83 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:20.085717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1ff742b3-9156-401b-b40f-b42115c26b83 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:20.094521 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1ff742b3-9156-401b-b40f-b42115c26b83 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.174618 Aug 28 01:24:20.095098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1557/2891] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:19 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:20.549289 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-092bccb8-e593-4e1d-b62f-56ab78656dd4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:20.549942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-092bccb8-e593-4e1d-b62f-56ab78656dd4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:20.556817 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-092bccb8-e593-4e1d-b62f-56ab78656dd4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:20.557317 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-092bccb8-e593-4e1d-b62f-56ab78656dd4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:20.557767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-092bccb8-e593-4e1d-b62f-56ab78656dd4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:20.558468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-092bccb8-e593-4e1d-b62f-56ab78656dd4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:20.558921 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-092bccb8-e593-4e1d-b62f-56ab78656dd4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:20.559342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-092bccb8-e593-4e1d-b62f-56ab78656dd4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:20.624803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-092bccb8-e593-4e1d-b62f-56ab78656dd4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:20.634213 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-092bccb8-e593-4e1d-b62f-56ab78656dd4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.088858 Aug 28 01:24:20.634789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1335/2892] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:20 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:21.116128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-787e80f8-c56b-43a5-9a7d-684988b099cb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:21.116773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-787e80f8-c56b-43a5-9a7d-684988b099cb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:21.123902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-787e80f8-c56b-43a5-9a7d-684988b099cb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:21.124407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-787e80f8-c56b-43a5-9a7d-684988b099cb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:21.124837 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-787e80f8-c56b-43a5-9a7d-684988b099cb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:21.125565 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-787e80f8-c56b-43a5-9a7d-684988b099cb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:21.126014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-787e80f8-c56b-43a5-9a7d-684988b099cb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:21.126458 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-787e80f8-c56b-43a5-9a7d-684988b099cb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:21.268824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-787e80f8-c56b-43a5-9a7d-684988b099cb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:21.278157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-787e80f8-c56b-43a5-9a7d-684988b099cb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.166003 Aug 28 01:24:21.278824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1558/2893] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:21 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:21.655856 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-36c2efbc-851b-4283-9146-d2ec03790be9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:21.656522 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-36c2efbc-851b-4283-9146-d2ec03790be9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:21.663267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36c2efbc-851b-4283-9146-d2ec03790be9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:21.663743 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36c2efbc-851b-4283-9146-d2ec03790be9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:21.664181 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36c2efbc-851b-4283-9146-d2ec03790be9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:21.664878 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36c2efbc-851b-4283-9146-d2ec03790be9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:21.665361 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36c2efbc-851b-4283-9146-d2ec03790be9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:21.665770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-36c2efbc-851b-4283-9146-d2ec03790be9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:21.731162 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-36c2efbc-851b-4283-9146-d2ec03790be9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:21.739960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-36c2efbc-851b-4283-9146-d2ec03790be9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.088020 Aug 28 01:24:21.740546 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1336/2894] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:21 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:22.300328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-774769f2-f46d-443f-910c-3e896d2cca3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:22.300950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-774769f2-f46d-443f-910c-3e896d2cca3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:22.308074 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-774769f2-f46d-443f-910c-3e896d2cca3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:22.308753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-774769f2-f46d-443f-910c-3e896d2cca3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:22.308990 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-774769f2-f46d-443f-910c-3e896d2cca3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:22.309799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-774769f2-f46d-443f-910c-3e896d2cca3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:22.310216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-774769f2-f46d-443f-910c-3e896d2cca3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:22.310643 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-774769f2-f46d-443f-910c-3e896d2cca3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:22.450119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-774769f2-f46d-443f-910c-3e896d2cca3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:22.459813 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-774769f2-f46d-443f-910c-3e896d2cca3e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.163558 Aug 28 01:24:22.460406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1559/2895] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:22 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:22.761522 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-373d42f5-ddc7-4ce1-8fdf-afb44d2a19b2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:22.762203 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-373d42f5-ddc7-4ce1-8fdf-afb44d2a19b2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:22.768892 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-373d42f5-ddc7-4ce1-8fdf-afb44d2a19b2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:22.769417 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-373d42f5-ddc7-4ce1-8fdf-afb44d2a19b2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:22.769850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-373d42f5-ddc7-4ce1-8fdf-afb44d2a19b2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:22.770545 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-373d42f5-ddc7-4ce1-8fdf-afb44d2a19b2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:22.770995 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-373d42f5-ddc7-4ce1-8fdf-afb44d2a19b2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:22.771413 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-373d42f5-ddc7-4ce1-8fdf-afb44d2a19b2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:22.833679 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-373d42f5-ddc7-4ce1-8fdf-afb44d2a19b2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:22.842592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-373d42f5-ddc7-4ce1-8fdf-afb44d2a19b2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.085057 Aug 28 01:24:22.843157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1337/2896] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:22 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:23.482396 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-655d2135-3430-4b66-9fd3-9538ff9895b2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:23.483348 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-655d2135-3430-4b66-9fd3-9538ff9895b2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:23.490170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-655d2135-3430-4b66-9fd3-9538ff9895b2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:23.490991 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-655d2135-3430-4b66-9fd3-9538ff9895b2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:23.491416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-655d2135-3430-4b66-9fd3-9538ff9895b2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:23.492101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-655d2135-3430-4b66-9fd3-9538ff9895b2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:23.492542 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-655d2135-3430-4b66-9fd3-9538ff9895b2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:23.492994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-655d2135-3430-4b66-9fd3-9538ff9895b2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:23.656483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-655d2135-3430-4b66-9fd3-9538ff9895b2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:23.668087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-655d2135-3430-4b66-9fd3-9538ff9895b2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.189594 Aug 28 01:24:23.668804 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1560/2897] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:23 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:23.864269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3d5e24fc-9556-41e8-a7c3-d915567da65c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:23.865047 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3d5e24fc-9556-41e8-a7c3-d915567da65c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:23.872003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d5e24fc-9556-41e8-a7c3-d915567da65c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:23.872495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d5e24fc-9556-41e8-a7c3-d915567da65c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:23.872929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d5e24fc-9556-41e8-a7c3-d915567da65c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:23.873663 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d5e24fc-9556-41e8-a7c3-d915567da65c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:23.874111 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d5e24fc-9556-41e8-a7c3-d915567da65c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:23.874532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3d5e24fc-9556-41e8-a7c3-d915567da65c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:23.945665 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3d5e24fc-9556-41e8-a7c3-d915567da65c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:23.954462 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3d5e24fc-9556-41e8-a7c3-d915567da65c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.094098 Aug 28 01:24:23.955045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1338/2898] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:23 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:24.692676 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-806f9826-ecaa-4a6e-b3ff-592ed7ad3dce tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:24.693508 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-806f9826-ecaa-4a6e-b3ff-592ed7ad3dce tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:24.701926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-806f9826-ecaa-4a6e-b3ff-592ed7ad3dce tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:24.702414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-806f9826-ecaa-4a6e-b3ff-592ed7ad3dce tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:24.702823 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-806f9826-ecaa-4a6e-b3ff-592ed7ad3dce tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:24.703541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-806f9826-ecaa-4a6e-b3ff-592ed7ad3dce tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:24.703991 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-806f9826-ecaa-4a6e-b3ff-592ed7ad3dce tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:24.704412 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-806f9826-ecaa-4a6e-b3ff-592ed7ad3dce tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:24.707004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:24:24.870724 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-806f9826-ecaa-4a6e-b3ff-592ed7ad3dce tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:24.879538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-806f9826-ecaa-4a6e-b3ff-592ed7ad3dce tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.192475 Aug 28 01:24:24.880113 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1561/2899] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:24 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:24.978359 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-38746b24-934b-4048-a80f-c2ecb0d598f9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:24.978951 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-38746b24-934b-4048-a80f-c2ecb0d598f9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:24.987333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38746b24-934b-4048-a80f-c2ecb0d598f9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:24.987918 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38746b24-934b-4048-a80f-c2ecb0d598f9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:24.988425 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38746b24-934b-4048-a80f-c2ecb0d598f9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:24.989312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38746b24-934b-4048-a80f-c2ecb0d598f9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:24.989857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38746b24-934b-4048-a80f-c2ecb0d598f9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:24.990360 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-38746b24-934b-4048-a80f-c2ecb0d598f9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:25.054105 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-38746b24-934b-4048-a80f-c2ecb0d598f9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:25.062428 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-38746b24-934b-4048-a80f-c2ecb0d598f9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089143 Aug 28 01:24:25.062994 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1339/2900] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:24 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:25.910210 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1a258fed-a0cd-46d2-8087-f5fdb7c4e810 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:25.911092 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1a258fed-a0cd-46d2-8087-f5fdb7c4e810 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:25.914607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=50 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:24:25.921765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a258fed-a0cd-46d2-8087-f5fdb7c4e810 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:25.922242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a258fed-a0cd-46d2-8087-f5fdb7c4e810 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:25.922673 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a258fed-a0cd-46d2-8087-f5fdb7c4e810 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:25.923450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a258fed-a0cd-46d2-8087-f5fdb7c4e810 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:25.923837 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a258fed-a0cd-46d2-8087-f5fdb7c4e810 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:25.924257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a258fed-a0cd-46d2-8087-f5fdb7c4e810 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:26.060221 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1a258fed-a0cd-46d2-8087-f5fdb7c4e810 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:26.069382 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1a258fed-a0cd-46d2-8087-f5fdb7c4e810 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.164835 Aug 28 01:24:26.070280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1562/2901] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:25 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:26.090390 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d1a45f52-178f-4aeb-9cf5-6f319961c4e2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:26.091026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d1a45f52-178f-4aeb-9cf5-6f319961c4e2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:26.097644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d1a45f52-178f-4aeb-9cf5-6f319961c4e2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:26.098120 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d1a45f52-178f-4aeb-9cf5-6f319961c4e2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:26.098545 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d1a45f52-178f-4aeb-9cf5-6f319961c4e2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:26.099223 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d1a45f52-178f-4aeb-9cf5-6f319961c4e2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:26.099648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d1a45f52-178f-4aeb-9cf5-6f319961c4e2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:26.100082 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d1a45f52-178f-4aeb-9cf5-6f319961c4e2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:26.168143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d1a45f52-178f-4aeb-9cf5-6f319961c4e2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:26.177188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d1a45f52-178f-4aeb-9cf5-6f319961c4e2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.090465 Aug 28 01:24:26.177738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1340/2902] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:26 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:24:27.092725 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-50634496-0a05-4332-b398-08abf377ffff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:27.093389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-50634496-0a05-4332-b398-08abf377ffff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:27.100317 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-50634496-0a05-4332-b398-08abf377ffff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:27.100783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-50634496-0a05-4332-b398-08abf377ffff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:27.101264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-50634496-0a05-4332-b398-08abf377ffff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:27.101944 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-50634496-0a05-4332-b398-08abf377ffff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:27.102420 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-50634496-0a05-4332-b398-08abf377ffff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:27.102838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-50634496-0a05-4332-b398-08abf377ffff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:27.199565 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7b0a1169-66f3-456d-b9a6-8a726dc415a9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:27.200251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7b0a1169-66f3-456d-b9a6-8a726dc415a9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:27.207502 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7b0a1169-66f3-456d-b9a6-8a726dc415a9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:27.207955 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7b0a1169-66f3-456d-b9a6-8a726dc415a9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:27.208408 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7b0a1169-66f3-456d-b9a6-8a726dc415a9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:27.209252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7b0a1169-66f3-456d-b9a6-8a726dc415a9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:27.209705 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7b0a1169-66f3-456d-b9a6-8a726dc415a9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:27.210162 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7b0a1169-66f3-456d-b9a6-8a726dc415a9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:27.266236 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-50634496-0a05-4332-b398-08abf377ffff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:27.274713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-50634496-0a05-4332-b398-08abf377ffff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.187467 Aug 28 01:24:27.275278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1563/2903] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:27 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:27.288472 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7b0a1169-66f3-456d-b9a6-8a726dc415a9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:27.297055 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7b0a1169-66f3-456d-b9a6-8a726dc415a9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.101481 Aug 28 01:24:27.297613 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1341/2904] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:27 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:28.296534 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4bed5ef0-8fbf-44c0-9b84-da20355a2049 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:28.297193 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4bed5ef0-8fbf-44c0-9b84-da20355a2049 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:28.305020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bed5ef0-8fbf-44c0-9b84-da20355a2049 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:28.305504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bed5ef0-8fbf-44c0-9b84-da20355a2049 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:28.305934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bed5ef0-8fbf-44c0-9b84-da20355a2049 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:28.306625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bed5ef0-8fbf-44c0-9b84-da20355a2049 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:28.307068 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bed5ef0-8fbf-44c0-9b84-da20355a2049 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:28.307484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4bed5ef0-8fbf-44c0-9b84-da20355a2049 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:28.318777 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0623fffa-7402-472c-8bb6-68094edf4f0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:28.319385 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0623fffa-7402-472c-8bb6-68094edf4f0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:28.326204 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0623fffa-7402-472c-8bb6-68094edf4f0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:28.326687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0623fffa-7402-472c-8bb6-68094edf4f0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:28.327153 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0623fffa-7402-472c-8bb6-68094edf4f0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:28.327839 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0623fffa-7402-472c-8bb6-68094edf4f0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:28.328279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0623fffa-7402-472c-8bb6-68094edf4f0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:28.328703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0623fffa-7402-472c-8bb6-68094edf4f0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:28.412571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0623fffa-7402-472c-8bb6-68094edf4f0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:28.421800 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0623fffa-7402-472c-8bb6-68094edf4f0b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.107181 Aug 28 01:24:28.422370 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1342/2905] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:28 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:28.478420 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4bed5ef0-8fbf-44c0-9b84-da20355a2049 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:28.489023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4bed5ef0-8fbf-44c0-9b84-da20355a2049 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.196342 Aug 28 01:24:28.489693 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1564/2906] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:28 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:29.444317 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a9fc0a4b-701a-4921-822b-293c830dde48 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:29.444924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a9fc0a4b-701a-4921-822b-293c830dde48 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:29.453166 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a9fc0a4b-701a-4921-822b-293c830dde48 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:29.453661 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a9fc0a4b-701a-4921-822b-293c830dde48 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:29.454122 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a9fc0a4b-701a-4921-822b-293c830dde48 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:29.455045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a9fc0a4b-701a-4921-822b-293c830dde48 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:29.455769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a9fc0a4b-701a-4921-822b-293c830dde48 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:29.456243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a9fc0a4b-701a-4921-822b-293c830dde48 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:29.512165 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-435f33d5-9a82-4c67-9a05-dc068d4b331c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:29.512805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-435f33d5-9a82-4c67-9a05-dc068d4b331c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:29.521719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-435f33d5-9a82-4c67-9a05-dc068d4b331c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:29.522238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-435f33d5-9a82-4c67-9a05-dc068d4b331c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:29.522962 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-435f33d5-9a82-4c67-9a05-dc068d4b331c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:29.523829 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-435f33d5-9a82-4c67-9a05-dc068d4b331c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:29.524500 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-435f33d5-9a82-4c67-9a05-dc068d4b331c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:29.524822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a9fc0a4b-701a-4921-822b-293c830dde48 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:29.525140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-435f33d5-9a82-4c67-9a05-dc068d4b331c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:29.535754 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a9fc0a4b-701a-4921-822b-293c830dde48 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.095270 Aug 28 01:24:29.536477 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1343/2907] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:29 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:29.665201 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-435f33d5-9a82-4c67-9a05-dc068d4b331c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:29.673932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-435f33d5-9a82-4c67-9a05-dc068d4b331c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.165977 Aug 28 01:24:29.674508 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1565/2908] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:29 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:30.557752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-73dc3861-64f8-422b-8644-24ea806cd22e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:30.558371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-73dc3861-64f8-422b-8644-24ea806cd22e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:30.566081 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73dc3861-64f8-422b-8644-24ea806cd22e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:30.566706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73dc3861-64f8-422b-8644-24ea806cd22e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:30.567164 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73dc3861-64f8-422b-8644-24ea806cd22e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:30.567964 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73dc3861-64f8-422b-8644-24ea806cd22e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:30.568461 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73dc3861-64f8-422b-8644-24ea806cd22e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:30.568927 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-73dc3861-64f8-422b-8644-24ea806cd22e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:30.649467 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-73dc3861-64f8-422b-8644-24ea806cd22e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:30.659813 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-73dc3861-64f8-422b-8644-24ea806cd22e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.105737 Aug 28 01:24:30.660557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1344/2909] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:30 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:30.697647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-842acbf3-785d-4d7f-8ddf-ef57dd19e7be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:30.698242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-842acbf3-785d-4d7f-8ddf-ef57dd19e7be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:30.706202 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-842acbf3-785d-4d7f-8ddf-ef57dd19e7be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:30.706682 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-842acbf3-785d-4d7f-8ddf-ef57dd19e7be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:30.707113 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-842acbf3-785d-4d7f-8ddf-ef57dd19e7be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:30.707799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-842acbf3-785d-4d7f-8ddf-ef57dd19e7be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:30.708241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-842acbf3-785d-4d7f-8ddf-ef57dd19e7be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:30.708658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-842acbf3-785d-4d7f-8ddf-ef57dd19e7be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:30.839538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-842acbf3-785d-4d7f-8ddf-ef57dd19e7be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:30.848147 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-842acbf3-785d-4d7f-8ddf-ef57dd19e7be tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.156760 Aug 28 01:24:30.848716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1566/2910] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:30 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:31.684163 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-58bb657b-753e-4830-b2c2-9a76ada956e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:31.684760 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-58bb657b-753e-4830-b2c2-9a76ada956e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:31.692535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-58bb657b-753e-4830-b2c2-9a76ada956e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:31.693058 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-58bb657b-753e-4830-b2c2-9a76ada956e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:31.693489 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-58bb657b-753e-4830-b2c2-9a76ada956e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:31.694183 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-58bb657b-753e-4830-b2c2-9a76ada956e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:31.694631 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-58bb657b-753e-4830-b2c2-9a76ada956e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:31.695059 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-58bb657b-753e-4830-b2c2-9a76ada956e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:31.757146 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-58bb657b-753e-4830-b2c2-9a76ada956e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:31.765854 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-58bb657b-753e-4830-b2c2-9a76ada956e4 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.087857 Aug 28 01:24:31.766410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1345/2911] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:31 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:31.870457 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-43e07797-ee89-4c29-ad1d-f2f2e8e6910f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:31.871152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-43e07797-ee89-4c29-ad1d-f2f2e8e6910f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:31.878562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-43e07797-ee89-4c29-ad1d-f2f2e8e6910f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:31.879084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-43e07797-ee89-4c29-ad1d-f2f2e8e6910f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:31.879555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-43e07797-ee89-4c29-ad1d-f2f2e8e6910f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:31.880294 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-43e07797-ee89-4c29-ad1d-f2f2e8e6910f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:31.880782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-43e07797-ee89-4c29-ad1d-f2f2e8e6910f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:31.881263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-43e07797-ee89-4c29-ad1d-f2f2e8e6910f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:32.023865 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-43e07797-ee89-4c29-ad1d-f2f2e8e6910f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:32.032617 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-43e07797-ee89-4c29-ad1d-f2f2e8e6910f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.166384 Aug 28 01:24:32.033250 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1567/2912] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:31 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:32.788517 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-10d6b7c6-a71e-447e-b387-0e5d3bc6d476 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:32.789188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-10d6b7c6-a71e-447e-b387-0e5d3bc6d476 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:32.796690 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10d6b7c6-a71e-447e-b387-0e5d3bc6d476 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:32.797205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10d6b7c6-a71e-447e-b387-0e5d3bc6d476 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:32.797634 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10d6b7c6-a71e-447e-b387-0e5d3bc6d476 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:32.798324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10d6b7c6-a71e-447e-b387-0e5d3bc6d476 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:32.798773 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10d6b7c6-a71e-447e-b387-0e5d3bc6d476 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:32.799198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-10d6b7c6-a71e-447e-b387-0e5d3bc6d476 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:32.863159 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-10d6b7c6-a71e-447e-b387-0e5d3bc6d476 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:32.871449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-10d6b7c6-a71e-447e-b387-0e5d3bc6d476 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.087638 Aug 28 01:24:32.872008 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1346/2913] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:32 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:33.058526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-dbda661d-3142-4835-a81c-5a8da3c424d8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:33.059255 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-dbda661d-3142-4835-a81c-5a8da3c424d8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:33.066329 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbda661d-3142-4835-a81c-5a8da3c424d8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:33.066808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbda661d-3142-4835-a81c-5a8da3c424d8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:33.067238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbda661d-3142-4835-a81c-5a8da3c424d8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:33.068083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbda661d-3142-4835-a81c-5a8da3c424d8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:33.068533 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbda661d-3142-4835-a81c-5a8da3c424d8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:33.068953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbda661d-3142-4835-a81c-5a8da3c424d8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:33.209946 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dbda661d-3142-4835-a81c-5a8da3c424d8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:33.220366 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-dbda661d-3142-4835-a81c-5a8da3c424d8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.168515 Aug 28 01:24:33.220921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1568/2914] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:33 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:33.895704 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c3bd862b-d42b-4b93-b017-acb67a0d4658 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:33.896629 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c3bd862b-d42b-4b93-b017-acb67a0d4658 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:33.905680 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3bd862b-d42b-4b93-b017-acb67a0d4658 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:33.906156 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3bd862b-d42b-4b93-b017-acb67a0d4658 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:33.906582 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3bd862b-d42b-4b93-b017-acb67a0d4658 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:33.907265 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3bd862b-d42b-4b93-b017-acb67a0d4658 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:33.907715 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3bd862b-d42b-4b93-b017-acb67a0d4658 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:33.908127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c3bd862b-d42b-4b93-b017-acb67a0d4658 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:34.129738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c3bd862b-d42b-4b93-b017-acb67a0d4658 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:34.138436 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c3bd862b-d42b-4b93-b017-acb67a0d4658 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.248207 Aug 28 01:24:34.138999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1347/2915] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:33 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:34.242145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f8ad9a23-eeb5-40fe-97cd-4295c28ccbf8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:34.242757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f8ad9a23-eeb5-40fe-97cd-4295c28ccbf8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:34.249590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f8ad9a23-eeb5-40fe-97cd-4295c28ccbf8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:34.250072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f8ad9a23-eeb5-40fe-97cd-4295c28ccbf8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:34.250511 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f8ad9a23-eeb5-40fe-97cd-4295c28ccbf8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:34.251185 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f8ad9a23-eeb5-40fe-97cd-4295c28ccbf8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:34.251636 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f8ad9a23-eeb5-40fe-97cd-4295c28ccbf8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:34.252055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f8ad9a23-eeb5-40fe-97cd-4295c28ccbf8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:34.396370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f8ad9a23-eeb5-40fe-97cd-4295c28ccbf8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:34.404839 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f8ad9a23-eeb5-40fe-97cd-4295c28ccbf8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.166714 Aug 28 01:24:34.405462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1569/2916] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:34 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:35.163904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-efaefcdb-a03c-4589-ba70-521eab497445 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:35.164865 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-efaefcdb-a03c-4589-ba70-521eab497445 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:35.173553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-efaefcdb-a03c-4589-ba70-521eab497445 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:35.174035 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-efaefcdb-a03c-4589-ba70-521eab497445 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:35.174469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-efaefcdb-a03c-4589-ba70-521eab497445 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:35.175154 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-efaefcdb-a03c-4589-ba70-521eab497445 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:35.175604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-efaefcdb-a03c-4589-ba70-521eab497445 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:35.176027 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-efaefcdb-a03c-4589-ba70-521eab497445 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:35.238656 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-efaefcdb-a03c-4589-ba70-521eab497445 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:35.246979 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-efaefcdb-a03c-4589-ba70-521eab497445 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089452 Aug 28 01:24:35.247543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1348/2917] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:35 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:35.426686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-304618cc-5ed3-442a-a151-41f9581e0db8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:35.427342 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-304618cc-5ed3-442a-a151-41f9581e0db8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:35.434476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-304618cc-5ed3-442a-a151-41f9581e0db8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:35.434962 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-304618cc-5ed3-442a-a151-41f9581e0db8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:35.435485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-304618cc-5ed3-442a-a151-41f9581e0db8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:35.436219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-304618cc-5ed3-442a-a151-41f9581e0db8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:35.436671 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-304618cc-5ed3-442a-a151-41f9581e0db8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:35.437355 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-304618cc-5ed3-442a-a151-41f9581e0db8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:35.574159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-304618cc-5ed3-442a-a151-41f9581e0db8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:35.583095 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-304618cc-5ed3-442a-a151-41f9581e0db8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.160409 Aug 28 01:24:35.583690 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1570/2918] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:35 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:36.268331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c0961bcc-0288-4df2-a5a7-1e65adf7ba87 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:36.268985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c0961bcc-0288-4df2-a5a7-1e65adf7ba87 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:36.276396 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0961bcc-0288-4df2-a5a7-1e65adf7ba87 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:36.276874 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0961bcc-0288-4df2-a5a7-1e65adf7ba87 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:36.277327 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0961bcc-0288-4df2-a5a7-1e65adf7ba87 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:36.278051 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0961bcc-0288-4df2-a5a7-1e65adf7ba87 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:36.278495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0961bcc-0288-4df2-a5a7-1e65adf7ba87 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:36.278911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c0961bcc-0288-4df2-a5a7-1e65adf7ba87 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:36.342451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c0961bcc-0288-4df2-a5a7-1e65adf7ba87 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:36.351182 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c0961bcc-0288-4df2-a5a7-1e65adf7ba87 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.086866 Aug 28 01:24:36.351751 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1349/2919] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:36 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:36.608366 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-08423efe-ff38-43b3-bc43-51ac38ef9ba8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:36.609286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-08423efe-ff38-43b3-bc43-51ac38ef9ba8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:36.615935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08423efe-ff38-43b3-bc43-51ac38ef9ba8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:36.616415 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08423efe-ff38-43b3-bc43-51ac38ef9ba8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:36.616849 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08423efe-ff38-43b3-bc43-51ac38ef9ba8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:36.617572 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08423efe-ff38-43b3-bc43-51ac38ef9ba8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:36.618021 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08423efe-ff38-43b3-bc43-51ac38ef9ba8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:36.618444 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08423efe-ff38-43b3-bc43-51ac38ef9ba8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:36.754373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-08423efe-ff38-43b3-bc43-51ac38ef9ba8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:36.763265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-08423efe-ff38-43b3-bc43-51ac38ef9ba8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.160846 Aug 28 01:24:36.763794 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1571/2920] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:36 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:37.373929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-526eeffc-3d18-4671-bc76-4330ea846512 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:37.374524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-526eeffc-3d18-4671-bc76-4330ea846512 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:37.381290 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-526eeffc-3d18-4671-bc76-4330ea846512 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:37.381770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-526eeffc-3d18-4671-bc76-4330ea846512 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:37.382197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-526eeffc-3d18-4671-bc76-4330ea846512 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:37.382911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-526eeffc-3d18-4671-bc76-4330ea846512 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:37.383354 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-526eeffc-3d18-4671-bc76-4330ea846512 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:37.383774 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-526eeffc-3d18-4671-bc76-4330ea846512 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:37.451024 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-526eeffc-3d18-4671-bc76-4330ea846512 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:37.453280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:24:37.460824 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-526eeffc-3d18-4671-bc76-4330ea846512 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.090942 Aug 28 01:24:37.461415 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1350/2921] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:37 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:37.784895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2ca9191b-4027-41c4-b298-55a14261800d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:37.785675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2ca9191b-4027-41c4-b298-55a14261800d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:37.792887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ca9191b-4027-41c4-b298-55a14261800d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:37.793380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ca9191b-4027-41c4-b298-55a14261800d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:37.793810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ca9191b-4027-41c4-b298-55a14261800d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:37.794494 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ca9191b-4027-41c4-b298-55a14261800d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:37.794937 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ca9191b-4027-41c4-b298-55a14261800d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:37.795353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2ca9191b-4027-41c4-b298-55a14261800d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:37.930667 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2ca9191b-4027-41c4-b298-55a14261800d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:37.943401 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2ca9191b-4027-41c4-b298-55a14261800d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.162219 Aug 28 01:24:37.944175 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1572/2922] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:37 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:38.486253 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e4a042b3-b7cb-4aef-adba-c8328db1e314 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:38.486883 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e4a042b3-b7cb-4aef-adba-c8328db1e314 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:38.496431 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e4a042b3-b7cb-4aef-adba-c8328db1e314 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:38.497051 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e4a042b3-b7cb-4aef-adba-c8328db1e314 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:38.497845 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e4a042b3-b7cb-4aef-adba-c8328db1e314 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:38.498530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e4a042b3-b7cb-4aef-adba-c8328db1e314 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:38.498971 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e4a042b3-b7cb-4aef-adba-c8328db1e314 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:38.499386 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e4a042b3-b7cb-4aef-adba-c8328db1e314 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:38.571283 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e4a042b3-b7cb-4aef-adba-c8328db1e314 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:38.584019 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e4a042b3-b7cb-4aef-adba-c8328db1e314 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.105233 Aug 28 01:24:38.584838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1351/2923] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:38 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:38.965097 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-287166fd-03e3-47b0-b300-b2bb721a03f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:38.965760 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-287166fd-03e3-47b0-b300-b2bb721a03f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:38.973363 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-287166fd-03e3-47b0-b300-b2bb721a03f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:38.973854 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-287166fd-03e3-47b0-b300-b2bb721a03f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:38.974284 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-287166fd-03e3-47b0-b300-b2bb721a03f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:38.974971 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-287166fd-03e3-47b0-b300-b2bb721a03f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:38.975420 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-287166fd-03e3-47b0-b300-b2bb721a03f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:38.975865 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-287166fd-03e3-47b0-b300-b2bb721a03f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:39.111639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-287166fd-03e3-47b0-b300-b2bb721a03f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:39.120687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-287166fd-03e3-47b0-b300-b2bb721a03f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.159463 Aug 28 01:24:39.121309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1573/2924] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:38 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:39.607821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-31c8d57c-18f8-4cfe-8cab-09919bccd291 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:39.608422 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-31c8d57c-18f8-4cfe-8cab-09919bccd291 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:39.615378 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31c8d57c-18f8-4cfe-8cab-09919bccd291 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:39.615863 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31c8d57c-18f8-4cfe-8cab-09919bccd291 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:39.616291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31c8d57c-18f8-4cfe-8cab-09919bccd291 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:39.617020 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31c8d57c-18f8-4cfe-8cab-09919bccd291 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:39.617451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31c8d57c-18f8-4cfe-8cab-09919bccd291 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:39.617924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-31c8d57c-18f8-4cfe-8cab-09919bccd291 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:39.687605 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-31c8d57c-18f8-4cfe-8cab-09919bccd291 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:39.699699 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-31c8d57c-18f8-4cfe-8cab-09919bccd291 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.095726 Aug 28 01:24:39.700509 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1352/2925] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:39 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:40.142779 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4c5bd1fc-1cbd-49b4-9b09-be483434fc14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:40.143387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4c5bd1fc-1cbd-49b4-9b09-be483434fc14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:40.150421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c5bd1fc-1cbd-49b4-9b09-be483434fc14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:40.150905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c5bd1fc-1cbd-49b4-9b09-be483434fc14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:40.151337 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c5bd1fc-1cbd-49b4-9b09-be483434fc14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:40.152030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c5bd1fc-1cbd-49b4-9b09-be483434fc14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:40.152481 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c5bd1fc-1cbd-49b4-9b09-be483434fc14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:40.152906 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4c5bd1fc-1cbd-49b4-9b09-be483434fc14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:40.290451 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4c5bd1fc-1cbd-49b4-9b09-be483434fc14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:40.302686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4c5bd1fc-1cbd-49b4-9b09-be483434fc14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.163621 Aug 28 01:24:40.303433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1574/2926] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:40 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:40.722342 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-20e0ae53-89db-43ce-b9e6-08fc3400bddb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:40.722986 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-20e0ae53-89db-43ce-b9e6-08fc3400bddb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:40.730108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20e0ae53-89db-43ce-b9e6-08fc3400bddb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:40.730592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20e0ae53-89db-43ce-b9e6-08fc3400bddb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:40.731046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20e0ae53-89db-43ce-b9e6-08fc3400bddb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:40.731737 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20e0ae53-89db-43ce-b9e6-08fc3400bddb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:40.732183 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20e0ae53-89db-43ce-b9e6-08fc3400bddb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:40.732602 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-20e0ae53-89db-43ce-b9e6-08fc3400bddb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:40.801427 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-20e0ae53-89db-43ce-b9e6-08fc3400bddb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:40.810106 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-20e0ae53-89db-43ce-b9e6-08fc3400bddb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.091764 Aug 28 01:24:40.810675 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1353/2927] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:40 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:41.333335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7dd41983-d65f-4e08-a52c-38584c9390e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:41.334062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7dd41983-d65f-4e08-a52c-38584c9390e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:41.342299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7dd41983-d65f-4e08-a52c-38584c9390e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:41.342876 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7dd41983-d65f-4e08-a52c-38584c9390e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:41.343432 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7dd41983-d65f-4e08-a52c-38584c9390e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:41.344263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7dd41983-d65f-4e08-a52c-38584c9390e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:41.344850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7dd41983-d65f-4e08-a52c-38584c9390e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:41.345439 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7dd41983-d65f-4e08-a52c-38584c9390e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:41.484955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7dd41983-d65f-4e08-a52c-38584c9390e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:41.494093 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7dd41983-d65f-4e08-a52c-38584c9390e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.165423 Aug 28 01:24:41.494685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1575/2928] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:41 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:41.832164 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-057f8d25-653e-4d8f-afbd-de2d9ce36f66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:41.832734 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-057f8d25-653e-4d8f-afbd-de2d9ce36f66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:41.839553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-057f8d25-653e-4d8f-afbd-de2d9ce36f66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:41.840072 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-057f8d25-653e-4d8f-afbd-de2d9ce36f66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:41.840506 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-057f8d25-653e-4d8f-afbd-de2d9ce36f66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:41.841235 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-057f8d25-653e-4d8f-afbd-de2d9ce36f66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:41.841688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-057f8d25-653e-4d8f-afbd-de2d9ce36f66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:41.842107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-057f8d25-653e-4d8f-afbd-de2d9ce36f66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:41.917400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-057f8d25-653e-4d8f-afbd-de2d9ce36f66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:41.925890 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-057f8d25-653e-4d8f-afbd-de2d9ce36f66 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.097899 Aug 28 01:24:41.926442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1354/2929] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:41 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:42.518935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-626b2fdb-a8d6-48b9-b0c1-bbcf310f118d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:42.519575 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-626b2fdb-a8d6-48b9-b0c1-bbcf310f118d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:42.526620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-626b2fdb-a8d6-48b9-b0c1-bbcf310f118d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:42.527106 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-626b2fdb-a8d6-48b9-b0c1-bbcf310f118d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:42.527537 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-626b2fdb-a8d6-48b9-b0c1-bbcf310f118d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:42.528234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-626b2fdb-a8d6-48b9-b0c1-bbcf310f118d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:42.528719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-626b2fdb-a8d6-48b9-b0c1-bbcf310f118d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:42.529175 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-626b2fdb-a8d6-48b9-b0c1-bbcf310f118d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:42.665217 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-626b2fdb-a8d6-48b9-b0c1-bbcf310f118d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:42.678042 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-626b2fdb-a8d6-48b9-b0c1-bbcf310f118d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.164084 Aug 28 01:24:42.678711 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1576/2930] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:42 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:42.949598 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bf9326c3-b36d-4ef2-a6d7-279c901b8d64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:42.950228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bf9326c3-b36d-4ef2-a6d7-279c901b8d64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:42.957217 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bf9326c3-b36d-4ef2-a6d7-279c901b8d64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:42.957706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bf9326c3-b36d-4ef2-a6d7-279c901b8d64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:42.958138 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bf9326c3-b36d-4ef2-a6d7-279c901b8d64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:42.958833 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bf9326c3-b36d-4ef2-a6d7-279c901b8d64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:42.959283 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bf9326c3-b36d-4ef2-a6d7-279c901b8d64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:42.959706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bf9326c3-b36d-4ef2-a6d7-279c901b8d64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:43.023070 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bf9326c3-b36d-4ef2-a6d7-279c901b8d64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:43.033454 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bf9326c3-b36d-4ef2-a6d7-279c901b8d64 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.087825 Aug 28 01:24:43.034003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1355/2931] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:42 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:43.700564 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-48583cc3-16e1-4c48-8649-0282781e8051 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:43.701176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-48583cc3-16e1-4c48-8649-0282781e8051 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:43.708179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48583cc3-16e1-4c48-8649-0282781e8051 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:43.708662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48583cc3-16e1-4c48-8649-0282781e8051 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:43.709135 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48583cc3-16e1-4c48-8649-0282781e8051 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:43.709835 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48583cc3-16e1-4c48-8649-0282781e8051 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:43.710292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48583cc3-16e1-4c48-8649-0282781e8051 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:43.710710 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48583cc3-16e1-4c48-8649-0282781e8051 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:43.848725 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-48583cc3-16e1-4c48-8649-0282781e8051 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:43.851834 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:24:43.861414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-48583cc3-16e1-4c48-8649-0282781e8051 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.164808 Aug 28 01:24:43.861986 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1577/2932] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:43 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:44.055069 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a56adbf9-e09f-44c4-b1db-515267274ebb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:44.055690 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a56adbf9-e09f-44c4-b1db-515267274ebb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:44.062661 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a56adbf9-e09f-44c4-b1db-515267274ebb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:44.063148 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a56adbf9-e09f-44c4-b1db-515267274ebb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:44.063575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a56adbf9-e09f-44c4-b1db-515267274ebb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:44.064562 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a56adbf9-e09f-44c4-b1db-515267274ebb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:44.065064 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a56adbf9-e09f-44c4-b1db-515267274ebb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:44.065484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a56adbf9-e09f-44c4-b1db-515267274ebb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:44.135528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a56adbf9-e09f-44c4-b1db-515267274ebb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:44.144156 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a56adbf9-e09f-44c4-b1db-515267274ebb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.093122 Aug 28 01:24:44.144754 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1356/2933] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:44 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:44.894632 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0eaf52f4-b0e7-47ec-8f0c-6b5be1572be0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:44.895406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0eaf52f4-b0e7-47ec-8f0c-6b5be1572be0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:44.904372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0eaf52f4-b0e7-47ec-8f0c-6b5be1572be0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:44.905065 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0eaf52f4-b0e7-47ec-8f0c-6b5be1572be0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:44.905797 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0eaf52f4-b0e7-47ec-8f0c-6b5be1572be0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:44.906712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0eaf52f4-b0e7-47ec-8f0c-6b5be1572be0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:44.907401 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0eaf52f4-b0e7-47ec-8f0c-6b5be1572be0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:44.907854 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0eaf52f4-b0e7-47ec-8f0c-6b5be1572be0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:45.043729 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0eaf52f4-b0e7-47ec-8f0c-6b5be1572be0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:45.056642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0eaf52f4-b0e7-47ec-8f0c-6b5be1572be0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.167252 Aug 28 01:24:45.057533 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1578/2934] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:44 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:45.168825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-67d69188-841a-445a-a44c-14d4e4780bef tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:45.169449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-67d69188-841a-445a-a44c-14d4e4780bef tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:45.178883 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67d69188-841a-445a-a44c-14d4e4780bef tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:45.179545 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67d69188-841a-445a-a44c-14d4e4780bef tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:45.180133 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67d69188-841a-445a-a44c-14d4e4780bef tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:45.181264 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67d69188-841a-445a-a44c-14d4e4780bef tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:45.181837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67d69188-841a-445a-a44c-14d4e4780bef tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:45.182442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67d69188-841a-445a-a44c-14d4e4780bef tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:45.248248 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-67d69188-841a-445a-a44c-14d4e4780bef tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:45.256824 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-67d69188-841a-445a-a44c-14d4e4780bef tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.092022 Aug 28 01:24:45.257404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1357/2935] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:45 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:46.082327 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e881a94c-e8cf-48d1-9b29-1d1ab2c7e144 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:46.082957 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e881a94c-e8cf-48d1-9b29-1d1ab2c7e144 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:46.089876 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e881a94c-e8cf-48d1-9b29-1d1ab2c7e144 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:46.090365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e881a94c-e8cf-48d1-9b29-1d1ab2c7e144 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:46.090802 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e881a94c-e8cf-48d1-9b29-1d1ab2c7e144 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:46.091503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e881a94c-e8cf-48d1-9b29-1d1ab2c7e144 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:46.091948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e881a94c-e8cf-48d1-9b29-1d1ab2c7e144 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:46.092377 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e881a94c-e8cf-48d1-9b29-1d1ab2c7e144 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:46.232829 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e881a94c-e8cf-48d1-9b29-1d1ab2c7e144 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:46.241209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e881a94c-e8cf-48d1-9b29-1d1ab2c7e144 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.163886 Aug 28 01:24:46.241775 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1579/2936] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:46 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:46.278520 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c1d54933-7dc5-4105-b7d0-1817368593bb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:46.279143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c1d54933-7dc5-4105-b7d0-1817368593bb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:46.286412 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1d54933-7dc5-4105-b7d0-1817368593bb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:46.287009 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1d54933-7dc5-4105-b7d0-1817368593bb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:46.287440 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1d54933-7dc5-4105-b7d0-1817368593bb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:46.288124 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1d54933-7dc5-4105-b7d0-1817368593bb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:46.288569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1d54933-7dc5-4105-b7d0-1817368593bb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:46.288992 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c1d54933-7dc5-4105-b7d0-1817368593bb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:46.361068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c1d54933-7dc5-4105-b7d0-1817368593bb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:46.369685 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c1d54933-7dc5-4105-b7d0-1817368593bb tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.094970 Aug 28 01:24:46.370250 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1358/2937] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:46 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:47.265524 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2d540879-740a-4610-8cba-f6c9a806c6e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:47.266160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2d540879-740a-4610-8cba-f6c9a806c6e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:47.275669 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d540879-740a-4610-8cba-f6c9a806c6e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:47.276362 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d540879-740a-4610-8cba-f6c9a806c6e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:47.276951 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d540879-740a-4610-8cba-f6c9a806c6e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:47.278025 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d540879-740a-4610-8cba-f6c9a806c6e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:47.278520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d540879-740a-4610-8cba-f6c9a806c6e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:47.279123 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2d540879-740a-4610-8cba-f6c9a806c6e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:47.391071 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5002fb36-20fb-4f4d-906a-7fbab8b91632 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:47.391758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5002fb36-20fb-4f4d-906a-7fbab8b91632 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:47.394052 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=54 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:24:47.400620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5002fb36-20fb-4f4d-906a-7fbab8b91632 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:47.401155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5002fb36-20fb-4f4d-906a-7fbab8b91632 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:47.401583 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5002fb36-20fb-4f4d-906a-7fbab8b91632 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:47.402284 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5002fb36-20fb-4f4d-906a-7fbab8b91632 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:47.402728 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5002fb36-20fb-4f4d-906a-7fbab8b91632 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:47.403140 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5002fb36-20fb-4f4d-906a-7fbab8b91632 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:47.412751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2d540879-740a-4610-8cba-f6c9a806c6e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:47.428513 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2d540879-740a-4610-8cba-f6c9a806c6e9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.161862 Aug 28 01:24:47.428513 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1580/2938] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:47 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:47.475443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5002fb36-20fb-4f4d-906a-7fbab8b91632 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:47.487388 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5002fb36-20fb-4f4d-906a-7fbab8b91632 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.100215 Aug 28 01:24:47.487954 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1359/2939] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:47 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:48.450636 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-98209929-705e-46f8-aaf2-f5f8a88fcb3b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:48.451278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-98209929-705e-46f8-aaf2-f5f8a88fcb3b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:48.458993 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98209929-705e-46f8-aaf2-f5f8a88fcb3b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:48.459473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98209929-705e-46f8-aaf2-f5f8a88fcb3b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:48.459875 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98209929-705e-46f8-aaf2-f5f8a88fcb3b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:48.460590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98209929-705e-46f8-aaf2-f5f8a88fcb3b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:48.461063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98209929-705e-46f8-aaf2-f5f8a88fcb3b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:48.461482 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-98209929-705e-46f8-aaf2-f5f8a88fcb3b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:48.516340 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-64c10f38-fbfd-4a90-b125-6945a94f7030 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:48.517015 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-64c10f38-fbfd-4a90-b125-6945a94f7030 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:48.523807 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-64c10f38-fbfd-4a90-b125-6945a94f7030 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:48.524307 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-64c10f38-fbfd-4a90-b125-6945a94f7030 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:48.524736 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-64c10f38-fbfd-4a90-b125-6945a94f7030 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:48.525459 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-64c10f38-fbfd-4a90-b125-6945a94f7030 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:48.525900 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-64c10f38-fbfd-4a90-b125-6945a94f7030 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:48.526315 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-64c10f38-fbfd-4a90-b125-6945a94f7030 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:48.598304 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-64c10f38-fbfd-4a90-b125-6945a94f7030 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:48.607228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-64c10f38-fbfd-4a90-b125-6945a94f7030 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.094655 Aug 28 01:24:48.607831 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1360/2940] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:48 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:48.624385 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-98209929-705e-46f8-aaf2-f5f8a88fcb3b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:48.633875 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-98209929-705e-46f8-aaf2-f5f8a88fcb3b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.188806 Aug 28 01:24:48.634378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1581/2941] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:48 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:49.630412 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-647d487d-3278-4595-a43c-b17608d0e634 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:49.631051 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-647d487d-3278-4595-a43c-b17608d0e634 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:49.637955 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-647d487d-3278-4595-a43c-b17608d0e634 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:49.638446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-647d487d-3278-4595-a43c-b17608d0e634 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:49.638879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-647d487d-3278-4595-a43c-b17608d0e634 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:49.639588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-647d487d-3278-4595-a43c-b17608d0e634 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:49.640039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-647d487d-3278-4595-a43c-b17608d0e634 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:49.640460 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-647d487d-3278-4595-a43c-b17608d0e634 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:49.656721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6471f571-e713-4c60-925f-dc9e739e3560 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:49.658217 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6471f571-e713-4c60-925f-dc9e739e3560 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:49.666620 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6471f571-e713-4c60-925f-dc9e739e3560 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:49.667066 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6471f571-e713-4c60-925f-dc9e739e3560 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:49.667650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6471f571-e713-4c60-925f-dc9e739e3560 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:49.668426 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6471f571-e713-4c60-925f-dc9e739e3560 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:49.668880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6471f571-e713-4c60-925f-dc9e739e3560 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:49.669267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6471f571-e713-4c60-925f-dc9e739e3560 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:49.705188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-647d487d-3278-4595-a43c-b17608d0e634 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:49.713853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-647d487d-3278-4595-a43c-b17608d0e634 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.087510 Aug 28 01:24:49.714410 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1361/2942] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:49 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:49.804982 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6471f571-e713-4c60-925f-dc9e739e3560 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:49.813451 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6471f571-e713-4c60-925f-dc9e739e3560 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.161421 Aug 28 01:24:49.814022 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1582/2943] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:49 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:50.735503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-08d5b72e-7391-4692-a8b9-2c99665f1f6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:50.736125 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-08d5b72e-7391-4692-a8b9-2c99665f1f6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:50.743974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08d5b72e-7391-4692-a8b9-2c99665f1f6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:50.744462 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08d5b72e-7391-4692-a8b9-2c99665f1f6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:50.744890 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08d5b72e-7391-4692-a8b9-2c99665f1f6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:50.745634 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08d5b72e-7391-4692-a8b9-2c99665f1f6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:50.746116 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08d5b72e-7391-4692-a8b9-2c99665f1f6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:50.746541 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-08d5b72e-7391-4692-a8b9-2c99665f1f6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:50.813571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-08d5b72e-7391-4692-a8b9-2c99665f1f6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:50.827320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-08d5b72e-7391-4692-a8b9-2c99665f1f6f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.095299 Aug 28 01:24:50.828039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1362/2944] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:50 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:50.835425 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9a91995c-410b-491d-9162-c595963da041 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:50.836119 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9a91995c-410b-491d-9162-c595963da041 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:50.843359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a91995c-410b-491d-9162-c595963da041 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:50.843847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a91995c-410b-491d-9162-c595963da041 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:50.844280 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a91995c-410b-491d-9162-c595963da041 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:50.845022 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a91995c-410b-491d-9162-c595963da041 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:50.845478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a91995c-410b-491d-9162-c595963da041 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:50.845996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a91995c-410b-491d-9162-c595963da041 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:50.979713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9a91995c-410b-491d-9162-c595963da041 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:50.988747 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9a91995c-410b-491d-9162-c595963da041 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.156993 Aug 28 01:24:50.989251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1583/2945] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:50 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:51.849457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b227cb5a-aef3-4661-86e2-027e71863788 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:51.850129 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b227cb5a-aef3-4661-86e2-027e71863788 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:51.857010 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b227cb5a-aef3-4661-86e2-027e71863788 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:51.857533 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b227cb5a-aef3-4661-86e2-027e71863788 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:51.857970 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b227cb5a-aef3-4661-86e2-027e71863788 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:51.858744 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b227cb5a-aef3-4661-86e2-027e71863788 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:51.859120 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b227cb5a-aef3-4661-86e2-027e71863788 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:51.859544 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b227cb5a-aef3-4661-86e2-027e71863788 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:51.926244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b227cb5a-aef3-4661-86e2-027e71863788 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:51.935384 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b227cb5a-aef3-4661-86e2-027e71863788 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089922 Aug 28 01:24:51.935945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1363/2946] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:51 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:52.014596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9803a5d1-0354-4e5e-bd2a-7d0bee2b91aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:52.015635 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9803a5d1-0354-4e5e-bd2a-7d0bee2b91aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:52.023212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9803a5d1-0354-4e5e-bd2a-7d0bee2b91aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:52.023717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9803a5d1-0354-4e5e-bd2a-7d0bee2b91aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:52.024149 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9803a5d1-0354-4e5e-bd2a-7d0bee2b91aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:52.024921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9803a5d1-0354-4e5e-bd2a-7d0bee2b91aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:52.025514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9803a5d1-0354-4e5e-bd2a-7d0bee2b91aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:52.025963 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9803a5d1-0354-4e5e-bd2a-7d0bee2b91aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:52.162454 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9803a5d1-0354-4e5e-bd2a-7d0bee2b91aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:52.173846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9803a5d1-0354-4e5e-bd2a-7d0bee2b91aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.164641 Aug 28 01:24:52.175097 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1584/2947] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:52 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:52.371607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9d038924-9383-4ddc-89a4-13e9b57d705a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:52.372028 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9d038924-9383-4ddc-89a4-13e9b57d705a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:52.378526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d038924-9383-4ddc-89a4-13e9b57d705a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:52.379026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d038924-9383-4ddc-89a4-13e9b57d705a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:52.379452 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d038924-9383-4ddc-89a4-13e9b57d705a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:52.380155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d038924-9383-4ddc-89a4-13e9b57d705a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:52.380604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d038924-9383-4ddc-89a4-13e9b57d705a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:52.381045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d038924-9383-4ddc-89a4-13e9b57d705a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:52.406118 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9d038924-9383-4ddc-89a4-13e9b57d705a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc/ips" status: 200 len: 149 microversion: 2.1 time: 0.038486 Aug 28 01:24:52.406700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1364/2948] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Aug 28 01:24:52 2022] GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc/ips => generated 149 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:24:52.957649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c0322a33-bb32-4639-81b7-7e02d4583b40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:52.958253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c0322a33-bb32-4639-81b7-7e02d4583b40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:52.965008 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0322a33-bb32-4639-81b7-7e02d4583b40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:52.965500 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0322a33-bb32-4639-81b7-7e02d4583b40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:52.965927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0322a33-bb32-4639-81b7-7e02d4583b40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:52.966623 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0322a33-bb32-4639-81b7-7e02d4583b40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:52.967068 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0322a33-bb32-4639-81b7-7e02d4583b40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:52.967481 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0322a33-bb32-4639-81b7-7e02d4583b40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:53.030824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c0322a33-bb32-4639-81b7-7e02d4583b40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:53.039697 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c0322a33-bb32-4639-81b7-7e02d4583b40 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.085990 Aug 28 01:24:53.040288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1585/2949] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:52 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:53.197618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6d2a03ed-694c-4862-990c-dfdac2ee1f3d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:53.198727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6d2a03ed-694c-4862-990c-dfdac2ee1f3d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:53.207063 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6d2a03ed-694c-4862-990c-dfdac2ee1f3d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:53.207834 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6d2a03ed-694c-4862-990c-dfdac2ee1f3d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:53.207834 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6d2a03ed-694c-4862-990c-dfdac2ee1f3d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:53.208455 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6d2a03ed-694c-4862-990c-dfdac2ee1f3d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:53.208936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6d2a03ed-694c-4862-990c-dfdac2ee1f3d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:53.209661 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6d2a03ed-694c-4862-990c-dfdac2ee1f3d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:53.349652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6d2a03ed-694c-4862-990c-dfdac2ee1f3d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:53.358740 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6d2a03ed-694c-4862-990c-dfdac2ee1f3d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.166206 Aug 28 01:24:53.359350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1365/2950] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:53 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:24:54.061364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4fe1063f-b6e2-42d4-866e-3f13774e3466 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:54.061988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4fe1063f-b6e2-42d4-866e-3f13774e3466 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:54.069490 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fe1063f-b6e2-42d4-866e-3f13774e3466 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:54.069958 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fe1063f-b6e2-42d4-866e-3f13774e3466 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:54.070398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fe1063f-b6e2-42d4-866e-3f13774e3466 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:54.071086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fe1063f-b6e2-42d4-866e-3f13774e3466 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:54.071533 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fe1063f-b6e2-42d4-866e-3f13774e3466 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:54.071948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fe1063f-b6e2-42d4-866e-3f13774e3466 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:54.134856 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4fe1063f-b6e2-42d4-866e-3f13774e3466 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:54.143520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4fe1063f-b6e2-42d4-866e-3f13774e3466 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.085965 Aug 28 01:24:54.144098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1586/2951] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:54 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:54.380671 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-056fbf3b-9e9b-4ef3-b8ad-3380468a1863 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:54.381318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-056fbf3b-9e9b-4ef3-b8ad-3380468a1863 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:54.388266 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-056fbf3b-9e9b-4ef3-b8ad-3380468a1863 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:54.388758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-056fbf3b-9e9b-4ef3-b8ad-3380468a1863 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:54.389240 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-056fbf3b-9e9b-4ef3-b8ad-3380468a1863 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:54.390276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-056fbf3b-9e9b-4ef3-b8ad-3380468a1863 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:54.391022 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-056fbf3b-9e9b-4ef3-b8ad-3380468a1863 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:54.391728 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-056fbf3b-9e9b-4ef3-b8ad-3380468a1863 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:54.524755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-056fbf3b-9e9b-4ef3-b8ad-3380468a1863 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:54.533309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-056fbf3b-9e9b-4ef3-b8ad-3380468a1863 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.156629 Aug 28 01:24:54.533847 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1366/2952] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:54 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:55.164168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-867a04e5-4445-4e75-9588-9b0c3ed3ffd0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:55.164796 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-867a04e5-4445-4e75-9588-9b0c3ed3ffd0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:55.171761 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-867a04e5-4445-4e75-9588-9b0c3ed3ffd0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:55.172279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-867a04e5-4445-4e75-9588-9b0c3ed3ffd0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:55.172726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-867a04e5-4445-4e75-9588-9b0c3ed3ffd0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:55.173432 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-867a04e5-4445-4e75-9588-9b0c3ed3ffd0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:55.173923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-867a04e5-4445-4e75-9588-9b0c3ed3ffd0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:55.174340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-867a04e5-4445-4e75-9588-9b0c3ed3ffd0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:55.240039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-867a04e5-4445-4e75-9588-9b0c3ed3ffd0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:55.248478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-867a04e5-4445-4e75-9588-9b0c3ed3ffd0 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.088167 Aug 28 01:24:55.249088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1587/2953] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:55 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:55.441471 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0189972a-72eb-4803-8488-eea0897d2d59 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:55.442130 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0189972a-72eb-4803-8488-eea0897d2d59 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:55.448710 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0189972a-72eb-4803-8488-eea0897d2d59 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:55.449229 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0189972a-72eb-4803-8488-eea0897d2d59 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:55.449671 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0189972a-72eb-4803-8488-eea0897d2d59 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:55.450371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0189972a-72eb-4803-8488-eea0897d2d59 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:55.450817 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0189972a-72eb-4803-8488-eea0897d2d59 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:55.451229 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0189972a-72eb-4803-8488-eea0897d2d59 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:55.477276 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0189972a-72eb-4803-8488-eea0897d2d59 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce/ips" status: 200 len: 152 microversion: 2.20 time: 0.040151 Aug 28 01:24:55.477850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1367/2954] 10.222.0.42 () {60 vars in 1289 bytes} [Sun Aug 28 01:24:55 2022] GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce/ips => generated 152 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Aug 28 01:24:55.555043 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d8a2ca5d-e942-41b4-9b31-e261b2769827 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:55.555660 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d8a2ca5d-e942-41b4-9b31-e261b2769827 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:55.562440 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d8a2ca5d-e942-41b4-9b31-e261b2769827 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:55.562909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d8a2ca5d-e942-41b4-9b31-e261b2769827 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:55.563340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d8a2ca5d-e942-41b4-9b31-e261b2769827 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:55.564029 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d8a2ca5d-e942-41b4-9b31-e261b2769827 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:55.564931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d8a2ca5d-e942-41b4-9b31-e261b2769827 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:55.565387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d8a2ca5d-e942-41b4-9b31-e261b2769827 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:55.695659 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d8a2ca5d-e942-41b4-9b31-e261b2769827 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:55.704413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d8a2ca5d-e942-41b4-9b31-e261b2769827 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.153253 Aug 28 01:24:55.705006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1588/2955] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:55 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:56.269845 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0199ae39-728a-4abe-b103-e5213acadb28 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:56.270451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0199ae39-728a-4abe-b103-e5213acadb28 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:56.277048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0199ae39-728a-4abe-b103-e5213acadb28 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:56.277561 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0199ae39-728a-4abe-b103-e5213acadb28 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:56.277988 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0199ae39-728a-4abe-b103-e5213acadb28 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:56.278698 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0199ae39-728a-4abe-b103-e5213acadb28 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:56.279116 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0199ae39-728a-4abe-b103-e5213acadb28 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:56.279518 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0199ae39-728a-4abe-b103-e5213acadb28 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:56.282088 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:24:56.343048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0199ae39-728a-4abe-b103-e5213acadb28 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:56.351706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0199ae39-728a-4abe-b103-e5213acadb28 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.085885 Aug 28 01:24:56.352267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1368/2956] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:56 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:56.726068 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-08817bdd-9cb9-4665-868a-a81cc667d288 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:56.726882 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-08817bdd-9cb9-4665-868a-a81cc667d288 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:56.729369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=54 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:24:56.736636 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08817bdd-9cb9-4665-868a-a81cc667d288 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:56.737152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08817bdd-9cb9-4665-868a-a81cc667d288 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:56.737584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08817bdd-9cb9-4665-868a-a81cc667d288 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:56.738283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08817bdd-9cb9-4665-868a-a81cc667d288 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:56.738743 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08817bdd-9cb9-4665-868a-a81cc667d288 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:56.739160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08817bdd-9cb9-4665-868a-a81cc667d288 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:56.881787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-08817bdd-9cb9-4665-868a-a81cc667d288 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:56.894165 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-08817bdd-9cb9-4665-868a-a81cc667d288 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.171751 Aug 28 01:24:56.895129 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1589/2957] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:56 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:57.373538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-856c4c29-b15c-46df-9f40-145d739dd032 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:57.374154 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-856c4c29-b15c-46df-9f40-145d739dd032 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:57.380789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-856c4c29-b15c-46df-9f40-145d739dd032 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:57.381319 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-856c4c29-b15c-46df-9f40-145d739dd032 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:57.381738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-856c4c29-b15c-46df-9f40-145d739dd032 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:57.382420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-856c4c29-b15c-46df-9f40-145d739dd032 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:57.382868 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-856c4c29-b15c-46df-9f40-145d739dd032 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:57.383280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-856c4c29-b15c-46df-9f40-145d739dd032 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:57.448941 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-856c4c29-b15c-46df-9f40-145d739dd032 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:57.457675 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-856c4c29-b15c-46df-9f40-145d739dd032 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.088175 Aug 28 01:24:57.458289 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1369/2958] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:57 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:57.918720 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f281348e-260e-4fd0-a9ae-5fb573c7a6bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:57.919348 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f281348e-260e-4fd0-a9ae-5fb573c7a6bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:57.928417 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f281348e-260e-4fd0-a9ae-5fb573c7a6bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:57.928906 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f281348e-260e-4fd0-a9ae-5fb573c7a6bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:57.929370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f281348e-260e-4fd0-a9ae-5fb573c7a6bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:57.930054 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f281348e-260e-4fd0-a9ae-5fb573c7a6bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:57.930541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f281348e-260e-4fd0-a9ae-5fb573c7a6bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:57.930947 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f281348e-260e-4fd0-a9ae-5fb573c7a6bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:58.066707 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f281348e-260e-4fd0-a9ae-5fb573c7a6bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:58.075117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f281348e-260e-4fd0-a9ae-5fb573c7a6bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.162701 Aug 28 01:24:58.075702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1590/2959] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:57 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:58.487178 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b7871886-7e3b-4a48-8752-db997a72393b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:58.487835 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b7871886-7e3b-4a48-8752-db997a72393b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:58.497026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b7871886-7e3b-4a48-8752-db997a72393b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:58.497642 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b7871886-7e3b-4a48-8752-db997a72393b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:58.498150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b7871886-7e3b-4a48-8752-db997a72393b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:58.499061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b7871886-7e3b-4a48-8752-db997a72393b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:58.499603 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b7871886-7e3b-4a48-8752-db997a72393b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:58.500115 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b7871886-7e3b-4a48-8752-db997a72393b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:58.566785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b7871886-7e3b-4a48-8752-db997a72393b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:58.575048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b7871886-7e3b-4a48-8752-db997a72393b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.093997 Aug 28 01:24:58.575593 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1370/2960] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:58 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:59.096845 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bed9d006-9d84-4774-895e-9d5ef3f084c8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:59.097501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bed9d006-9d84-4774-895e-9d5ef3f084c8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:59.104278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bed9d006-9d84-4774-895e-9d5ef3f084c8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:59.104756 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bed9d006-9d84-4774-895e-9d5ef3f084c8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:59.105216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bed9d006-9d84-4774-895e-9d5ef3f084c8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:59.105898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bed9d006-9d84-4774-895e-9d5ef3f084c8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:59.106352 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bed9d006-9d84-4774-895e-9d5ef3f084c8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:59.106786 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bed9d006-9d84-4774-895e-9d5ef3f084c8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:59.244487 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bed9d006-9d84-4774-895e-9d5ef3f084c8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:59.253514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bed9d006-9d84-4774-895e-9d5ef3f084c8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.160756 Aug 28 01:24:59.254062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1591/2961] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:59 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:24:59.596173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b0706f7e-262b-4e2a-b5dd-f4b3be6a92ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:24:59.596809 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b0706f7e-262b-4e2a-b5dd-f4b3be6a92ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:24:59.604567 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0706f7e-262b-4e2a-b5dd-f4b3be6a92ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:59.605181 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0706f7e-262b-4e2a-b5dd-f4b3be6a92ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:59.605651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0706f7e-262b-4e2a-b5dd-f4b3be6a92ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:59.606577 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0706f7e-262b-4e2a-b5dd-f4b3be6a92ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:24:59.607215 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0706f7e-262b-4e2a-b5dd-f4b3be6a92ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:24:59.607653 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0706f7e-262b-4e2a-b5dd-f4b3be6a92ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:24:59.672124 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b0706f7e-262b-4e2a-b5dd-f4b3be6a92ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:24:59.681257 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b0706f7e-262b-4e2a-b5dd-f4b3be6a92ec tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.088991 Aug 28 01:24:59.681825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1371/2962] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:24:59 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:00.277281 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2bc72fb1-bf2a-4ef5-9a1c-7afacd66c831 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:00.277915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-2bc72fb1-bf2a-4ef5-9a1c-7afacd66c831 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:00.286921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2bc72fb1-bf2a-4ef5-9a1c-7afacd66c831 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:00.287403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2bc72fb1-bf2a-4ef5-9a1c-7afacd66c831 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:00.287831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2bc72fb1-bf2a-4ef5-9a1c-7afacd66c831 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:00.288572 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2bc72fb1-bf2a-4ef5-9a1c-7afacd66c831 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:00.289054 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2bc72fb1-bf2a-4ef5-9a1c-7afacd66c831 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:00.289499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-2bc72fb1-bf2a-4ef5-9a1c-7afacd66c831 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:00.426478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-2bc72fb1-bf2a-4ef5-9a1c-7afacd66c831 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:00.438500 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2bc72fb1-bf2a-4ef5-9a1c-7afacd66c831 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.166497 Aug 28 01:25:00.439651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1592/2963] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:00 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:00.703529 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-97358f02-069d-4468-afc7-83edfc25aee2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:00.704121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-97358f02-069d-4468-afc7-83edfc25aee2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:00.711009 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-97358f02-069d-4468-afc7-83edfc25aee2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:00.711499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-97358f02-069d-4468-afc7-83edfc25aee2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:00.711933 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-97358f02-069d-4468-afc7-83edfc25aee2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:00.712631 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-97358f02-069d-4468-afc7-83edfc25aee2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:00.713110 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-97358f02-069d-4468-afc7-83edfc25aee2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:00.713537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-97358f02-069d-4468-afc7-83edfc25aee2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:00.776518 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-97358f02-069d-4468-afc7-83edfc25aee2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:00.785302 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-97358f02-069d-4468-afc7-83edfc25aee2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.085949 Aug 28 01:25:00.785827 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1372/2964] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:00 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:01.462237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5809a616-c6cc-4ffd-bd08-0d66de86ad8c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:01.462838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5809a616-c6cc-4ffd-bd08-0d66de86ad8c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:01.469789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5809a616-c6cc-4ffd-bd08-0d66de86ad8c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:01.470596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5809a616-c6cc-4ffd-bd08-0d66de86ad8c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:01.470759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5809a616-c6cc-4ffd-bd08-0d66de86ad8c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:01.471429 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5809a616-c6cc-4ffd-bd08-0d66de86ad8c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:01.471905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5809a616-c6cc-4ffd-bd08-0d66de86ad8c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:01.472339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5809a616-c6cc-4ffd-bd08-0d66de86ad8c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:01.612825 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5809a616-c6cc-4ffd-bd08-0d66de86ad8c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:01.621287 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5809a616-c6cc-4ffd-bd08-0d66de86ad8c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.162887 Aug 28 01:25:01.621854 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1593/2965] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:01 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:01.807272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ac0b2a4f-e205-411c-a504-758c010be067 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:01.807847 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ac0b2a4f-e205-411c-a504-758c010be067 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:01.815785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac0b2a4f-e205-411c-a504-758c010be067 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:01.816401 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac0b2a4f-e205-411c-a504-758c010be067 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:01.817139 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac0b2a4f-e205-411c-a504-758c010be067 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:01.817886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac0b2a4f-e205-411c-a504-758c010be067 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:01.818345 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac0b2a4f-e205-411c-a504-758c010be067 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:01.818770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac0b2a4f-e205-411c-a504-758c010be067 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:01.887050 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ac0b2a4f-e205-411c-a504-758c010be067 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:01.896538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ac0b2a4f-e205-411c-a504-758c010be067 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.093244 Aug 28 01:25:01.897181 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1373/2966] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:01 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:02.642876 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f12b1c4a-ec9c-424f-922a-6d2e6a8ace08 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:02.643530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f12b1c4a-ec9c-424f-922a-6d2e6a8ace08 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:02.650399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f12b1c4a-ec9c-424f-922a-6d2e6a8ace08 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:02.650862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f12b1c4a-ec9c-424f-922a-6d2e6a8ace08 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:02.651286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f12b1c4a-ec9c-424f-922a-6d2e6a8ace08 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:02.651969 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f12b1c4a-ec9c-424f-922a-6d2e6a8ace08 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:02.652413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f12b1c4a-ec9c-424f-922a-6d2e6a8ace08 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:02.652823 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f12b1c4a-ec9c-424f-922a-6d2e6a8ace08 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:02.667599 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:25:02.785916 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f12b1c4a-ec9c-424f-922a-6d2e6a8ace08 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:02.795415 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f12b1c4a-ec9c-424f-922a-6d2e6a8ace08 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.156434 Aug 28 01:25:02.795969 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1594/2967] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:02 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:02.918611 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-246c594a-fa21-4d3d-847d-adce4e784c0c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:02.919292 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-246c594a-fa21-4d3d-847d-adce4e784c0c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:02.926917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-246c594a-fa21-4d3d-847d-adce4e784c0c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:02.927409 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-246c594a-fa21-4d3d-847d-adce4e784c0c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:02.927843 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-246c594a-fa21-4d3d-847d-adce4e784c0c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:02.928538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-246c594a-fa21-4d3d-847d-adce4e784c0c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:02.928990 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-246c594a-fa21-4d3d-847d-adce4e784c0c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:02.929439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-246c594a-fa21-4d3d-847d-adce4e784c0c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:03.003059 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-246c594a-fa21-4d3d-847d-adce4e784c0c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:03.015205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-246c594a-fa21-4d3d-847d-adce4e784c0c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.100648 Aug 28 01:25:03.016024 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1374/2968] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:02 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:03.824981 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-69b982c5-5962-4915-aa15-5ea5150d5393 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:03.825585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-69b982c5-5962-4915-aa15-5ea5150d5393 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:03.832497 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69b982c5-5962-4915-aa15-5ea5150d5393 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:03.833094 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69b982c5-5962-4915-aa15-5ea5150d5393 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:03.833554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69b982c5-5962-4915-aa15-5ea5150d5393 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:03.834251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69b982c5-5962-4915-aa15-5ea5150d5393 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:03.834700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69b982c5-5962-4915-aa15-5ea5150d5393 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:03.835121 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69b982c5-5962-4915-aa15-5ea5150d5393 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:03.981324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-69b982c5-5962-4915-aa15-5ea5150d5393 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:03.990198 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-69b982c5-5962-4915-aa15-5ea5150d5393 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.169216 Aug 28 01:25:03.990775 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1595/2969] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:03 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:04.037827 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-85730a4d-37ce-4479-a7ec-2f46e3a99493 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:04.038420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-85730a4d-37ce-4479-a7ec-2f46e3a99493 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:04.045397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85730a4d-37ce-4479-a7ec-2f46e3a99493 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:04.045874 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85730a4d-37ce-4479-a7ec-2f46e3a99493 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:04.046303 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85730a4d-37ce-4479-a7ec-2f46e3a99493 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:04.046989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85730a4d-37ce-4479-a7ec-2f46e3a99493 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:04.047430 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85730a4d-37ce-4479-a7ec-2f46e3a99493 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:04.047845 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-85730a4d-37ce-4479-a7ec-2f46e3a99493 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:04.115291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-85730a4d-37ce-4479-a7ec-2f46e3a99493 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:04.123896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-85730a4d-37ce-4479-a7ec-2f46e3a99493 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.089959 Aug 28 01:25:04.124455 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1375/2970] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:04 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:05.017964 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0e38c4d3-8495-4a1e-935d-8987f1598dc1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:05.018626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0e38c4d3-8495-4a1e-935d-8987f1598dc1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:05.026251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e38c4d3-8495-4a1e-935d-8987f1598dc1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:05.026878 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e38c4d3-8495-4a1e-935d-8987f1598dc1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:05.027406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e38c4d3-8495-4a1e-935d-8987f1598dc1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:05.028432 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e38c4d3-8495-4a1e-935d-8987f1598dc1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:05.029093 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e38c4d3-8495-4a1e-935d-8987f1598dc1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:05.029832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0e38c4d3-8495-4a1e-935d-8987f1598dc1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:05.153805 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d1910129-851f-4966-a32d-b1296f9d62ac tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:05.154446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d1910129-851f-4966-a32d-b1296f9d62ac tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:05.161544 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d1910129-851f-4966-a32d-b1296f9d62ac tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:05.162038 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d1910129-851f-4966-a32d-b1296f9d62ac tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:05.162458 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d1910129-851f-4966-a32d-b1296f9d62ac tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:05.163153 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d1910129-851f-4966-a32d-b1296f9d62ac tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:05.163601 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d1910129-851f-4966-a32d-b1296f9d62ac tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:05.164018 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d1910129-851f-4966-a32d-b1296f9d62ac tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:05.175539 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0e38c4d3-8495-4a1e-935d-8987f1598dc1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:05.185273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0e38c4d3-8495-4a1e-935d-8987f1598dc1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.171838 Aug 28 01:25:05.185866 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1596/2971] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:05 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:05.231902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d1910129-851f-4966-a32d-b1296f9d62ac tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:05.240549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d1910129-851f-4966-a32d-b1296f9d62ac tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.090528 Aug 28 01:25:05.241157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1376/2972] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:05 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:06.209333 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6b1cf22c-dfa8-4127-a93e-ee8fae8d41dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:06.209921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6b1cf22c-dfa8-4127-a93e-ee8fae8d41dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:06.218490 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b1cf22c-dfa8-4127-a93e-ee8fae8d41dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:06.218980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b1cf22c-dfa8-4127-a93e-ee8fae8d41dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:06.219399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b1cf22c-dfa8-4127-a93e-ee8fae8d41dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:06.220202 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b1cf22c-dfa8-4127-a93e-ee8fae8d41dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:06.220867 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b1cf22c-dfa8-4127-a93e-ee8fae8d41dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:06.221472 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6b1cf22c-dfa8-4127-a93e-ee8fae8d41dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:06.261411 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f805a90f-cfbe-47ae-947a-bbf8775ab383 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:06.261997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f805a90f-cfbe-47ae-947a-bbf8775ab383 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:06.269094 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f805a90f-cfbe-47ae-947a-bbf8775ab383 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:06.269630 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f805a90f-cfbe-47ae-947a-bbf8775ab383 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:06.270084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f805a90f-cfbe-47ae-947a-bbf8775ab383 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:06.270787 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f805a90f-cfbe-47ae-947a-bbf8775ab383 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:06.271209 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f805a90f-cfbe-47ae-947a-bbf8775ab383 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:06.271683 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f805a90f-cfbe-47ae-947a-bbf8775ab383 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:06.343947 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f805a90f-cfbe-47ae-947a-bbf8775ab383 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:06.352421 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f805a90f-cfbe-47ae-947a-bbf8775ab383 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.094801 Aug 28 01:25:06.353005 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1377/2973] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:06 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:06.373642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6b1cf22c-dfa8-4127-a93e-ee8fae8d41dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:06.382285 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6b1cf22c-dfa8-4127-a93e-ee8fae8d41dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.178265 Aug 28 01:25:06.382850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1597/2974] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:06 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:07.374224 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a929103e-3e1f-4413-ba48-222456462193 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:07.374821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a929103e-3e1f-4413-ba48-222456462193 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:07.382045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a929103e-3e1f-4413-ba48-222456462193 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:07.382507 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a929103e-3e1f-4413-ba48-222456462193 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:07.382909 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a929103e-3e1f-4413-ba48-222456462193 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:07.383623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a929103e-3e1f-4413-ba48-222456462193 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:07.384169 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a929103e-3e1f-4413-ba48-222456462193 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:07.384495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a929103e-3e1f-4413-ba48-222456462193 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:07.410334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8b3496d5-9a17-4147-8df4-b43bd1355cf3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:07.410782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8b3496d5-9a17-4147-8df4-b43bd1355cf3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:07.417778 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b3496d5-9a17-4147-8df4-b43bd1355cf3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:07.418262 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b3496d5-9a17-4147-8df4-b43bd1355cf3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:07.418685 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b3496d5-9a17-4147-8df4-b43bd1355cf3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:07.419376 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b3496d5-9a17-4147-8df4-b43bd1355cf3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:07.419824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b3496d5-9a17-4147-8df4-b43bd1355cf3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:07.420243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8b3496d5-9a17-4147-8df4-b43bd1355cf3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:07.447993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a929103e-3e1f-4413-ba48-222456462193 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:07.456407 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a929103e-3e1f-4413-ba48-222456462193 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.086190 Aug 28 01:25:07.456990 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1378/2975] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:07 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:07.560908 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8b3496d5-9a17-4147-8df4-b43bd1355cf3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:07.574515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8b3496d5-9a17-4147-8df4-b43bd1355cf3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.168060 Aug 28 01:25:07.575408 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1598/2976] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:07 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:08.477978 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b74755b4-f6e3-4412-aa4f-ea17b7978032 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:08.478584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b74755b4-f6e3-4412-aa4f-ea17b7978032 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:08.485784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b74755b4-f6e3-4412-aa4f-ea17b7978032 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:08.486277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b74755b4-f6e3-4412-aa4f-ea17b7978032 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:08.486687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b74755b4-f6e3-4412-aa4f-ea17b7978032 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:08.487415 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b74755b4-f6e3-4412-aa4f-ea17b7978032 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:08.487867 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b74755b4-f6e3-4412-aa4f-ea17b7978032 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:08.488290 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b74755b4-f6e3-4412-aa4f-ea17b7978032 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:08.551861 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b74755b4-f6e3-4412-aa4f-ea17b7978032 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:08.560936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b74755b4-f6e3-4412-aa4f-ea17b7978032 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.086972 Aug 28 01:25:08.561556 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1379/2977] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:08 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:08.602112 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-12d6ab6b-80af-429f-8116-24323fc10458 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:08.602886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-12d6ab6b-80af-429f-8116-24323fc10458 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:08.610202 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12d6ab6b-80af-429f-8116-24323fc10458 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:08.610699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12d6ab6b-80af-429f-8116-24323fc10458 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:08.611140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12d6ab6b-80af-429f-8116-24323fc10458 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:08.611852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12d6ab6b-80af-429f-8116-24323fc10458 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:08.612308 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12d6ab6b-80af-429f-8116-24323fc10458 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:08.612772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12d6ab6b-80af-429f-8116-24323fc10458 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:08.749683 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-12d6ab6b-80af-429f-8116-24323fc10458 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:08.758479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-12d6ab6b-80af-429f-8116-24323fc10458 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.161739 Aug 28 01:25:08.758936 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1599/2978] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:08 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:09.585004 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-23284263-2130-4645-bee8-9e108eb07e51 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:09.585627 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-23284263-2130-4645-bee8-9e108eb07e51 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:09.592687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23284263-2130-4645-bee8-9e108eb07e51 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:09.593227 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23284263-2130-4645-bee8-9e108eb07e51 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:09.593714 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23284263-2130-4645-bee8-9e108eb07e51 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:09.594369 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23284263-2130-4645-bee8-9e108eb07e51 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:09.594832 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23284263-2130-4645-bee8-9e108eb07e51 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:09.595243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23284263-2130-4645-bee8-9e108eb07e51 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:09.666012 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-23284263-2130-4645-bee8-9e108eb07e51 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:09.675495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-23284263-2130-4645-bee8-9e108eb07e51 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.095631 Aug 28 01:25:09.676090 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1380/2979] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:09 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:09.783191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cd28ef57-d7a4-4051-9e9b-f6929b230486 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:09.783977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cd28ef57-d7a4-4051-9e9b-f6929b230486 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:09.794546 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cd28ef57-d7a4-4051-9e9b-f6929b230486 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:09.795226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cd28ef57-d7a4-4051-9e9b-f6929b230486 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:09.795861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cd28ef57-d7a4-4051-9e9b-f6929b230486 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:09.796845 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cd28ef57-d7a4-4051-9e9b-f6929b230486 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:09.797555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cd28ef57-d7a4-4051-9e9b-f6929b230486 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:09.798213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cd28ef57-d7a4-4051-9e9b-f6929b230486 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:09.939975 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-cd28ef57-d7a4-4051-9e9b-f6929b230486 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:09.948876 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cd28ef57-d7a4-4051-9e9b-f6929b230486 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.171432 Aug 28 01:25:09.949476 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1600/2980] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:09 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:10.697958 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-72bacf80-55e8-4eaa-8cda-fecce2272425 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:10.698617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-72bacf80-55e8-4eaa-8cda-fecce2272425 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:10.706102 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-72bacf80-55e8-4eaa-8cda-fecce2272425 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:10.706596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-72bacf80-55e8-4eaa-8cda-fecce2272425 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:10.707034 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-72bacf80-55e8-4eaa-8cda-fecce2272425 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:10.707745 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-72bacf80-55e8-4eaa-8cda-fecce2272425 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:10.708196 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-72bacf80-55e8-4eaa-8cda-fecce2272425 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:10.708620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-72bacf80-55e8-4eaa-8cda-fecce2272425 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:10.772167 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-72bacf80-55e8-4eaa-8cda-fecce2272425 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:10.780936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-72bacf80-55e8-4eaa-8cda-fecce2272425 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.086931 Aug 28 01:25:10.781546 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1381/2981] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:10 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:10.977344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bc06f5f6-96ac-43d9-a49d-fd7650c4b809 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:10.977939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bc06f5f6-96ac-43d9-a49d-fd7650c4b809 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:10.985704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc06f5f6-96ac-43d9-a49d-fd7650c4b809 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:10.986349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc06f5f6-96ac-43d9-a49d-fd7650c4b809 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:10.986779 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc06f5f6-96ac-43d9-a49d-fd7650c4b809 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:10.987484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc06f5f6-96ac-43d9-a49d-fd7650c4b809 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:10.987919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc06f5f6-96ac-43d9-a49d-fd7650c4b809 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:10.988324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bc06f5f6-96ac-43d9-a49d-fd7650c4b809 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:11.365735 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bc06f5f6-96ac-43d9-a49d-fd7650c4b809 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:11.374525 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bc06f5f6-96ac-43d9-a49d-fd7650c4b809 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.401296 Aug 28 01:25:11.375095 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1601/2982] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:10 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:11.803371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ac938e27-c5e6-48a9-9104-40429ebeaf7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:11.804070 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ac938e27-c5e6-48a9-9104-40429ebeaf7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:11.811100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac938e27-c5e6-48a9-9104-40429ebeaf7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:11.811591 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac938e27-c5e6-48a9-9104-40429ebeaf7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:11.812023 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac938e27-c5e6-48a9-9104-40429ebeaf7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:11.812730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac938e27-c5e6-48a9-9104-40429ebeaf7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:11.813223 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac938e27-c5e6-48a9-9104-40429ebeaf7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:11.813644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ac938e27-c5e6-48a9-9104-40429ebeaf7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:11.878101 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ac938e27-c5e6-48a9-9104-40429ebeaf7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:11.887119 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ac938e27-c5e6-48a9-9104-40429ebeaf7e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.087691 Aug 28 01:25:11.887707 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1382/2983] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:11 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:12.396243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b0512dcd-d54f-4b88-9568-b0d7f4dae688 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:12.396859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b0512dcd-d54f-4b88-9568-b0d7f4dae688 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:12.407422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b0512dcd-d54f-4b88-9568-b0d7f4dae688 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:12.407914 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b0512dcd-d54f-4b88-9568-b0d7f4dae688 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:12.408352 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b0512dcd-d54f-4b88-9568-b0d7f4dae688 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:12.409554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b0512dcd-d54f-4b88-9568-b0d7f4dae688 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:12.409999 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b0512dcd-d54f-4b88-9568-b0d7f4dae688 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:12.410639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b0512dcd-d54f-4b88-9568-b0d7f4dae688 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:12.551263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b0512dcd-d54f-4b88-9568-b0d7f4dae688 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:12.560712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b0512dcd-d54f-4b88-9568-b0d7f4dae688 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.168519 Aug 28 01:25:12.561329 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1602/2984] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:12 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:12.908157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a24dad50-832a-4a94-851d-17676360274f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:12.908799 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a24dad50-832a-4a94-851d-17676360274f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:12.915838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a24dad50-832a-4a94-851d-17676360274f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:12.916340 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a24dad50-832a-4a94-851d-17676360274f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:12.916770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a24dad50-832a-4a94-851d-17676360274f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:12.917483 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a24dad50-832a-4a94-851d-17676360274f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:12.917958 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a24dad50-832a-4a94-851d-17676360274f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:12.918380 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a24dad50-832a-4a94-851d-17676360274f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:12.985996 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a24dad50-832a-4a94-851d-17676360274f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:12.994857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a24dad50-832a-4a94-851d-17676360274f tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.090599 Aug 28 01:25:12.995428 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1383/2985] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:12 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:13.583210 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4989fcc8-5f76-4e7d-a4e2-76a90ccdf57c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:13.583838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4989fcc8-5f76-4e7d-a4e2-76a90ccdf57c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:13.590910 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4989fcc8-5f76-4e7d-a4e2-76a90ccdf57c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:13.591409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4989fcc8-5f76-4e7d-a4e2-76a90ccdf57c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:13.591823 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4989fcc8-5f76-4e7d-a4e2-76a90ccdf57c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:13.592521 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4989fcc8-5f76-4e7d-a4e2-76a90ccdf57c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:13.592998 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4989fcc8-5f76-4e7d-a4e2-76a90ccdf57c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:13.593430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4989fcc8-5f76-4e7d-a4e2-76a90ccdf57c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:13.726803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4989fcc8-5f76-4e7d-a4e2-76a90ccdf57c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:13.735894 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4989fcc8-5f76-4e7d-a4e2-76a90ccdf57c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.156632 Aug 28 01:25:13.736515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1603/2986] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:13 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:14.016397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1f23989f-67c5-4f77-ade4-3a37ed72aa4a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:14.017010 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1f23989f-67c5-4f77-ade4-3a37ed72aa4a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:14.023877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1f23989f-67c5-4f77-ade4-3a37ed72aa4a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:14.024363 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1f23989f-67c5-4f77-ade4-3a37ed72aa4a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:14.024851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1f23989f-67c5-4f77-ade4-3a37ed72aa4a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:14.025625 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1f23989f-67c5-4f77-ade4-3a37ed72aa4a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:14.026075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1f23989f-67c5-4f77-ade4-3a37ed72aa4a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:14.026504 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1f23989f-67c5-4f77-ade4-3a37ed72aa4a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:14.089157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1f23989f-67c5-4f77-ade4-3a37ed72aa4a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:14.091453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:25:14.110556 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1f23989f-67c5-4f77-ade4-3a37ed72aa4a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 200 len: 1369 microversion: 2.1 time: 0.098094 Aug 28 01:25:14.111117 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1384/2987] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:14 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 1369 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:14.132689 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3ee8a28a-60ed-4373-a183-490ffcc28cb9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '.version_select at 0x7f0e6e85f550>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:14.155413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3ee8a28a-60ed-4373-a183-490ffcc28cb9 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1424971404" status: 202 len: 0 microversion: 2.1 time: 0.026523 Aug 28 01:25:14.156004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1604/2988] 10.222.0.42 () {58 vars in 1221 bytes} [Sun Aug 28 01:25:14 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1424971404 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 01:25:14.756589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c7ea81fd-e19c-4528-93ce-bd091d0eaaab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:14.757247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c7ea81fd-e19c-4528-93ce-bd091d0eaaab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:14.763285 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c7ea81fd-e19c-4528-93ce-bd091d0eaaab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:14.763771 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c7ea81fd-e19c-4528-93ce-bd091d0eaaab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:14.764197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c7ea81fd-e19c-4528-93ce-bd091d0eaaab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:14.764891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c7ea81fd-e19c-4528-93ce-bd091d0eaaab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:14.765387 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c7ea81fd-e19c-4528-93ce-bd091d0eaaab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:14.765804 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c7ea81fd-e19c-4528-93ce-bd091d0eaaab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:14.901307 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c7ea81fd-e19c-4528-93ce-bd091d0eaaab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:14.909758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c7ea81fd-e19c-4528-93ce-bd091d0eaaab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.157021 Aug 28 01:25:14.910333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1385/2989] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:14 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:14.976179 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bbc9df0e-0b5d-40ef-922a-7d29336d6200 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Action: 'create', calling method: .version_select at 0x7f0e6e8dc1f0>, body: {"keypair": {"name": "tempest-keypair-1248497303", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBN2HEOBa4FHit4ViYxjmh3b6u78vxYi88TZ+y1Oe/leOEhk6dKyXInbK/TrtUq7QvgBnNo7wxy+N+rMcHsA6aoe4TTJYCUzRWz3FbG6A4f/C0bU5mLxFkRCfpOlyrwzfEw=="}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:25:14.995409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-bbc9df0e-0b5d-40ef-922a-7d29336d6200 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Getting quotas for project 2348c1ddbec1476bacb194214794015d. Resources: {'key_pairs'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 01:25:14.999200 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-bbc9df0e-0b5d-40ef-922a-7d29336d6200 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Getting quotas for user 3de5d68f67784e33bdcd19a9cdfed744 and project 2348c1ddbec1476bacb194214794015d. Resources: {'key_pairs'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 01:25:15.021921 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bbc9df0e-0b5d-40ef-922a-7d29336d6200 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.050389 Aug 28 01:25:15.022594 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1605/2990] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Aug 28 01:25:14 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:25:15.932497 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b8edebaf-13c0-4930-bbbc-c1fa6c67a07b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:15.933240 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b8edebaf-13c0-4930-bbbc-c1fa6c67a07b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:15.940330 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b8edebaf-13c0-4930-bbbc-c1fa6c67a07b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:15.940993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b8edebaf-13c0-4930-bbbc-c1fa6c67a07b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:15.941555 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b8edebaf-13c0-4930-bbbc-c1fa6c67a07b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:15.942348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b8edebaf-13c0-4930-bbbc-c1fa6c67a07b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:15.942871 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b8edebaf-13c0-4930-bbbc-c1fa6c67a07b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:15.943389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b8edebaf-13c0-4930-bbbc-c1fa6c67a07b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:16.079132 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b8edebaf-13c0-4930-bbbc-c1fa6c67a07b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:16.087942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b8edebaf-13c0-4930-bbbc-c1fa6c67a07b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.160339 Aug 28 01:25:16.088515 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1386/2991] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:15 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:16.366530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f7a28965-eab7-408a-94db-7ae62a78f195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-519786080", "imageRef": "2c2585b2-bde5-4652-aed7-9f53b1bd5d94", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "9173759e-b079-4264-9516-c7ce5b7af7ce"}], "security_groups": [{"name": "tempest-securitygroup--157538649"}], "key_name": "tempest-keypair-1248497303", "user_data": "}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:25:16.477687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.network.neutron [None req-f7a28965-eab7-408a-94db-7ae62a78f195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] validate_networks() for [('9173759e-b079-4264-9516-c7ce5b7af7ce', None, None, None, None, None)] {{(pid=120525) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 28 01:25:16.883216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-f7a28965-eab7-408a-94db-7ae62a78f195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Flavor limits 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 28 01:25:16.883683 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-f7a28965-eab7-408a-94db-7ae62a78f195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Image limits 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 28 01:25:16.884161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-f7a28965-eab7-408a-94db-7ae62a78f195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Flavor pref 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 28 01:25:16.884559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-f7a28965-eab7-408a-94db-7ae62a78f195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Image pref 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 28 01:25:16.884980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-f7a28965-eab7-408a-94db-7ae62a78f195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 28 01:25:16.897747 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7a28965-eab7-408a-94db-7ae62a78f195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:16.898407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7a28965-eab7-408a-94db-7ae62a78f195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:16.899076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7a28965-eab7-408a-94db-7ae62a78f195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:16.900214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7a28965-eab7-408a-94db-7ae62a78f195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:16.900848 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7a28965-eab7-408a-94db-7ae62a78f195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:16.901634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7a28965-eab7-408a-94db-7ae62a78f195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:16.923801 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-f7a28965-eab7-408a-94db-7ae62a78f195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Getting quotas for project 2348c1ddbec1476bacb194214794015d. Resources: {'ram', 'instances', 'cores'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 01:25:16.927837 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-f7a28965-eab7-408a-94db-7ae62a78f195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Getting quotas for user 3de5d68f67784e33bdcd19a9cdfed744 and project 2348c1ddbec1476bacb194214794015d. Resources: {'ram', 'instances', 'cores'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 01:25:16.939681 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f7a28965-eab7-408a-94db-7ae62a78f195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Going to run 1 instances... {{(pid=120525) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 28 01:25:16.942048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f7a28965-eab7-408a-94db-7ae62a78f195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2c2585b2-bde5-4652-aed7-9f53b1bd5d94',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120525) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 28 01:25:17.002541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f7a28965-eab7-408a-94db-7ae62a78f195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.643359 Aug 28 01:25:17.003470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1606/2992] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Aug 28 01:25:16 2022] POST /compute/v2.1/servers => generated 401 bytes in 645 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 28 01:25:17.014513 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-11f4290c-2b10-4ce1-911b-e330067c71da tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:17.015222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-11f4290c-2b10-4ce1-911b-e330067c71da tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:17.074697 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-11f4290c-2b10-4ce1-911b-e330067c71da tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:17.084478 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-11f4290c-2b10-4ce1-911b-e330067c71da tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df" status: 200 len: 1237 microversion: 2.1 time: 0.073805 Aug 28 01:25:17.093082 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1387/2993] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:17 2022] GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df => generated 1237 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:17.110427 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-eb60dc46-e8f4-4b83-a890-806effa75930 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:17.111055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-eb60dc46-e8f4-4b83-a890-806effa75930 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:17.121071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb60dc46-e8f4-4b83-a890-806effa75930 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:17.121556 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb60dc46-e8f4-4b83-a890-806effa75930 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:17.121986 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb60dc46-e8f4-4b83-a890-806effa75930 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:17.122680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb60dc46-e8f4-4b83-a890-806effa75930 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:17.123122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb60dc46-e8f4-4b83-a890-806effa75930 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:17.123538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-eb60dc46-e8f4-4b83-a890-806effa75930 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:17.259048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-eb60dc46-e8f4-4b83-a890-806effa75930 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:17.267403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-eb60dc46-e8f4-4b83-a890-806effa75930 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.161615 Aug 28 01:25:17.267970 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1607/2994] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:17 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:18.105995 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f4036600-265b-49fa-b234-f23f8ef786f8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:18.106623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f4036600-265b-49fa-b234-f23f8ef786f8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:18.108922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=60 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:25:18.114165 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4036600-265b-49fa-b234-f23f8ef786f8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:18.114657 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4036600-265b-49fa-b234-f23f8ef786f8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:18.115063 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4036600-265b-49fa-b234-f23f8ef786f8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:18.115775 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4036600-265b-49fa-b234-f23f8ef786f8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:18.116218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4036600-265b-49fa-b234-f23f8ef786f8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:18.116635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f4036600-265b-49fa-b234-f23f8ef786f8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:18.185611 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f4036600-265b-49fa-b234-f23f8ef786f8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:18.194547 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f4036600-265b-49fa-b234-f23f8ef786f8 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df" status: 200 len: 1289 microversion: 2.1 time: 0.092495 Aug 28 01:25:18.195114 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1388/2995] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:18 2022] GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df => generated 1289 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:18.292386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3414f4b5-cbd5-4618-8990-70b9b29f5848 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:18.293696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3414f4b5-cbd5-4618-8990-70b9b29f5848 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:18.300213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3414f4b5-cbd5-4618-8990-70b9b29f5848 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:18.300698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3414f4b5-cbd5-4618-8990-70b9b29f5848 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:18.301160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3414f4b5-cbd5-4618-8990-70b9b29f5848 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:18.301850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3414f4b5-cbd5-4618-8990-70b9b29f5848 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:18.302296 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3414f4b5-cbd5-4618-8990-70b9b29f5848 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:18.302707 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3414f4b5-cbd5-4618-8990-70b9b29f5848 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:18.518591 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3414f4b5-cbd5-4618-8990-70b9b29f5848 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:18.527498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3414f4b5-cbd5-4618-8990-70b9b29f5848 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.240740 Aug 28 01:25:18.528072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1608/2996] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:18 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:19.216299 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ae9f4370-7066-4b45-a41a-53b081a9cc8b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:19.217648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ae9f4370-7066-4b45-a41a-53b081a9cc8b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:19.224336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae9f4370-7066-4b45-a41a-53b081a9cc8b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:19.224828 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae9f4370-7066-4b45-a41a-53b081a9cc8b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:19.225287 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae9f4370-7066-4b45-a41a-53b081a9cc8b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:19.225994 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae9f4370-7066-4b45-a41a-53b081a9cc8b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:19.226447 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae9f4370-7066-4b45-a41a-53b081a9cc8b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:19.226871 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ae9f4370-7066-4b45-a41a-53b081a9cc8b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:19.331077 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ae9f4370-7066-4b45-a41a-53b081a9cc8b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:19.339380 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ae9f4370-7066-4b45-a41a-53b081a9cc8b tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df" status: 200 len: 1295 microversion: 2.1 time: 0.127223 Aug 28 01:25:19.339955 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1389/2997] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:19 2022] GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df => generated 1295 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:19.548511 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1e0f04fa-6255-44a3-8ea1-ee442458cb9a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:19.549206 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1e0f04fa-6255-44a3-8ea1-ee442458cb9a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:19.555866 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1e0f04fa-6255-44a3-8ea1-ee442458cb9a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:19.556493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1e0f04fa-6255-44a3-8ea1-ee442458cb9a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:19.556923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1e0f04fa-6255-44a3-8ea1-ee442458cb9a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:19.557648 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1e0f04fa-6255-44a3-8ea1-ee442458cb9a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:19.558098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1e0f04fa-6255-44a3-8ea1-ee442458cb9a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:19.558520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1e0f04fa-6255-44a3-8ea1-ee442458cb9a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:19.752813 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1e0f04fa-6255-44a3-8ea1-ee442458cb9a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:19.768031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1e0f04fa-6255-44a3-8ea1-ee442458cb9a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.223257 Aug 28 01:25:19.768593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1609/2998] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:19 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:20.360264 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0d608e27-ae83-46b9-a19f-6b2642df361e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:20.360897 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0d608e27-ae83-46b9-a19f-6b2642df361e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:20.368558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0d608e27-ae83-46b9-a19f-6b2642df361e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:20.369158 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0d608e27-ae83-46b9-a19f-6b2642df361e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:20.369633 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0d608e27-ae83-46b9-a19f-6b2642df361e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:20.370438 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0d608e27-ae83-46b9-a19f-6b2642df361e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:20.370889 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0d608e27-ae83-46b9-a19f-6b2642df361e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:20.371313 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0d608e27-ae83-46b9-a19f-6b2642df361e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:20.465034 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-7d7ed503-8f30-463d-9abd-67c649414265 req-793dc04d-d18d-4161-8fe2-700e20cc7800 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "004da63e-f25e-4dcc-a519-d253e534d6df", "tag": "417c327c-0418-496e-87db-69ace013e663"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:25:20.474628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-7d7ed503-8f30-463d-9abd-67c649414265 req-793dc04d-d18d-4161-8fe2-700e20cc7800 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:20.475137 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-7d7ed503-8f30-463d-9abd-67c649414265 req-793dc04d-d18d-4161-8fe2-700e20cc7800 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:20.475528 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-7d7ed503-8f30-463d-9abd-67c649414265 req-793dc04d-d18d-4161-8fe2-700e20cc7800 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:20.486710 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-7d7ed503-8f30-463d-9abd-67c649414265 req-793dc04d-d18d-4161-8fe2-700e20cc7800 service nova] Creating event network-changed:417c327c-0418-496e-87db-69ace013e663 for instance 004da63e-f25e-4dcc-a519-d253e534d6df on hv-cinder-maste Aug 28 01:25:20.491931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-7d7ed503-8f30-463d-9abd-67c649414265 req-793dc04d-d18d-4161-8fe2-700e20cc7800 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.086797 Aug 28 01:25:20.492546 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1610/2999] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:25:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:25:20.545298 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0d608e27-ae83-46b9-a19f-6b2642df361e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:20.553120 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0d608e27-ae83-46b9-a19f-6b2642df361e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df" status: 200 len: 1362 microversion: 2.1 time: 0.196944 Aug 28 01:25:20.553690 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1390/3000] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:20 2022] GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df => generated 1362 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:20.792634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1fa6acda-9200-4624-9ba9-b2944d6c5122 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:20.793468 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1fa6acda-9200-4624-9ba9-b2944d6c5122 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:20.800032 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1fa6acda-9200-4624-9ba9-b2944d6c5122 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:20.800507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1fa6acda-9200-4624-9ba9-b2944d6c5122 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:20.800942 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1fa6acda-9200-4624-9ba9-b2944d6c5122 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:20.801898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1fa6acda-9200-4624-9ba9-b2944d6c5122 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:20.802361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1fa6acda-9200-4624-9ba9-b2944d6c5122 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:20.802781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1fa6acda-9200-4624-9ba9-b2944d6c5122 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:20.975915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1fa6acda-9200-4624-9ba9-b2944d6c5122 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:20.984855 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1fa6acda-9200-4624-9ba9-b2944d6c5122 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.197263 Aug 28 01:25:20.985484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1611/3001] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:20 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:21.575392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7ea8db4f-df1e-4a63-a03a-b33c83556a18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:21.576002 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7ea8db4f-df1e-4a63-a03a-b33c83556a18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:21.583182 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ea8db4f-df1e-4a63-a03a-b33c83556a18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:21.583660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ea8db4f-df1e-4a63-a03a-b33c83556a18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:21.584086 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ea8db4f-df1e-4a63-a03a-b33c83556a18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:21.584772 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ea8db4f-df1e-4a63-a03a-b33c83556a18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:21.585265 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ea8db4f-df1e-4a63-a03a-b33c83556a18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:21.585656 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ea8db4f-df1e-4a63-a03a-b33c83556a18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:21.731152 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7ea8db4f-df1e-4a63-a03a-b33c83556a18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:21.742701 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7ea8db4f-df1e-4a63-a03a-b33c83556a18 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df" status: 200 len: 1362 microversion: 2.1 time: 0.171112 Aug 28 01:25:21.743537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1391/3002] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:21 2022] GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df => generated 1362 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:22.007172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3e820f4d-1f58-4c5b-8752-bab1346e003b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:22.007831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3e820f4d-1f58-4c5b-8752-bab1346e003b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:22.015063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e820f4d-1f58-4c5b-8752-bab1346e003b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:22.015543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e820f4d-1f58-4c5b-8752-bab1346e003b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:22.015999 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e820f4d-1f58-4c5b-8752-bab1346e003b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:22.016679 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e820f4d-1f58-4c5b-8752-bab1346e003b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:22.017157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e820f4d-1f58-4c5b-8752-bab1346e003b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:22.017597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e820f4d-1f58-4c5b-8752-bab1346e003b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:22.156098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3e820f4d-1f58-4c5b-8752-bab1346e003b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:22.165545 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3e820f4d-1f58-4c5b-8752-bab1346e003b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.161866 Aug 28 01:25:22.166148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1612/3003] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:22 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:22.764302 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e1b75cdc-8b86-4dbb-817e-453a32445e6e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:22.764918 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e1b75cdc-8b86-4dbb-817e-453a32445e6e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:22.772300 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1b75cdc-8b86-4dbb-817e-453a32445e6e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:22.772859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1b75cdc-8b86-4dbb-817e-453a32445e6e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:22.773318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1b75cdc-8b86-4dbb-817e-453a32445e6e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:22.774036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1b75cdc-8b86-4dbb-817e-453a32445e6e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:22.774592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1b75cdc-8b86-4dbb-817e-453a32445e6e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:22.775091 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1b75cdc-8b86-4dbb-817e-453a32445e6e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:22.925166 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e1b75cdc-8b86-4dbb-817e-453a32445e6e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:22.934339 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e1b75cdc-8b86-4dbb-817e-453a32445e6e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df" status: 200 len: 1362 microversion: 2.1 time: 0.173899 Aug 28 01:25:22.934922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1392/3004] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:22 2022] GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df => generated 1362 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:23.193186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f0e00589-c7d1-4318-8a84-f8e22c75f717 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:23.194057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f0e00589-c7d1-4318-8a84-f8e22c75f717 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:23.203633 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0e00589-c7d1-4318-8a84-f8e22c75f717 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:23.204170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0e00589-c7d1-4318-8a84-f8e22c75f717 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:23.205024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0e00589-c7d1-4318-8a84-f8e22c75f717 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:23.205805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0e00589-c7d1-4318-8a84-f8e22c75f717 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:23.206386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0e00589-c7d1-4318-8a84-f8e22c75f717 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:23.206861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0e00589-c7d1-4318-8a84-f8e22c75f717 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:23.370973 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f0e00589-c7d1-4318-8a84-f8e22c75f717 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:23.384020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f0e00589-c7d1-4318-8a84-f8e22c75f717 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.194822 Aug 28 01:25:23.385053 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1613/3005] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:23 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:23.958890 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e1ee8039-c362-4882-8253-c018314aa0b2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:23.959549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e1ee8039-c362-4882-8253-c018314aa0b2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:23.966701 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1ee8039-c362-4882-8253-c018314aa0b2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:23.967191 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1ee8039-c362-4882-8253-c018314aa0b2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:23.967614 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1ee8039-c362-4882-8253-c018314aa0b2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:23.968308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1ee8039-c362-4882-8253-c018314aa0b2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:23.968753 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1ee8039-c362-4882-8253-c018314aa0b2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:23.969208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e1ee8039-c362-4882-8253-c018314aa0b2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:24.115039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e1ee8039-c362-4882-8253-c018314aa0b2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:24.123446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e1ee8039-c362-4882-8253-c018314aa0b2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df" status: 200 len: 1362 microversion: 2.1 time: 0.170526 Aug 28 01:25:24.124022 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1393/3006] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:23 2022] GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df => generated 1362 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:24.411601 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f043f420-8aeb-4ad0-b203-38c9b843c3ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:24.412263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f043f420-8aeb-4ad0-b203-38c9b843c3ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:24.419282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f043f420-8aeb-4ad0-b203-38c9b843c3ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:24.419795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f043f420-8aeb-4ad0-b203-38c9b843c3ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:24.420203 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f043f420-8aeb-4ad0-b203-38c9b843c3ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:24.420903 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f043f420-8aeb-4ad0-b203-38c9b843c3ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:24.421385 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f043f420-8aeb-4ad0-b203-38c9b843c3ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:24.421809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f043f420-8aeb-4ad0-b203-38c9b843c3ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:24.438134 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:25:24.562212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f043f420-8aeb-4ad0-b203-38c9b843c3ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:24.571460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f043f420-8aeb-4ad0-b203-38c9b843c3ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.163837 Aug 28 01:25:24.572127 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1614/3007] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:24 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:25.144072 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4127d9c9-dd24-43e6-860c-21df962b6fb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:25.144687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4127d9c9-dd24-43e6-860c-21df962b6fb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:25.151846 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4127d9c9-dd24-43e6-860c-21df962b6fb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:25.152322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4127d9c9-dd24-43e6-860c-21df962b6fb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:25.152752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4127d9c9-dd24-43e6-860c-21df962b6fb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:25.153504 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4127d9c9-dd24-43e6-860c-21df962b6fb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:25.153946 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4127d9c9-dd24-43e6-860c-21df962b6fb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:25.154364 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4127d9c9-dd24-43e6-860c-21df962b6fb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:25.295729 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4127d9c9-dd24-43e6-860c-21df962b6fb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:25.303945 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4127d9c9-dd24-43e6-860c-21df962b6fb6 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df" status: 200 len: 1362 microversion: 2.1 time: 0.163789 Aug 28 01:25:25.304786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1394/3008] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:25 2022] GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df => generated 1362 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:25.593680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c0fe087a-d32e-4939-9ed1-a7d1c5cc3b41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:25.594334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c0fe087a-d32e-4939-9ed1-a7d1c5cc3b41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:25.601269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0fe087a-d32e-4939-9ed1-a7d1c5cc3b41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:25.601747 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0fe087a-d32e-4939-9ed1-a7d1c5cc3b41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:25.602174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0fe087a-d32e-4939-9ed1-a7d1c5cc3b41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:25.602872 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0fe087a-d32e-4939-9ed1-a7d1c5cc3b41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:25.603319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0fe087a-d32e-4939-9ed1-a7d1c5cc3b41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:25.603810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c0fe087a-d32e-4939-9ed1-a7d1c5cc3b41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:25.739929 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c0fe087a-d32e-4939-9ed1-a7d1c5cc3b41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:25.752012 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c0fe087a-d32e-4939-9ed1-a7d1c5cc3b41 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.161963 Aug 28 01:25:25.752773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1615/3009] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:25 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:26.325515 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b451fbcf-0c56-4cf4-94d8-dfc2952ee195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:26.326768 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b451fbcf-0c56-4cf4-94d8-dfc2952ee195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:26.333999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b451fbcf-0c56-4cf4-94d8-dfc2952ee195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:26.334504 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b451fbcf-0c56-4cf4-94d8-dfc2952ee195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:26.334931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b451fbcf-0c56-4cf4-94d8-dfc2952ee195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:26.335664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b451fbcf-0c56-4cf4-94d8-dfc2952ee195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:26.336104 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b451fbcf-0c56-4cf4-94d8-dfc2952ee195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:26.336550 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b451fbcf-0c56-4cf4-94d8-dfc2952ee195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:26.476109 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b451fbcf-0c56-4cf4-94d8-dfc2952ee195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:26.484584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b451fbcf-0c56-4cf4-94d8-dfc2952ee195 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df" status: 200 len: 1362 microversion: 2.1 time: 0.162952 Aug 28 01:25:26.485446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1395/3010] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:26 2022] GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df => generated 1362 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:26.716989 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-5031b99f-fb9b-409e-909a-b5412de80261 req-2f9c3f5e-8174-4aa5-b391-3846bfb37683 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "004da63e-f25e-4dcc-a519-d253e534d6df", "name": "network-vif-plugged", "status": "completed", "tag": "417c327c-0418-496e-87db-69ace013e663"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:25:26.727630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-5031b99f-fb9b-409e-909a-b5412de80261 req-2f9c3f5e-8174-4aa5-b391-3846bfb37683 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:26.728109 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-5031b99f-fb9b-409e-909a-b5412de80261 req-2f9c3f5e-8174-4aa5-b391-3846bfb37683 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:26.728533 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-5031b99f-fb9b-409e-909a-b5412de80261 req-2f9c3f5e-8174-4aa5-b391-3846bfb37683 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:26.741479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-5031b99f-fb9b-409e-909a-b5412de80261 req-2f9c3f5e-8174-4aa5-b391-3846bfb37683 service nova] Creating event network-vif-plugged:417c327c-0418-496e-87db-69ace013e663 for instance 004da63e-f25e-4dcc-a519-d253e534d6df on hv-cinder-maste Aug 28 01:25:26.747509 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-5031b99f-fb9b-409e-909a-b5412de80261 req-2f9c3f5e-8174-4aa5-b391-3846bfb37683 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034553 Aug 28 01:25:26.748129 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1616/3011] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:25:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:25:26.774349 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c2e234df-ee18-4413-9acb-5c6a6bb61237 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:26.774993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c2e234df-ee18-4413-9acb-5c6a6bb61237 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:26.783249 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c2e234df-ee18-4413-9acb-5c6a6bb61237 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:26.783753 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c2e234df-ee18-4413-9acb-5c6a6bb61237 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:26.784295 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c2e234df-ee18-4413-9acb-5c6a6bb61237 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:26.785752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c2e234df-ee18-4413-9acb-5c6a6bb61237 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:26.786386 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c2e234df-ee18-4413-9acb-5c6a6bb61237 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:26.787356 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c2e234df-ee18-4413-9acb-5c6a6bb61237 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:26.944708 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c2e234df-ee18-4413-9acb-5c6a6bb61237 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:26.953566 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c2e234df-ee18-4413-9acb-5c6a6bb61237 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.183225 Aug 28 01:25:26.954174 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1396/3012] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:26 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:27.503499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=2 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:25:27.504299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=10 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:25:27.508503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7b527283-a247-4981-80bc-8b9c5bf9424c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:27.509232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7b527283-a247-4981-80bc-8b9c5bf9424c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:27.511225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=79,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:25:27.517523 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b527283-a247-4981-80bc-8b9c5bf9424c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:27.517993 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b527283-a247-4981-80bc-8b9c5bf9424c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:27.518437 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b527283-a247-4981-80bc-8b9c5bf9424c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:27.519124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b527283-a247-4981-80bc-8b9c5bf9424c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:27.519566 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b527283-a247-4981-80bc-8b9c5bf9424c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:27.519984 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7b527283-a247-4981-80bc-8b9c5bf9424c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:27.658058 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7b527283-a247-4981-80bc-8b9c5bf9424c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:27.666425 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7b527283-a247-4981-80bc-8b9c5bf9424c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df" status: 200 len: 1362 microversion: 2.1 time: 0.163718 Aug 28 01:25:27.667008 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1617/3013] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:27 2022] GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df => generated 1362 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:27.975493 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9dd329e4-c81a-4fe6-af68-3fbc987253fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:27.976150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9dd329e4-c81a-4fe6-af68-3fbc987253fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:27.978971 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=2 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:25:27.985488 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9dd329e4-c81a-4fe6-af68-3fbc987253fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:27.986109 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9dd329e4-c81a-4fe6-af68-3fbc987253fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:27.986543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9dd329e4-c81a-4fe6-af68-3fbc987253fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:27.987260 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9dd329e4-c81a-4fe6-af68-3fbc987253fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:27.987707 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9dd329e4-c81a-4fe6-af68-3fbc987253fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:27.988126 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9dd329e4-c81a-4fe6-af68-3fbc987253fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:28.123440 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9dd329e4-c81a-4fe6-af68-3fbc987253fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:28.132218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9dd329e4-c81a-4fe6-af68-3fbc987253fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.160714 Aug 28 01:25:28.132840 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1397/3014] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:27 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:28.694487 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c9c1c183-194e-4a01-bec1-199fc643c3a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:28.695122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c9c1c183-194e-4a01-bec1-199fc643c3a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:28.702144 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c9c1c183-194e-4a01-bec1-199fc643c3a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:28.702626 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c9c1c183-194e-4a01-bec1-199fc643c3a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:28.703062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c9c1c183-194e-4a01-bec1-199fc643c3a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:28.703765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c9c1c183-194e-4a01-bec1-199fc643c3a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:28.704216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c9c1c183-194e-4a01-bec1-199fc643c3a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:28.704641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c9c1c183-194e-4a01-bec1-199fc643c3a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:29.110390 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c9c1c183-194e-4a01-bec1-199fc643c3a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:29.121970 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c9c1c183-194e-4a01-bec1-199fc643c3a2 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df" status: 200 len: 1362 microversion: 2.1 time: 0.430928 Aug 28 01:25:29.122993 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1618/3015] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:28 2022] GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df => generated 1362 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:29.154754 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0eb9e894-00a6-4438-b1d7-1eed38c45c64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:29.155392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0eb9e894-00a6-4438-b1d7-1eed38c45c64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:29.162314 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0eb9e894-00a6-4438-b1d7-1eed38c45c64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:29.162805 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0eb9e894-00a6-4438-b1d7-1eed38c45c64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:29.163233 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0eb9e894-00a6-4438-b1d7-1eed38c45c64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:29.163933 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0eb9e894-00a6-4438-b1d7-1eed38c45c64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:29.164378 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0eb9e894-00a6-4438-b1d7-1eed38c45c64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:29.164796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0eb9e894-00a6-4438-b1d7-1eed38c45c64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:29.299137 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0eb9e894-00a6-4438-b1d7-1eed38c45c64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:29.311075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0eb9e894-00a6-4438-b1d7-1eed38c45c64 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.160060 Aug 28 01:25:29.311772 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1398/3016] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:29 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:30.144131 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f48f058e-54bc-4246-be74-15f5aed49d5d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:30.145102 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f48f058e-54bc-4246-be74-15f5aed49d5d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:30.151486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f48f058e-54bc-4246-be74-15f5aed49d5d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:30.151968 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f48f058e-54bc-4246-be74-15f5aed49d5d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:30.152393 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f48f058e-54bc-4246-be74-15f5aed49d5d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:30.153143 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f48f058e-54bc-4246-be74-15f5aed49d5d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:30.153574 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f48f058e-54bc-4246-be74-15f5aed49d5d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:30.153991 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f48f058e-54bc-4246-be74-15f5aed49d5d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:30.294481 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f48f058e-54bc-4246-be74-15f5aed49d5d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:30.302584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f48f058e-54bc-4246-be74-15f5aed49d5d tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df" status: 200 len: 1548 microversion: 2.1 time: 0.162534 Aug 28 01:25:30.303156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1619/3017] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:30 2022] GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df => generated 1548 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:30.323910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7b3e921d-533f-4763-819b-9e6a19f9129c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:30.324584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7b3e921d-533f-4763-819b-9e6a19f9129c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:30.332002 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7b3e921d-533f-4763-819b-9e6a19f9129c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:30.332508 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7b3e921d-533f-4763-819b-9e6a19f9129c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:30.332924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7b3e921d-533f-4763-819b-9e6a19f9129c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:30.333700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7b3e921d-533f-4763-819b-9e6a19f9129c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:30.334153 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7b3e921d-533f-4763-819b-9e6a19f9129c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:30.334574 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7b3e921d-533f-4763-819b-9e6a19f9129c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:30.337646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6420a0a9-8af1-4e61-a5ed-619aaa5f26c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:30.338700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6420a0a9-8af1-4e61-a5ed-619aaa5f26c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:30.348705 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6420a0a9-8af1-4e61-a5ed-619aaa5f26c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:30.349555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6420a0a9-8af1-4e61-a5ed-619aaa5f26c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:30.350050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6420a0a9-8af1-4e61-a5ed-619aaa5f26c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:30.350867 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6420a0a9-8af1-4e61-a5ed-619aaa5f26c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:30.351521 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6420a0a9-8af1-4e61-a5ed-619aaa5f26c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:30.352173 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6420a0a9-8af1-4e61-a5ed-619aaa5f26c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:30.443233 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7b3e921d-533f-4763-819b-9e6a19f9129c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.123410 Aug 28 01:25:30.443809 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1399/3018] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Aug 28 01:25:30 2022] GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df/os-interface => generated 285 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:25:30.523839 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6420a0a9-8af1-4e61-a5ed-619aaa5f26c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:30.539100 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6420a0a9-8af1-4e61-a5ed-619aaa5f26c5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.208388 Aug 28 01:25:30.540113 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1620/3019] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:30 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:31.568866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cdc07c95-e1a3-44dd-baab-39f66c9342b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:31.569740 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cdc07c95-e1a3-44dd-baab-39f66c9342b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:31.577582 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cdc07c95-e1a3-44dd-baab-39f66c9342b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:31.578404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cdc07c95-e1a3-44dd-baab-39f66c9342b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:31.579167 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cdc07c95-e1a3-44dd-baab-39f66c9342b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:31.580201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cdc07c95-e1a3-44dd-baab-39f66c9342b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:31.580814 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cdc07c95-e1a3-44dd-baab-39f66c9342b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:31.581334 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cdc07c95-e1a3-44dd-baab-39f66c9342b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:31.686090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-b371f22b-8b40-4213-9312-88d927781591 req-4a25d113-8d77-4731-bf55-b629f3517cd2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "004da63e-f25e-4dcc-a519-d253e534d6df", "tag": "417c327c-0418-496e-87db-69ace013e663"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:25:31.696905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-b371f22b-8b40-4213-9312-88d927781591 req-4a25d113-8d77-4731-bf55-b629f3517cd2 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:31.697406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-b371f22b-8b40-4213-9312-88d927781591 req-4a25d113-8d77-4731-bf55-b629f3517cd2 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:31.697832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-b371f22b-8b40-4213-9312-88d927781591 req-4a25d113-8d77-4731-bf55-b629f3517cd2 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:31.709803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-b371f22b-8b40-4213-9312-88d927781591 req-4a25d113-8d77-4731-bf55-b629f3517cd2 service nova] Creating event network-changed:417c327c-0418-496e-87db-69ace013e663 for instance 004da63e-f25e-4dcc-a519-d253e534d6df on hv-cinder-maste Aug 28 01:25:31.715764 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-b371f22b-8b40-4213-9312-88d927781591 req-4a25d113-8d77-4731-bf55-b629f3517cd2 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034471 Aug 28 01:25:31.716376 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1621/3020] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:25:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:25:31.740708 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-cdc07c95-e1a3-44dd-baab-39f66c9342b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:31.749367 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cdc07c95-e1a3-44dd-baab-39f66c9342b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.185351 Aug 28 01:25:31.749942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1400/3021] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:31 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:32.771687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c6bfba6d-fb7e-409f-9169-262e5d8d0178 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:32.772640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c6bfba6d-fb7e-409f-9169-262e5d8d0178 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:32.780448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6bfba6d-fb7e-409f-9169-262e5d8d0178 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:32.780915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6bfba6d-fb7e-409f-9169-262e5d8d0178 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:32.781424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6bfba6d-fb7e-409f-9169-262e5d8d0178 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:32.782107 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6bfba6d-fb7e-409f-9169-262e5d8d0178 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:32.782560 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6bfba6d-fb7e-409f-9169-262e5d8d0178 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:32.783014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6bfba6d-fb7e-409f-9169-262e5d8d0178 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:32.927088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c6bfba6d-fb7e-409f-9169-262e5d8d0178 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:32.936290 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c6bfba6d-fb7e-409f-9169-262e5d8d0178 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.168875 Aug 28 01:25:32.936919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1622/3022] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:32 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:33.958327 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2b50da54-7bb3-4beb-9dbf-00a7efd6f921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:33.958977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2b50da54-7bb3-4beb-9dbf-00a7efd6f921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:33.965907 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b50da54-7bb3-4beb-9dbf-00a7efd6f921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:33.966391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b50da54-7bb3-4beb-9dbf-00a7efd6f921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:33.966824 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b50da54-7bb3-4beb-9dbf-00a7efd6f921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:33.967521 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b50da54-7bb3-4beb-9dbf-00a7efd6f921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:33.967971 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b50da54-7bb3-4beb-9dbf-00a7efd6f921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:33.968382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b50da54-7bb3-4beb-9dbf-00a7efd6f921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:34.106423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2b50da54-7bb3-4beb-9dbf-00a7efd6f921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:34.115059 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2b50da54-7bb3-4beb-9dbf-00a7efd6f921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.160575 Aug 28 01:25:34.115637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1401/3023] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:33 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:35.137635 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-23c47c60-fc58-42ed-800d-f92759d007d9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:35.138427 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-23c47c60-fc58-42ed-800d-f92759d007d9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:35.145890 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23c47c60-fc58-42ed-800d-f92759d007d9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:35.146376 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23c47c60-fc58-42ed-800d-f92759d007d9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:35.146806 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23c47c60-fc58-42ed-800d-f92759d007d9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:35.147513 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23c47c60-fc58-42ed-800d-f92759d007d9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:35.147956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23c47c60-fc58-42ed-800d-f92759d007d9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:35.148378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-23c47c60-fc58-42ed-800d-f92759d007d9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:35.291889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-23c47c60-fc58-42ed-800d-f92759d007d9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:35.300644 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-23c47c60-fc58-42ed-800d-f92759d007d9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.167435 Aug 28 01:25:35.301259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1623/3024] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:35 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:36.324919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-57c1471c-bb8b-4e39-bbd2-30afe56a7fb1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:36.325635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-57c1471c-bb8b-4e39-bbd2-30afe56a7fb1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:36.332779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-57c1471c-bb8b-4e39-bbd2-30afe56a7fb1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:36.333278 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-57c1471c-bb8b-4e39-bbd2-30afe56a7fb1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:36.333704 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-57c1471c-bb8b-4e39-bbd2-30afe56a7fb1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:36.334383 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-57c1471c-bb8b-4e39-bbd2-30afe56a7fb1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:36.334840 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-57c1471c-bb8b-4e39-bbd2-30afe56a7fb1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:36.335261 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-57c1471c-bb8b-4e39-bbd2-30afe56a7fb1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:36.476796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-57c1471c-bb8b-4e39-bbd2-30afe56a7fb1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:36.479057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:25:36.488864 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-57c1471c-bb8b-4e39-bbd2-30afe56a7fb1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.169527 Aug 28 01:25:36.489449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1402/3025] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:36 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:37.510739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7de558f5-1027-417d-8c47-3500307602dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:37.511525 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7de558f5-1027-417d-8c47-3500307602dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:37.518748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7de558f5-1027-417d-8c47-3500307602dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:37.519241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7de558f5-1027-417d-8c47-3500307602dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:37.519675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7de558f5-1027-417d-8c47-3500307602dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:37.520378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7de558f5-1027-417d-8c47-3500307602dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:37.520822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7de558f5-1027-417d-8c47-3500307602dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:37.521269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7de558f5-1027-417d-8c47-3500307602dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:37.661331 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7de558f5-1027-417d-8c47-3500307602dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:37.669905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7de558f5-1027-417d-8c47-3500307602dd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.163090 Aug 28 01:25:37.670480 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1624/3026] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:37 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:38.692375 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-17f2363d-e6bd-4114-96c5-d392e5fecf5c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:38.692990 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-17f2363d-e6bd-4114-96c5-d392e5fecf5c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:38.700992 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-17f2363d-e6bd-4114-96c5-d392e5fecf5c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:38.701508 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-17f2363d-e6bd-4114-96c5-d392e5fecf5c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:38.701934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-17f2363d-e6bd-4114-96c5-d392e5fecf5c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:38.702633 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-17f2363d-e6bd-4114-96c5-d392e5fecf5c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:38.703083 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-17f2363d-e6bd-4114-96c5-d392e5fecf5c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:38.703499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-17f2363d-e6bd-4114-96c5-d392e5fecf5c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:38.842627 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-17f2363d-e6bd-4114-96c5-d392e5fecf5c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:38.851179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-17f2363d-e6bd-4114-96c5-d392e5fecf5c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.163398 Aug 28 01:25:38.851750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1403/3027] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:38 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:39.873891 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-94bef851-1b6e-420a-803d-6732ab86912d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:39.874486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-94bef851-1b6e-420a-803d-6732ab86912d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:39.881810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-94bef851-1b6e-420a-803d-6732ab86912d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:39.882309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-94bef851-1b6e-420a-803d-6732ab86912d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:39.882748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-94bef851-1b6e-420a-803d-6732ab86912d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:39.883502 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-94bef851-1b6e-420a-803d-6732ab86912d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:39.883954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-94bef851-1b6e-420a-803d-6732ab86912d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:39.884383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-94bef851-1b6e-420a-803d-6732ab86912d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:40.025762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-94bef851-1b6e-420a-803d-6732ab86912d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:40.034664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-94bef851-1b6e-420a-803d-6732ab86912d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.164812 Aug 28 01:25:40.035244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1625/3028] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:39 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:41.056915 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3f56c58a-7d03-416f-bbba-de2ab38c3a59 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:41.057656 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3f56c58a-7d03-416f-bbba-de2ab38c3a59 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:41.065178 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3f56c58a-7d03-416f-bbba-de2ab38c3a59 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:41.065645 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3f56c58a-7d03-416f-bbba-de2ab38c3a59 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:41.066089 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3f56c58a-7d03-416f-bbba-de2ab38c3a59 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:41.066780 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3f56c58a-7d03-416f-bbba-de2ab38c3a59 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:41.067230 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3f56c58a-7d03-416f-bbba-de2ab38c3a59 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:41.067665 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3f56c58a-7d03-416f-bbba-de2ab38c3a59 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:41.214054 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3f56c58a-7d03-416f-bbba-de2ab38c3a59 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:41.222949 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3f56c58a-7d03-416f-bbba-de2ab38c3a59 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.169918 Aug 28 01:25:41.223532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1404/3029] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:41 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:42.244917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c71b2772-e4de-4675-8c44-8657a7d4dbbe tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:42.245608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c71b2772-e4de-4675-8c44-8657a7d4dbbe tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:42.252315 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c71b2772-e4de-4675-8c44-8657a7d4dbbe tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:42.252809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c71b2772-e4de-4675-8c44-8657a7d4dbbe tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:42.253326 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c71b2772-e4de-4675-8c44-8657a7d4dbbe tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:42.254039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c71b2772-e4de-4675-8c44-8657a7d4dbbe tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:42.254466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c71b2772-e4de-4675-8c44-8657a7d4dbbe tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:42.254886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c71b2772-e4de-4675-8c44-8657a7d4dbbe tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:42.393681 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c71b2772-e4de-4675-8c44-8657a7d4dbbe tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:42.402310 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c71b2772-e4de-4675-8c44-8657a7d4dbbe tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.161378 Aug 28 01:25:42.402965 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1626/3030] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:42 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:43.425414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f0a4e64e-acbd-4fa3-9a76-033cc2525147 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:43.426000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f0a4e64e-acbd-4fa3-9a76-033cc2525147 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:43.432816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0a4e64e-acbd-4fa3-9a76-033cc2525147 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:43.433357 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0a4e64e-acbd-4fa3-9a76-033cc2525147 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:43.433786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0a4e64e-acbd-4fa3-9a76-033cc2525147 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:43.434575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0a4e64e-acbd-4fa3-9a76-033cc2525147 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:43.435002 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0a4e64e-acbd-4fa3-9a76-033cc2525147 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:43.435418 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0a4e64e-acbd-4fa3-9a76-033cc2525147 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:43.582449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f0a4e64e-acbd-4fa3-9a76-033cc2525147 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:43.591206 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f0a4e64e-acbd-4fa3-9a76-033cc2525147 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.169809 Aug 28 01:25:43.591793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1405/3031] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:43 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:44.737069 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c07b3f90-2c50-4787-873b-ebf2fa90d9ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:44.737947 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c07b3f90-2c50-4787-873b-ebf2fa90d9ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:44.744530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c07b3f90-2c50-4787-873b-ebf2fa90d9ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:44.745042 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c07b3f90-2c50-4787-873b-ebf2fa90d9ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:44.745480 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c07b3f90-2c50-4787-873b-ebf2fa90d9ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:44.746186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c07b3f90-2c50-4787-873b-ebf2fa90d9ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:44.746706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c07b3f90-2c50-4787-873b-ebf2fa90d9ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:44.747139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c07b3f90-2c50-4787-873b-ebf2fa90d9ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:44.891528 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c07b3f90-2c50-4787-873b-ebf2fa90d9ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:44.900492 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c07b3f90-2c50-4787-873b-ebf2fa90d9ad tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.290824 Aug 28 01:25:44.901117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1627/3032] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:44 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:45.935507 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b0ac4f74-3389-4690-b8b6-b8ace27cf43e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:45.936460 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b0ac4f74-3389-4690-b8b6-b8ace27cf43e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:45.943366 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0ac4f74-3389-4690-b8b6-b8ace27cf43e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:45.943935 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0ac4f74-3389-4690-b8b6-b8ace27cf43e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:45.944356 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0ac4f74-3389-4690-b8b6-b8ace27cf43e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:45.945197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0ac4f74-3389-4690-b8b6-b8ace27cf43e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:45.945659 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0ac4f74-3389-4690-b8b6-b8ace27cf43e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:45.946084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b0ac4f74-3389-4690-b8b6-b8ace27cf43e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:46.090568 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b0ac4f74-3389-4690-b8b6-b8ace27cf43e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:46.099784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b0ac4f74-3389-4690-b8b6-b8ace27cf43e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.168792 Aug 28 01:25:46.100376 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1406/3033] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:45 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:47.126523 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0734d439-1807-48d7-b658-74910a7ca6ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:47.127113 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0734d439-1807-48d7-b658-74910a7ca6ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:47.134054 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0734d439-1807-48d7-b658-74910a7ca6ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:47.134518 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0734d439-1807-48d7-b658-74910a7ca6ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:47.134953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0734d439-1807-48d7-b658-74910a7ca6ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:47.135662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0734d439-1807-48d7-b658-74910a7ca6ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:47.136114 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0734d439-1807-48d7-b658-74910a7ca6ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:47.136542 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0734d439-1807-48d7-b658-74910a7ca6ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:47.279575 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0734d439-1807-48d7-b658-74910a7ca6ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:47.291860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0734d439-1807-48d7-b658-74910a7ca6ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.174104 Aug 28 01:25:47.292473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1628/3034] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:47 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:25:48.314271 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ad6f680d-d6b1-43fb-8de4-01b0879eb95b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:48.314874 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ad6f680d-d6b1-43fb-8de4-01b0879eb95b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:48.317244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=38 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:25:48.323604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ad6f680d-d6b1-43fb-8de4-01b0879eb95b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:48.324087 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ad6f680d-d6b1-43fb-8de4-01b0879eb95b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:48.324639 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ad6f680d-d6b1-43fb-8de4-01b0879eb95b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:48.325362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ad6f680d-d6b1-43fb-8de4-01b0879eb95b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:48.325806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ad6f680d-d6b1-43fb-8de4-01b0879eb95b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:48.326207 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ad6f680d-d6b1-43fb-8de4-01b0879eb95b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:48.480895 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ad6f680d-d6b1-43fb-8de4-01b0879eb95b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:48.489999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ad6f680d-d6b1-43fb-8de4-01b0879eb95b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.180048 Aug 28 01:25:48.490585 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1407/3035] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:48 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:49.512176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-102b06ec-d172-4236-b067-782f080cc924 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:49.512985 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-102b06ec-d172-4236-b067-782f080cc924 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:49.519867 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-102b06ec-d172-4236-b067-782f080cc924 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:49.520353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-102b06ec-d172-4236-b067-782f080cc924 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:49.520782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-102b06ec-d172-4236-b067-782f080cc924 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:49.521512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-102b06ec-d172-4236-b067-782f080cc924 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:49.521927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-102b06ec-d172-4236-b067-782f080cc924 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:49.522375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-102b06ec-d172-4236-b067-782f080cc924 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:49.673203 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-102b06ec-d172-4236-b067-782f080cc924 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:49.689598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-102b06ec-d172-4236-b067-782f080cc924 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.180574 Aug 28 01:25:49.690630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1629/3036] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:49 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:50.725359 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-055433db-aed4-4d8a-a8e1-909b22c710e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:50.726310 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-055433db-aed4-4d8a-a8e1-909b22c710e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:50.737214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-055433db-aed4-4d8a-a8e1-909b22c710e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:50.737685 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-055433db-aed4-4d8a-a8e1-909b22c710e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:50.738140 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-055433db-aed4-4d8a-a8e1-909b22c710e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:50.738831 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-055433db-aed4-4d8a-a8e1-909b22c710e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:50.739309 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-055433db-aed4-4d8a-a8e1-909b22c710e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:50.739733 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-055433db-aed4-4d8a-a8e1-909b22c710e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:50.885086 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-055433db-aed4-4d8a-a8e1-909b22c710e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:50.894686 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-055433db-aed4-4d8a-a8e1-909b22c710e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.175478 Aug 28 01:25:50.895289 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1408/3037] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:50 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:51.917574 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3b2a4e83-b30d-444f-8ed2-20446636d447 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:51.918232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3b2a4e83-b30d-444f-8ed2-20446636d447 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:51.925275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3b2a4e83-b30d-444f-8ed2-20446636d447 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:51.925758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3b2a4e83-b30d-444f-8ed2-20446636d447 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:51.926192 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3b2a4e83-b30d-444f-8ed2-20446636d447 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:51.927514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3b2a4e83-b30d-444f-8ed2-20446636d447 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:51.927514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3b2a4e83-b30d-444f-8ed2-20446636d447 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:51.928281 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3b2a4e83-b30d-444f-8ed2-20446636d447 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:52.087455 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3b2a4e83-b30d-444f-8ed2-20446636d447 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:52.096512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3b2a4e83-b30d-444f-8ed2-20446636d447 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.182734 Aug 28 01:25:52.097214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1630/3038] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:51 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:52.602880 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-603ef97a-7c18-425f-ad49-6effeaf73747 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:52.603463 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-603ef97a-7c18-425f-ad49-6effeaf73747 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:52.609887 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-603ef97a-7c18-425f-ad49-6effeaf73747 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:52.610408 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-603ef97a-7c18-425f-ad49-6effeaf73747 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:52.610860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-603ef97a-7c18-425f-ad49-6effeaf73747 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:52.611561 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-603ef97a-7c18-425f-ad49-6effeaf73747 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:52.612010 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-603ef97a-7c18-425f-ad49-6effeaf73747 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:52.612429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-603ef97a-7c18-425f-ad49-6effeaf73747 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:52.639030 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-603ef97a-7c18-425f-ad49-6effeaf73747 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Going to try to terminate instance {{(pid=120524) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 28 01:25:52.725892 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-603ef97a-7c18-425f-ad49-6effeaf73747 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "DELETE /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc" status: 204 len: 0 microversion: 2.1 time: 0.242923 Aug 28 01:25:52.726712 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1409/3039] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 01:25:52 2022] DELETE /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc => generated 0 bytes in 246 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 28 01:25:52.737619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-460e7c79-6430-43d8-8ad5-12b8d58f32c0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:52.738232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-460e7c79-6430-43d8-8ad5-12b8d58f32c0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:52.745139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-460e7c79-6430-43d8-8ad5-12b8d58f32c0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:52.745642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-460e7c79-6430-43d8-8ad5-12b8d58f32c0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:52.746110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-460e7c79-6430-43d8-8ad5-12b8d58f32c0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:52.746914 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-460e7c79-6430-43d8-8ad5-12b8d58f32c0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:52.747418 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-460e7c79-6430-43d8-8ad5-12b8d58f32c0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:52.747867 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-460e7c79-6430-43d8-8ad5-12b8d58f32c0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:52.883997 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-460e7c79-6430-43d8-8ad5-12b8d58f32c0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:52.893339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-460e7c79-6430-43d8-8ad5-12b8d58f32c0 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc" status: 200 len: 1680 microversion: 2.1 time: 0.159990 Aug 28 01:25:52.893934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1631/3040] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:52 2022] GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc => generated 1680 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:53.120885 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-37bde28a-b3bf-4886-ac57-5616c2348ff3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:53.121522 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-37bde28a-b3bf-4886-ac57-5616c2348ff3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:53.128221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-37bde28a-b3bf-4886-ac57-5616c2348ff3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:53.128734 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-37bde28a-b3bf-4886-ac57-5616c2348ff3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:53.129197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-37bde28a-b3bf-4886-ac57-5616c2348ff3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:53.129912 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-37bde28a-b3bf-4886-ac57-5616c2348ff3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:53.130365 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-37bde28a-b3bf-4886-ac57-5616c2348ff3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:53.130796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-37bde28a-b3bf-4886-ac57-5616c2348ff3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:53.266647 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-37bde28a-b3bf-4886-ac57-5616c2348ff3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:53.275307 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-37bde28a-b3bf-4886-ac57-5616c2348ff3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.158888 Aug 28 01:25:53.275884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1410/3041] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:53 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:53.915900 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f373899a-a3be-4833-887e-fd48fca19f46 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:53.916528 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f373899a-a3be-4833-887e-fd48fca19f46 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:53.923405 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f373899a-a3be-4833-887e-fd48fca19f46 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:53.923884 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f373899a-a3be-4833-887e-fd48fca19f46 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:53.924308 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f373899a-a3be-4833-887e-fd48fca19f46 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:53.925005 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f373899a-a3be-4833-887e-fd48fca19f46 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:53.925458 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f373899a-a3be-4833-887e-fd48fca19f46 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:53.925855 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f373899a-a3be-4833-887e-fd48fca19f46 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:53.941862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:25:54.069440 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f373899a-a3be-4833-887e-fd48fca19f46 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:54.078521 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f373899a-a3be-4833-887e-fd48fca19f46 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc" status: 200 len: 1680 microversion: 2.1 time: 0.166686 Aug 28 01:25:54.079367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1632/3042] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:53 2022] GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc => generated 1680 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:54.298128 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fd1f4047-7c7a-4d91-adf3-9e2a4d636ecc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:54.298738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fd1f4047-7c7a-4d91-adf3-9e2a4d636ecc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:54.307488 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd1f4047-7c7a-4d91-adf3-9e2a4d636ecc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:54.308163 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd1f4047-7c7a-4d91-adf3-9e2a4d636ecc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:54.308782 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd1f4047-7c7a-4d91-adf3-9e2a4d636ecc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:54.309633 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd1f4047-7c7a-4d91-adf3-9e2a4d636ecc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:54.310236 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd1f4047-7c7a-4d91-adf3-9e2a4d636ecc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:54.310759 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fd1f4047-7c7a-4d91-adf3-9e2a4d636ecc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:54.459909 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fd1f4047-7c7a-4d91-adf3-9e2a4d636ecc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:54.469177 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fd1f4047-7c7a-4d91-adf3-9e2a4d636ecc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.175471 Aug 28 01:25:54.469767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1411/3043] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:54 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:55.106480 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6f0999ef-8598-48b0-8bba-980c9207261d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:55.107125 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6f0999ef-8598-48b0-8bba-980c9207261d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:55.120581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6f0999ef-8598-48b0-8bba-980c9207261d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:55.120734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6f0999ef-8598-48b0-8bba-980c9207261d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:55.121133 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6f0999ef-8598-48b0-8bba-980c9207261d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:55.122047 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6f0999ef-8598-48b0-8bba-980c9207261d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:55.122484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6f0999ef-8598-48b0-8bba-980c9207261d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:55.122925 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6f0999ef-8598-48b0-8bba-980c9207261d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:55.222263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6f0999ef-8598-48b0-8bba-980c9207261d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:55.230421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6f0999ef-8598-48b0-8bba-980c9207261d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc" status: 200 len: 1613 microversion: 2.1 time: 0.128377 Aug 28 01:25:55.231011 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1633/3044] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:55 2022] GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc => generated 1613 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:55.449260 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-d763a8bd-2a71-45c8-bcde-845a845fb995 req-fa58d3fa-cd8e-47d7-8b77-64bf0d72cc35 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2d365998-141e-4842-a129-31469afdb5cc", "name": "network-vif-deleted", "tag": "eef83555-5015-45c5-95ba-dac334fe4757"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:25:55.460555 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-d763a8bd-2a71-45c8-bcde-845a845fb995 req-fa58d3fa-cd8e-47d7-8b77-64bf0d72cc35 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:55.461057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-d763a8bd-2a71-45c8-bcde-845a845fb995 req-fa58d3fa-cd8e-47d7-8b77-64bf0d72cc35 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:55.461465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-d763a8bd-2a71-45c8-bcde-845a845fb995 req-fa58d3fa-cd8e-47d7-8b77-64bf0d72cc35 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:55.473659 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-d763a8bd-2a71-45c8-bcde-845a845fb995 req-fa58d3fa-cd8e-47d7-8b77-64bf0d72cc35 service nova] Creating event network-vif-deleted:eef83555-5015-45c5-95ba-dac334fe4757 for instance 2d365998-141e-4842-a129-31469afdb5cc on hv-cinder-maste Aug 28 01:25:55.479305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-d763a8bd-2a71-45c8-bcde-845a845fb995 req-fa58d3fa-cd8e-47d7-8b77-64bf0d72cc35 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034281 Aug 28 01:25:55.479985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1412/3045] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:25:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:25:55.490971 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a07edaf0-55e6-45e1-a14a-2d2b46caf897 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:55.492261 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a07edaf0-55e6-45e1-a14a-2d2b46caf897 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:55.498790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a07edaf0-55e6-45e1-a14a-2d2b46caf897 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:55.499279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a07edaf0-55e6-45e1-a14a-2d2b46caf897 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:55.499710 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a07edaf0-55e6-45e1-a14a-2d2b46caf897 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:55.500421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a07edaf0-55e6-45e1-a14a-2d2b46caf897 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:55.500945 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a07edaf0-55e6-45e1-a14a-2d2b46caf897 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:55.501415 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a07edaf0-55e6-45e1-a14a-2d2b46caf897 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:55.555752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8fe76351-7fa7-49fb-8767-5b1a421fafe8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:55.557007 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8fe76351-7fa7-49fb-8767-5b1a421fafe8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:55.572735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fe76351-7fa7-49fb-8767-5b1a421fafe8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:55.573251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fe76351-7fa7-49fb-8767-5b1a421fafe8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:55.574243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fe76351-7fa7-49fb-8767-5b1a421fafe8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:55.575097 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fe76351-7fa7-49fb-8767-5b1a421fafe8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:55.575758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fe76351-7fa7-49fb-8767-5b1a421fafe8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:55.576399 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fe76351-7fa7-49fb-8767-5b1a421fafe8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:55.604242 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8fe76351-7fa7-49fb-8767-5b1a421fafe8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Going to try to terminate instance {{(pid=120524) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 28 01:25:55.713386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a07edaf0-55e6-45e1-a14a-2d2b46caf897 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:55.732428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a07edaf0-55e6-45e1-a14a-2d2b46caf897 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.245442 Aug 28 01:25:55.734634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1634/3046] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:55 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:55.745824 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8fe76351-7fa7-49fb-8767-5b1a421fafe8 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "DELETE /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce" status: 204 len: 0 microversion: 2.20 time: 0.194117 Aug 28 01:25:55.746755 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1413/3047] 10.222.0.42 () {60 vars in 1280 bytes} [Sun Aug 28 01:25:55 2022] DELETE /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce => generated 0 bytes in 196 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Aug 28 01:25:55.756564 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f7b98dc1-1d01-400a-bffd-0000b01192fc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:55.757275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f7b98dc1-1d01-400a-bffd-0000b01192fc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:55.764296 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7b98dc1-1d01-400a-bffd-0000b01192fc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:55.764980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7b98dc1-1d01-400a-bffd-0000b01192fc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:55.765587 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7b98dc1-1d01-400a-bffd-0000b01192fc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:55.766365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7b98dc1-1d01-400a-bffd-0000b01192fc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:55.767273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7b98dc1-1d01-400a-bffd-0000b01192fc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:55.767863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f7b98dc1-1d01-400a-bffd-0000b01192fc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:55.905167 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f7b98dc1-1d01-400a-bffd-0000b01192fc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:55.913994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f7b98dc1-1d01-400a-bffd-0000b01192fc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:55.915466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f7b98dc1-1d01-400a-bffd-0000b01192fc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:55.916703 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f7b98dc1-1d01-400a-bffd-0000b01192fc tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce" status: 200 len: 1725 microversion: 2.20 time: 0.165731 Aug 28 01:25:55.917288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1635/3048] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:25:55 2022] GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce => generated 1725 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:25:56.253990 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5a944d34-3a08-4e97-a67a-324510ce576f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:56.255113 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5a944d34-3a08-4e97-a67a-324510ce576f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:56.261647 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a944d34-3a08-4e97-a67a-324510ce576f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:56.262126 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a944d34-3a08-4e97-a67a-324510ce576f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:56.262858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a944d34-3a08-4e97-a67a-324510ce576f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:56.263559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a944d34-3a08-4e97-a67a-324510ce576f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:56.264200 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a944d34-3a08-4e97-a67a-324510ce576f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:56.264414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a944d34-3a08-4e97-a67a-324510ce576f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:56.297566 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a944d34-3a08-4e97-a67a-324510ce576f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:56.298046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a944d34-3a08-4e97-a67a-324510ce576f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:56.298468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5a944d34-3a08-4e97-a67a-324510ce576f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:56.298938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.objects.instance [None req-5a944d34-3a08-4e97-a67a-324510ce576f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lazy-loading 'fault' on Instance uuid 2d365998-141e-4842-a129-31469afdb5cc {{(pid=120524) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:25:56.342012 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5a944d34-3a08-4e97-a67a-324510ce576f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:56.352623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5a944d34-3a08-4e97-a67a-324510ce576f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc" status: 200 len: 1331 microversion: 2.1 time: 0.104239 Aug 28 01:25:56.353307 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1414/3049] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:56 2022] GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc => generated 1331 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:25:56.756062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a6b183da-c6de-4787-89ed-58b045d758d4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:56.756688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a6b183da-c6de-4787-89ed-58b045d758d4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:56.763733 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6b183da-c6de-4787-89ed-58b045d758d4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:56.764222 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6b183da-c6de-4787-89ed-58b045d758d4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:56.764658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6b183da-c6de-4787-89ed-58b045d758d4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:56.765442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6b183da-c6de-4787-89ed-58b045d758d4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:56.765892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6b183da-c6de-4787-89ed-58b045d758d4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:56.766312 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a6b183da-c6de-4787-89ed-58b045d758d4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:56.906496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a6b183da-c6de-4787-89ed-58b045d758d4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:56.915575 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a6b183da-c6de-4787-89ed-58b045d758d4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.163172 Aug 28 01:25:56.916155 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1636/3050] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:56 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:56.940589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bae7a160-b575-4065-b650-3ddd0c555509 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:56.941254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bae7a160-b575-4065-b650-3ddd0c555509 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:56.947967 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bae7a160-b575-4065-b650-3ddd0c555509 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:56.948452 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bae7a160-b575-4065-b650-3ddd0c555509 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:56.948885 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bae7a160-b575-4065-b650-3ddd0c555509 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:56.949618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bae7a160-b575-4065-b650-3ddd0c555509 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:56.950191 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bae7a160-b575-4065-b650-3ddd0c555509 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:56.950596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bae7a160-b575-4065-b650-3ddd0c555509 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:57.097930 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bae7a160-b575-4065-b650-3ddd0c555509 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:57.112611 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bae7a160-b575-4065-b650-3ddd0c555509 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:57.114657 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bae7a160-b575-4065-b650-3ddd0c555509 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:57.115877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bae7a160-b575-4065-b650-3ddd0c555509 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce" status: 200 len: 1725 microversion: 2.20 time: 0.179911 Aug 28 01:25:57.116896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1415/3051] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:25:56 2022] GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce => generated 1725 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:25:57.374381 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-30a4d781-a145-49b0-bfd7-02714f1ca81b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:57.374892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-30a4d781-a145-49b0-bfd7-02714f1ca81b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:57.381235 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-30a4d781-a145-49b0-bfd7-02714f1ca81b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:57.381720 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-30a4d781-a145-49b0-bfd7-02714f1ca81b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:57.382148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-30a4d781-a145-49b0-bfd7-02714f1ca81b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:57.382838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-30a4d781-a145-49b0-bfd7-02714f1ca81b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:57.383284 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-30a4d781-a145-49b0-bfd7-02714f1ca81b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:57.383699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-30a4d781-a145-49b0-bfd7-02714f1ca81b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:57.416349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-30a4d781-a145-49b0-bfd7-02714f1ca81b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:57.416809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-30a4d781-a145-49b0-bfd7-02714f1ca81b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:57.417281 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-30a4d781-a145-49b0-bfd7-02714f1ca81b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:57.417718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.objects.instance [None req-30a4d781-a145-49b0-bfd7-02714f1ca81b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lazy-loading 'fault' on Instance uuid 2d365998-141e-4842-a129-31469afdb5cc {{(pid=120525) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:25:57.460198 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-30a4d781-a145-49b0-bfd7-02714f1ca81b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:57.470293 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-30a4d781-a145-49b0-bfd7-02714f1ca81b tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc" status: 200 len: 1331 microversion: 2.1 time: 0.100018 Aug 28 01:25:57.470872 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1637/3052] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:57 2022] GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc => generated 1331 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:57.939611 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-27f7dd9b-6cd4-4974-bffb-fb5437c690df tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:57.940227 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-27f7dd9b-6cd4-4974-bffb-fb5437c690df tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:57.947277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-27f7dd9b-6cd4-4974-bffb-fb5437c690df tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:57.947973 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-27f7dd9b-6cd4-4974-bffb-fb5437c690df tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:57.948156 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-27f7dd9b-6cd4-4974-bffb-fb5437c690df tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:57.948849 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-27f7dd9b-6cd4-4974-bffb-fb5437c690df tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:57.949321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-27f7dd9b-6cd4-4974-bffb-fb5437c690df tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:57.949747 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-27f7dd9b-6cd4-4974-bffb-fb5437c690df tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:57.966362 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:25:58.140550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-dba4c07c-d822-4a79-b0b9-214f0fc11754 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:58.142664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-dba4c07c-d822-4a79-b0b9-214f0fc11754 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:58.145339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=44 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:25:58.151127 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dba4c07c-d822-4a79-b0b9-214f0fc11754 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:58.151613 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dba4c07c-d822-4a79-b0b9-214f0fc11754 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:58.152336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dba4c07c-d822-4a79-b0b9-214f0fc11754 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:58.153257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dba4c07c-d822-4a79-b0b9-214f0fc11754 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:58.153805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dba4c07c-d822-4a79-b0b9-214f0fc11754 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:58.154400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dba4c07c-d822-4a79-b0b9-214f0fc11754 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:58.212237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-27f7dd9b-6cd4-4974-bffb-fb5437c690df tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:58.220769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-27f7dd9b-6cd4-4974-bffb-fb5437c690df tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.285456 Aug 28 01:25:58.221384 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1416/3053] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:57 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:25:58.391180 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dba4c07c-d822-4a79-b0b9-214f0fc11754 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:58.400430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dba4c07c-d822-4a79-b0b9-214f0fc11754 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:58.401941 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dba4c07c-d822-4a79-b0b9-214f0fc11754 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:58.403417 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-dba4c07c-d822-4a79-b0b9-214f0fc11754 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce" status: 200 len: 1725 microversion: 2.20 time: 0.267379 Aug 28 01:25:58.417197 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1638/3054] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:25:58 2022] GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce => generated 1725 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:25:58.494171 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f89d4d5e-da3d-45bd-ad83-5169114c8f03 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:58.495519 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f89d4d5e-da3d-45bd-ad83-5169114c8f03 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:58.505957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f89d4d5e-da3d-45bd-ad83-5169114c8f03 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:58.506543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f89d4d5e-da3d-45bd-ad83-5169114c8f03 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:58.507046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f89d4d5e-da3d-45bd-ad83-5169114c8f03 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:58.507799 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f89d4d5e-da3d-45bd-ad83-5169114c8f03 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:58.508850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f89d4d5e-da3d-45bd-ad83-5169114c8f03 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:58.509279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f89d4d5e-da3d-45bd-ad83-5169114c8f03 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:58.524454 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-f89d4d5e-da3d-45bd-ad83-5169114c8f03 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] HTTP exception thrown: Instance 2d365998-141e-4842-a129-31469afdb5cc could not be found. Aug 28 01:25:58.526252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f89d4d5e-da3d-45bd-ad83-5169114c8f03 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Request method failure captured: Aug 28 01:25:58.526252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc HTTP/1.1 Aug 28 01:25:58.526252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:25:58.526252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:25:58.526252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:25:58.526252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:25:58.526252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:25:58.526252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:25:58.526252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:25:58.526252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:25:58.526252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCrzduxHoOEhvI8P-Mde9cL1oxkmc4fMm9FmGsHxLSNCBjncK-ctZ1RF74JsMq7QMGP1Qh6DECNOIg5DjWuRWqO-CpXkeFE6YBKwQT8SoQT23SwG4DtFzbguCYZajUyKFut9Dn943LOjL5V3-apkT_NK3VlDjeI9Rm1KYQSsNW0BOiQg Aug 28 01:25:58.526252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:25:58.526252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:25:58.526252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:25:58.526252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:25:58.526252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:25:58.526252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:25:58.526252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:25:58.526252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:25:58.526252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:25:58.526252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:25:58.529989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_95df81e14a8549fca0851234f4a00b89", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/95df81e14a8549fca0851234f4a00b89"}]}] Aug 28 01:25:58.529989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:25:58.529989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:25:58.529989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:25:58.529989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:25:58.529989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:25:58.529989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:25:58.529989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: d209174d0db04b3584556824c6d0a3d4 Aug 28 01:25:58.529989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:25:58.529989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:25:58.529989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Instance 2d365998-141e-4842-a129-31469afdb5cc could not be found. Aug 28 01:25:58.529989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'id': '2d365998-141e-4842-a129-31469afdb5cc'} Aug 28 01:25:58.532936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:25:58.532936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:25:58.532936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:25:58.532936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:25:58.532936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:25:58.532936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:25:58.532936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:25:58.532936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:25:58.532936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:25:58.532936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:25:58.532936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2d365998-141e-4842-a129-31469afdb5cc could not be found. Aug 28 01:25:58.532936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:25:58.532936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:25:58.532936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:25:58.532936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:25:58.532936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:25:58.532936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:25:58.532936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:25:58.532936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:25:58.532936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:25:58.540046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:25:58.540046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 01:25:58.540046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 01:25:58.540046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:25:58.540046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:25:58.540046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:25:58.540046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:25:58.540046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2d365998-141e-4842-a129-31469afdb5cc could not be found. Aug 28 01:25:58.540046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:25:58.540046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:25:58.540046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:25:58.540046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:25:58.540046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:25:58.540046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:25:58.540046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:25:58.540046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:25:58.540046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2d365998-141e-4842-a129-31469afdb5cc could not be found. Aug 28 01:25:58.540046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:25:58.545203 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f89d4d5e-da3d-45bd-ad83-5169114c8f03 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Returning 404 to user: Instance 2d365998-141e-4842-a129-31469afdb5cc could not be found. {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:25:58.545203 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f89d4d5e-da3d-45bd-ad83-5169114c8f03 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc" status: 404 len: 111 microversion: 2.1 time: 0.040466 Aug 28 01:25:58.545203 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1417/3055] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:58 2022] GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:25:58.551470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a5cd0b9f-98f0-4eaa-b73e-82109d7a552a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '.version_select at 0x7f0e6e95d700>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:58.579618 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a5cd0b9f-98f0-4eaa-b73e-82109d7a552a tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-515734887" status: 202 len: 0 microversion: 2.1 time: 0.044027 Aug 28 01:25:58.580647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1639/3056] 10.222.0.42 () {58 vars in 1218 bytes} [Sun Aug 28 01:25:58 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-515734887 => generated 0 bytes in 47 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 01:25:59.125178 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-3e8757ff-6dcc-4818-8653-fda8e798c61f req-a0fad91a-f093-479e-a075-2fc6c9b163fc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "81d7ea67-14fe-4038-8519-dd1f56d18cce", "name": "network-vif-deleted", "tag": "84a74b5e-4dfc-48a0-ba25-a1e19ee5ff83"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:25:59.142855 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-3e8757ff-6dcc-4818-8653-fda8e798c61f req-a0fad91a-f093-479e-a075-2fc6c9b163fc service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:59.145287 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-3e8757ff-6dcc-4818-8653-fda8e798c61f req-a0fad91a-f093-479e-a075-2fc6c9b163fc service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:59.145890 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-3e8757ff-6dcc-4818-8653-fda8e798c61f req-a0fad91a-f093-479e-a075-2fc6c9b163fc service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:59.159257 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-3e8757ff-6dcc-4818-8653-fda8e798c61f req-a0fad91a-f093-479e-a075-2fc6c9b163fc service nova] Creating event network-vif-deleted:84a74b5e-4dfc-48a0-ba25-a1e19ee5ff83 for instance 81d7ea67-14fe-4038-8519-dd1f56d18cce on hv-cinder-maste Aug 28 01:25:59.165033 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-3e8757ff-6dcc-4818-8653-fda8e798c61f req-a0fad91a-f093-479e-a075-2fc6c9b163fc service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043616 Aug 28 01:25:59.166000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1418/3057] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:25:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:25:59.248457 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-230de6b2-ef35-4935-ba0c-0eed8876415d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:59.249286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-230de6b2-ef35-4935-ba0c-0eed8876415d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:59.263367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-230de6b2-ef35-4935-ba0c-0eed8876415d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:59.264715 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-230de6b2-ef35-4935-ba0c-0eed8876415d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:59.265558 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-230de6b2-ef35-4935-ba0c-0eed8876415d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:59.266634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-230de6b2-ef35-4935-ba0c-0eed8876415d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:59.267102 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-230de6b2-ef35-4935-ba0c-0eed8876415d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:59.267596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-230de6b2-ef35-4935-ba0c-0eed8876415d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:59.322717 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d39bb21c-7df3-4131-8a61-7f19f078d482 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Action: 'create', calling method: .version_select at 0x7f0e6e7c40d0>, body: {"keypair": {"name": "tempest-keypair-221832629", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPl3LdeAXYgStyA6hg5eZb2p7KPLrCop2kFeSGRvZj+RX+qQMi5e99PxxjnJS938umCGjmnK395dig2rBKIvq8pHwgIszwqtB/8jZV1gB17U7b1CU3ImxmR8Kp9VuXIhwQ=="}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:25:59.369270 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-d39bb21c-7df3-4131-8a61-7f19f078d482 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Getting quotas for project 95df81e14a8549fca0851234f4a00b89. Resources: {'key_pairs'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 01:25:59.374133 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-d39bb21c-7df3-4131-8a61-7f19f078d482 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Getting quotas for user d209174d0db04b3584556824c6d0a3d4 and project 95df81e14a8549fca0851234f4a00b89. Resources: {'key_pairs'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 01:25:59.414145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d39bb21c-7df3-4131-8a61-7f19f078d482 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.096321 Aug 28 01:25:59.423822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1419/3058] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Aug 28 01:25:59 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:25:59.460223 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7e31e521-8f75-42fb-a994-6de672ec76ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:25:59.461104 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7e31e521-8f75-42fb-a994-6de672ec76ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:25:59.472429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e31e521-8f75-42fb-a994-6de672ec76ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:59.473008 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e31e521-8f75-42fb-a994-6de672ec76ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:59.473424 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e31e521-8f75-42fb-a994-6de672ec76ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:59.474178 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e31e521-8f75-42fb-a994-6de672ec76ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:59.474605 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e31e521-8f75-42fb-a994-6de672ec76ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:59.475023 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e31e521-8f75-42fb-a994-6de672ec76ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:59.497554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-230de6b2-ef35-4935-ba0c-0eed8876415d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:59.508157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-230de6b2-ef35-4935-ba0c-0eed8876415d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.268657 Aug 28 01:25:59.521795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1640/3059] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:25:59 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:25:59.545591 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e31e521-8f75-42fb-a994-6de672ec76ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:25:59.546231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e31e521-8f75-42fb-a994-6de672ec76ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:25:59.546765 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e31e521-8f75-42fb-a994-6de672ec76ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:25:59.547328 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.objects.instance [None req-7e31e521-8f75-42fb-a994-6de672ec76ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lazy-loading 'fault' on Instance uuid 81d7ea67-14fe-4038-8519-dd1f56d18cce {{(pid=120524) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:25:59.617036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7e31e521-8f75-42fb-a994-6de672ec76ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:59.643225 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7e31e521-8f75-42fb-a994-6de672ec76ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:59.648202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7e31e521-8f75-42fb-a994-6de672ec76ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:25:59.650596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7e31e521-8f75-42fb-a994-6de672ec76ab tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce" status: 200 len: 1372 microversion: 2.20 time: 0.216018 Aug 28 01:25:59.651359 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1420/3060] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:25:59 2022] GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce => generated 1372 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:26:00.531648 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b946f1af-1fd5-417c-8238-df949df7d957 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:00.532269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b946f1af-1fd5-417c-8238-df949df7d957 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:00.539774 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b946f1af-1fd5-417c-8238-df949df7d957 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:00.540403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b946f1af-1fd5-417c-8238-df949df7d957 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:00.540877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b946f1af-1fd5-417c-8238-df949df7d957 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:00.541701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b946f1af-1fd5-417c-8238-df949df7d957 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:00.542155 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b946f1af-1fd5-417c-8238-df949df7d957 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:00.542581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b946f1af-1fd5-417c-8238-df949df7d957 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:00.677421 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-52618009-dd65-4854-86af-2cafc8fa55e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:00.678068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-52618009-dd65-4854-86af-2cafc8fa55e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:00.684447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b946f1af-1fd5-417c-8238-df949df7d957 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:00.691601 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-52618009-dd65-4854-86af-2cafc8fa55e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:00.692094 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-52618009-dd65-4854-86af-2cafc8fa55e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:00.692638 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-52618009-dd65-4854-86af-2cafc8fa55e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:00.693386 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-52618009-dd65-4854-86af-2cafc8fa55e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:00.694043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-52618009-dd65-4854-86af-2cafc8fa55e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:00.694360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b946f1af-1fd5-417c-8238-df949df7d957 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.166373 Aug 28 01:26:00.694532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-52618009-dd65-4854-86af-2cafc8fa55e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:00.695343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1641/3061] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:00 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:00.709646 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-52618009-dd65-4854-86af-2cafc8fa55e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] HTTP exception thrown: Instance 81d7ea67-14fe-4038-8519-dd1f56d18cce could not be found. Aug 28 01:26:00.711664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-52618009-dd65-4854-86af-2cafc8fa55e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Request method failure captured: Aug 28 01:26:00.711664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce HTTP/1.1 Aug 28 01:26:00.711664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:26:00.711664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:26:00.711664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:26:00.711664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:26:00.711664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:26:00.711664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:26:00.711664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:26:00.711664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:26:00.711664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCrzjmVndE3J0T1bKuUY8Ipz6UdCdHTla_ZBX1kD9euBtohEB_ihuwpRHdKiiS67obfjsh2qj6v6fsgqePClTJKl_tEuVTP07R5_yii_rH6-mTIVEymXqmkZjC4lF12x3ZpjUNycUL0XFzkb5MxCQ1wXhvXTMZ4cpU8MxkUTfk5aOilw Aug 28 01:26:00.711664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:26:00.711664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:26:00.711664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:26:00.711664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:26:00.711664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Openstack-Nova-Api-Version: 2.20 Aug 28 01:26:00.711664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:26:00.711664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:26:00.711664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 19f069b9ad7a4a5984fa947ad755db0a Aug 28 01:26:00.711664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-349481608 Aug 28 01:26:00.711664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:26:00.711664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:26:00.713644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_19f069b9ad7a4a5984fa947ad755db0a", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/19f069b9ad7a4a5984fa947ad755db0a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/19f069b9ad7a4a5984fa947ad755db0a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/19f069b9ad7a4a5984fa947ad755db0a"}]}] Aug 28 01:26:00.713644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-AttachVolumeShelveTestJSON-349481608 Aug 28 01:26:00.713644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 19f069b9ad7a4a5984fa947ad755db0a Aug 28 01:26:00.713644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-349481608 Aug 28 01:26:00.713644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-AttachVolumeShelveTestJSON-349481608-project Aug 28 01:26:00.713644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:26:00.713644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:26:00.713644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: 4894ef8b630e4bc084799c26096f7c66 Aug 28 01:26:00.713644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-AttachVolumeShelveTestJSON-349481608-project Aug 28 01:26:00.713644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:26:00.713644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Instance 81d7ea67-14fe-4038-8519-dd1f56d18cce could not be found. Aug 28 01:26:00.713644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'id': '81d7ea67-14fe-4038-8519-dd1f56d18cce'} Aug 28 01:26:00.715185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:26:00.715185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:26:00.715185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:26:00.715185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:26:00.715185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:26:00.715185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:26:00.715185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:26:00.715185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:26:00.715185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:26:00.715185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:26:00.715185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 81d7ea67-14fe-4038-8519-dd1f56d18cce could not be found. Aug 28 01:26:00.715185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:26:00.715185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:26:00.715185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:26:00.715185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:26:00.715185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:26:00.715185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:26:00.715185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:26:00.715185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:26:00.715185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:26:00.716858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:26:00.716858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 01:26:00.716858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 01:26:00.716858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:26:00.716858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:26:00.716858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:26:00.716858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:26:00.716858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 81d7ea67-14fe-4038-8519-dd1f56d18cce could not be found. Aug 28 01:26:00.716858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:26:00.716858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:26:00.716858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:26:00.716858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:26:00.716858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:26:00.716858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:26:00.716858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:26:00.716858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:26:00.716858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 81d7ea67-14fe-4038-8519-dd1f56d18cce could not be found. Aug 28 01:26:00.716858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:26:00.719458 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-52618009-dd65-4854-86af-2cafc8fa55e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Returning 404 to user: Instance 81d7ea67-14fe-4038-8519-dd1f56d18cce could not be found. {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:26:00.720812 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-52618009-dd65-4854-86af-2cafc8fa55e1 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce" status: 404 len: 111 microversion: 2.20 time: 0.047155 Aug 28 01:26:00.724975 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1421/3062] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:26:00 2022] GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 28 01:26:00.730411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2d0a4eeb-c1d7-4d38-9b8c-ec2b230a66a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '.version_select at 0x7f0e72016af0>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:00.753584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2d0a4eeb-c1d7-4d38-9b8c-ec2b230a66a7 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1126382633" status: 204 len: 0 microversion: 2.20 time: 0.026967 Aug 28 01:26:00.757033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1642/3063] 10.222.0.42 () {60 vars in 1262 bytes} [Sun Aug 28 01:26:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1126382633 => generated 0 bytes in 29 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Aug 28 01:26:00.905201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7ddf691b-8b74-4fcf-a415-6a80753ea4ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1921768038", "imageRef": "2c2585b2-bde5-4652-aed7-9f53b1bd5d94", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "e3dfdb02-2ba5-4b22-9a96-a212c53e9d60"}], "security_groups": [{"name": "tempest-securitygroup--587863520"}], "key_name": "tempest-keypair-221832629", "user_data": "}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:26:01.023611 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.network.neutron [None req-7ddf691b-8b74-4fcf-a415-6a80753ea4ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] validate_networks() for [('e3dfdb02-2ba5-4b22-9a96-a212c53e9d60', None, None, None, None, None)] {{(pid=120524) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 28 01:26:01.428126 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-7ddf691b-8b74-4fcf-a415-6a80753ea4ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Flavor limits 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 28 01:26:01.428558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-7ddf691b-8b74-4fcf-a415-6a80753ea4ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Image limits 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 28 01:26:01.429068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-7ddf691b-8b74-4fcf-a415-6a80753ea4ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Flavor pref 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 28 01:26:01.429485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-7ddf691b-8b74-4fcf-a415-6a80753ea4ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Image pref 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 28 01:26:01.429927 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-7ddf691b-8b74-4fcf-a415-6a80753ea4ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 28 01:26:01.440287 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ddf691b-8b74-4fcf-a415-6a80753ea4ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:01.440786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ddf691b-8b74-4fcf-a415-6a80753ea4ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:01.441256 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ddf691b-8b74-4fcf-a415-6a80753ea4ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:01.442285 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ddf691b-8b74-4fcf-a415-6a80753ea4ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:01.442997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ddf691b-8b74-4fcf-a415-6a80753ea4ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:01.443607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ddf691b-8b74-4fcf-a415-6a80753ea4ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:01.453905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b67d168d-b3f0-4901-8c32-b709f369bc4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:01.454403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b67d168d-b3f0-4901-8c32-b709f369bc4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:01.460365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b67d168d-b3f0-4901-8c32-b709f369bc4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:01.461095 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b67d168d-b3f0-4901-8c32-b709f369bc4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:01.461526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b67d168d-b3f0-4901-8c32-b709f369bc4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:01.461672 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-7ddf691b-8b74-4fcf-a415-6a80753ea4ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Getting quotas for project 95df81e14a8549fca0851234f4a00b89. Resources: {'ram', 'instances', 'cores'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 01:26:01.462216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b67d168d-b3f0-4901-8c32-b709f369bc4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:01.462794 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b67d168d-b3f0-4901-8c32-b709f369bc4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:01.463077 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b67d168d-b3f0-4901-8c32-b709f369bc4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:01.465746 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-7ddf691b-8b74-4fcf-a415-6a80753ea4ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Getting quotas for user d209174d0db04b3584556824c6d0a3d4 and project 95df81e14a8549fca0851234f4a00b89. Resources: {'ram', 'instances', 'cores'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 01:26:01.476716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-b67d168d-b3f0-4901-8c32-b709f369bc4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] HTTP exception thrown: Instance 81d7ea67-14fe-4038-8519-dd1f56d18cce could not be found. Aug 28 01:26:01.476942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7ddf691b-8b74-4fcf-a415-6a80753ea4ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Going to run 1 instances... {{(pid=120524) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 28 01:26:01.478436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b67d168d-b3f0-4901-8c32-b709f369bc4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Request method failure captured: Aug 28 01:26:01.478436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: DELETE /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce HTTP/1.1 Aug 28 01:26:01.478436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:26:01.478436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:26:01.478436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:26:01.478436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:26:01.478436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:26:01.478436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:26:01.478436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:26:01.478436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:26:01.478436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCrzjmVndE3J0T1bKuUY8Ipz6UdCdHTla_ZBX1kD9euBtohEB_ihuwpRHdKiiS67obfjsh2qj6v6fsgqePClTJKl_tEuVTP07R5_yii_rH6-mTIVEymXqmkZjC4lF12x3ZpjUNycUL0XFzkb5MxCQ1wXhvXTMZ4cpU8MxkUTfk5aOilw Aug 28 01:26:01.478436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:26:01.478436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:26:01.478436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:26:01.478436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:26:01.478436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Openstack-Nova-Api-Version: 2.20 Aug 28 01:26:01.478436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:26:01.478436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:26:01.478436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 19f069b9ad7a4a5984fa947ad755db0a Aug 28 01:26:01.478436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-349481608 Aug 28 01:26:01.478436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:26:01.478436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:26:01.480370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_19f069b9ad7a4a5984fa947ad755db0a", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/19f069b9ad7a4a5984fa947ad755db0a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/19f069b9ad7a4a5984fa947ad755db0a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/19f069b9ad7a4a5984fa947ad755db0a"}]}] Aug 28 01:26:01.480370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-AttachVolumeShelveTestJSON-349481608 Aug 28 01:26:01.480370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 19f069b9ad7a4a5984fa947ad755db0a Aug 28 01:26:01.480370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-349481608 Aug 28 01:26:01.480370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-AttachVolumeShelveTestJSON-349481608-project Aug 28 01:26:01.480370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:26:01.480370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:26:01.480370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: 4894ef8b630e4bc084799c26096f7c66 Aug 28 01:26:01.480370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-AttachVolumeShelveTestJSON-349481608-project Aug 28 01:26:01.480370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 01:26:01.480370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Instance 81d7ea67-14fe-4038-8519-dd1f56d18cce could not be found. Aug 28 01:26:01.480370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'id': '81d7ea67-14fe-4038-8519-dd1f56d18cce'} Aug 28 01:26:01.481353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:26:01.481353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:26:01.481353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:26:01.481353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:26:01.481353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:26:01.481353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:26:01.481353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:26:01.481353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:26:01.481353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:26:01.481353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:26:01.481353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 81d7ea67-14fe-4038-8519-dd1f56d18cce could not be found. Aug 28 01:26:01.481353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:26:01.481353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:26:01.481353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:26:01.481353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:26:01.481353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:26:01.481353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:26:01.481353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:26:01.481353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:26:01.481353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:26:01.482911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:26:01.482911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 28 01:26:01.482911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 28 01:26:01.482911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 28 01:26:01.482911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 28 01:26:01.482911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:26:01.482911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:26:01.482911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:26:01.482911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:26:01.482911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 81d7ea67-14fe-4038-8519-dd1f56d18cce could not be found. Aug 28 01:26:01.482911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:26:01.482911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:26:01.482911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:26:01.482911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:26:01.482911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:26:01.482911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:26:01.482911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:26:01.482911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:26:01.482911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 81d7ea67-14fe-4038-8519-dd1f56d18cce could not be found. Aug 28 01:26:01.482911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:26:01.484459 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b67d168d-b3f0-4901-8c32-b709f369bc4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Returning 404 to user: Instance 81d7ea67-14fe-4038-8519-dd1f56d18cce could not be found. {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:26:01.484568 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7ddf691b-8b74-4fcf-a415-6a80753ea4ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2c2585b2-bde5-4652-aed7-9f53b1bd5d94',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120524) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 28 01:26:01.484695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b67d168d-b3f0-4901-8c32-b709f369bc4d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "DELETE /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce" status: 404 len: 111 microversion: 2.20 time: 0.030052 Aug 28 01:26:01.484695 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1643/3064] 10.222.0.42 () {60 vars in 1280 bytes} [Sun Aug 28 01:26:01 2022] DELETE /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 28 01:26:01.488214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-54d23baf-e159-4848-9ede-33c4f59aeb04 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:01.488734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-54d23baf-e159-4848-9ede-33c4f59aeb04 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: 81d7ea67-14fe-4038-8519-dd1f56d18cce] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:01.494659 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54d23baf-e159-4848-9ede-33c4f59aeb04 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:01.495137 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54d23baf-e159-4848-9ede-33c4f59aeb04 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:01.495568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54d23baf-e159-4848-9ede-33c4f59aeb04 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:01.496241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54d23baf-e159-4848-9ede-33c4f59aeb04 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:01.496697 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54d23baf-e159-4848-9ede-33c4f59aeb04 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:01.497148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-54d23baf-e159-4848-9ede-33c4f59aeb04 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:01.514667 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-54d23baf-e159-4848-9ede-33c4f59aeb04 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] HTTP exception thrown: Instance 81d7ea67-14fe-4038-8519-dd1f56d18cce could not be found. Aug 28 01:26:01.516686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-54d23baf-e159-4848-9ede-33c4f59aeb04 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Request method failure captured: Aug 28 01:26:01.516686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce HTTP/1.1 Aug 28 01:26:01.516686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:26:01.516686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:26:01.516686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:26:01.516686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:26:01.516686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:26:01.516686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:26:01.516686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:26:01.516686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:26:01.516686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCrzjmVndE3J0T1bKuUY8Ipz6UdCdHTla_ZBX1kD9euBtohEB_ihuwpRHdKiiS67obfjsh2qj6v6fsgqePClTJKl_tEuVTP07R5_yii_rH6-mTIVEymXqmkZjC4lF12x3ZpjUNycUL0XFzkb5MxCQ1wXhvXTMZ4cpU8MxkUTfk5aOilw Aug 28 01:26:01.516686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:26:01.516686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:26:01.516686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:26:01.516686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:26:01.516686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Openstack-Nova-Api-Version: 2.20 Aug 28 01:26:01.516686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:26:01.516686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:26:01.516686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 19f069b9ad7a4a5984fa947ad755db0a Aug 28 01:26:01.516686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-349481608 Aug 28 01:26:01.516686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:26:01.516686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:26:01.518641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_19f069b9ad7a4a5984fa947ad755db0a", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/19f069b9ad7a4a5984fa947ad755db0a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/19f069b9ad7a4a5984fa947ad755db0a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/19f069b9ad7a4a5984fa947ad755db0a"}]}] Aug 28 01:26:01.518641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-AttachVolumeShelveTestJSON-349481608 Aug 28 01:26:01.518641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 19f069b9ad7a4a5984fa947ad755db0a Aug 28 01:26:01.518641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-349481608 Aug 28 01:26:01.518641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-AttachVolumeShelveTestJSON-349481608-project Aug 28 01:26:01.518641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:26:01.518641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:26:01.518641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: 4894ef8b630e4bc084799c26096f7c66 Aug 28 01:26:01.518641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-AttachVolumeShelveTestJSON-349481608-project Aug 28 01:26:01.518641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 01:26:01.518641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Instance 81d7ea67-14fe-4038-8519-dd1f56d18cce could not be found. Aug 28 01:26:01.518641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'id': '81d7ea67-14fe-4038-8519-dd1f56d18cce'} Aug 28 01:26:01.519637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:26:01.519637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:26:01.519637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:26:01.519637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:26:01.519637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:26:01.519637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:26:01.519637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:26:01.519637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:26:01.519637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:26:01.519637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:26:01.519637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 81d7ea67-14fe-4038-8519-dd1f56d18cce could not be found. Aug 28 01:26:01.519637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:26:01.519637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:26:01.519637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:26:01.519637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:26:01.519637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:26:01.519637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:26:01.519637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:26:01.519637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:26:01.519637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:26:01.521168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:26:01.521168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 01:26:01.521168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 01:26:01.521168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:26:01.521168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:26:01.521168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:26:01.521168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:26:01.521168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 81d7ea67-14fe-4038-8519-dd1f56d18cce could not be found. Aug 28 01:26:01.521168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:26:01.521168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:26:01.521168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:26:01.521168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:26:01.521168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:26:01.521168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:26:01.521168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:26:01.521168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:26:01.521168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 81d7ea67-14fe-4038-8519-dd1f56d18cce could not be found. Aug 28 01:26:01.521168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:26:01.522988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-54d23baf-e159-4848-9ede-33c4f59aeb04 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Returning 404 to user: Instance 81d7ea67-14fe-4038-8519-dd1f56d18cce could not be found. {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:26:01.522988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-54d23baf-e159-4848-9ede-33c4f59aeb04 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce" status: 404 len: 111 microversion: 2.20 time: 0.033611 Aug 28 01:26:01.522988 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1644/3065] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:26:01 2022] GET /compute/v2.1/servers/81d7ea67-14fe-4038-8519-dd1f56d18cce => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 28 01:26:01.530238 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7ddf691b-8b74-4fcf-a415-6a80753ea4ca tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.629913 Aug 28 01:26:01.531063 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1422/3066] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Aug 28 01:26:00 2022] POST /compute/v2.1/servers => generated 401 bytes in 632 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 28 01:26:01.542020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c7de72cf-757d-4f8b-a157-e70cadfd53c8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:01.542687 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c7de72cf-757d-4f8b-a157-e70cadfd53c8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:01.600355 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c7de72cf-757d-4f8b-a157-e70cadfd53c8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:01.607635 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c7de72cf-757d-4f8b-a157-e70cadfd53c8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2" status: 200 len: 1241 microversion: 2.1 time: 0.069376 Aug 28 01:26:01.608224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1645/3067] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:01 2022] GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2 => generated 1241 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:01.715784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-931fb122-bd79-4e33-9303-bcd15204a8a2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:01.716369 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-931fb122-bd79-4e33-9303-bcd15204a8a2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:01.724498 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-931fb122-bd79-4e33-9303-bcd15204a8a2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:01.725025 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-931fb122-bd79-4e33-9303-bcd15204a8a2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:01.725470 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-931fb122-bd79-4e33-9303-bcd15204a8a2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:01.726194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-931fb122-bd79-4e33-9303-bcd15204a8a2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:01.726885 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-931fb122-bd79-4e33-9303-bcd15204a8a2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:01.727317 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-931fb122-bd79-4e33-9303-bcd15204a8a2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:01.870045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-931fb122-bd79-4e33-9303-bcd15204a8a2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:01.879079 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-931fb122-bd79-4e33-9303-bcd15204a8a2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.167196 Aug 28 01:26:01.879777 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1423/3068] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:01 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:02.628800 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ea31ceed-8eb9-4dcd-b7a3-870ef1237a49 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:02.629479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ea31ceed-8eb9-4dcd-b7a3-870ef1237a49 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:02.637100 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea31ceed-8eb9-4dcd-b7a3-870ef1237a49 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:02.637625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea31ceed-8eb9-4dcd-b7a3-870ef1237a49 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:02.638056 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea31ceed-8eb9-4dcd-b7a3-870ef1237a49 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:02.638818 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea31ceed-8eb9-4dcd-b7a3-870ef1237a49 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:02.639259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea31ceed-8eb9-4dcd-b7a3-870ef1237a49 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:02.639671 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ea31ceed-8eb9-4dcd-b7a3-870ef1237a49 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:02.705699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ea31ceed-8eb9-4dcd-b7a3-870ef1237a49 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:02.713767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ea31ceed-8eb9-4dcd-b7a3-870ef1237a49 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2" status: 200 len: 1293 microversion: 2.1 time: 0.088864 Aug 28 01:26:02.714338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1646/3069] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:02 2022] GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2 => generated 1293 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:02.902339 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9da19f35-ee1f-410e-9ef8-fde0ebc45856 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:02.904045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9da19f35-ee1f-410e-9ef8-fde0ebc45856 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:02.910868 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9da19f35-ee1f-410e-9ef8-fde0ebc45856 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:02.911352 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9da19f35-ee1f-410e-9ef8-fde0ebc45856 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:02.911784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9da19f35-ee1f-410e-9ef8-fde0ebc45856 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:02.912480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9da19f35-ee1f-410e-9ef8-fde0ebc45856 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:02.912927 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9da19f35-ee1f-410e-9ef8-fde0ebc45856 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:02.913371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9da19f35-ee1f-410e-9ef8-fde0ebc45856 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:03.131949 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9da19f35-ee1f-410e-9ef8-fde0ebc45856 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:03.142173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9da19f35-ee1f-410e-9ef8-fde0ebc45856 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.244884 Aug 28 01:26:03.142737 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1424/3070] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:02 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:03.735750 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-083ef739-4f55-4954-bdd4-12dbb63b6441 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:03.736422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-083ef739-4f55-4954-bdd4-12dbb63b6441 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:03.743091 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-083ef739-4f55-4954-bdd4-12dbb63b6441 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:03.743578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-083ef739-4f55-4954-bdd4-12dbb63b6441 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:03.744011 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-083ef739-4f55-4954-bdd4-12dbb63b6441 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:03.744698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-083ef739-4f55-4954-bdd4-12dbb63b6441 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:03.745186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-083ef739-4f55-4954-bdd4-12dbb63b6441 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:03.745603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-083ef739-4f55-4954-bdd4-12dbb63b6441 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:03.977635 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-083ef739-4f55-4954-bdd4-12dbb63b6441 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:03.986273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-083ef739-4f55-4954-bdd4-12dbb63b6441 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2" status: 200 len: 1366 microversion: 2.1 time: 0.254504 Aug 28 01:26:03.986861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1647/3071] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:03 2022] GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2 => generated 1366 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:04.164012 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-241091f4-6e45-4867-8d99-b6d67006f59d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:04.164688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-241091f4-6e45-4867-8d99-b6d67006f59d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:04.171638 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-241091f4-6e45-4867-8d99-b6d67006f59d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:04.172226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-241091f4-6e45-4867-8d99-b6d67006f59d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:04.172672 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-241091f4-6e45-4867-8d99-b6d67006f59d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:04.173328 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-241091f4-6e45-4867-8d99-b6d67006f59d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:04.173980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-241091f4-6e45-4867-8d99-b6d67006f59d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:04.174449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-241091f4-6e45-4867-8d99-b6d67006f59d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:04.357930 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-241091f4-6e45-4867-8d99-b6d67006f59d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:04.366821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-241091f4-6e45-4867-8d99-b6d67006f59d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.207028 Aug 28 01:26:04.367396 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1425/3072] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:04 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:04.656652 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-b7ccb29d-5c27-4a14-87d7-720b3a575f5f req-2fad9aab-3a3a-4b84-a4c5-f424fd7b75ce service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9d45bf39-4417-4af3-869d-9761326e9eb2", "tag": "9e2ae2aa-ec13-4a1a-a862-13d3bf69d93a"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:26:04.666682 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-b7ccb29d-5c27-4a14-87d7-720b3a575f5f req-2fad9aab-3a3a-4b84-a4c5-f424fd7b75ce service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:04.667217 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-b7ccb29d-5c27-4a14-87d7-720b3a575f5f req-2fad9aab-3a3a-4b84-a4c5-f424fd7b75ce service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:04.667694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-b7ccb29d-5c27-4a14-87d7-720b3a575f5f req-2fad9aab-3a3a-4b84-a4c5-f424fd7b75ce service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:04.679434 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-b7ccb29d-5c27-4a14-87d7-720b3a575f5f req-2fad9aab-3a3a-4b84-a4c5-f424fd7b75ce service nova] Creating event network-changed:9e2ae2aa-ec13-4a1a-a862-13d3bf69d93a for instance 9d45bf39-4417-4af3-869d-9761326e9eb2 on hv-cinder-maste Aug 28 01:26:04.684410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-b7ccb29d-5c27-4a14-87d7-720b3a575f5f req-2fad9aab-3a3a-4b84-a4c5-f424fd7b75ce service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032179 Aug 28 01:26:04.685047 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1648/3073] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:26:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:26:05.007689 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4c998763-1977-4cbc-b9df-b789164370d9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:05.008297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4c998763-1977-4cbc-b9df-b789164370d9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:05.016083 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c998763-1977-4cbc-b9df-b789164370d9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:05.016762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c998763-1977-4cbc-b9df-b789164370d9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:05.017823 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c998763-1977-4cbc-b9df-b789164370d9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:05.018933 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c998763-1977-4cbc-b9df-b789164370d9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:05.019711 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c998763-1977-4cbc-b9df-b789164370d9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:05.020620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c998763-1977-4cbc-b9df-b789164370d9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:05.237668 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4c998763-1977-4cbc-b9df-b789164370d9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:05.246023 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4c998763-1977-4cbc-b9df-b789164370d9 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2" status: 200 len: 1366 microversion: 2.1 time: 0.242711 Aug 28 01:26:05.246688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1426/3074] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:05 2022] GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2 => generated 1366 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:05.391567 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f125f6c1-50f9-49a8-bacc-503198993b99 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:05.392258 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f125f6c1-50f9-49a8-bacc-503198993b99 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:05.399037 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f125f6c1-50f9-49a8-bacc-503198993b99 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:05.400530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f125f6c1-50f9-49a8-bacc-503198993b99 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:05.400530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f125f6c1-50f9-49a8-bacc-503198993b99 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:05.400821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f125f6c1-50f9-49a8-bacc-503198993b99 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:05.401099 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f125f6c1-50f9-49a8-bacc-503198993b99 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:05.401517 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f125f6c1-50f9-49a8-bacc-503198993b99 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:05.548594 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f125f6c1-50f9-49a8-bacc-503198993b99 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:05.561349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f125f6c1-50f9-49a8-bacc-503198993b99 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.174866 Aug 28 01:26:05.562008 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1649/3075] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:05 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:06.268893 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1df9d468-689e-4573-a243-63aed96da386 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:06.269594 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1df9d468-689e-4573-a243-63aed96da386 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:06.276801 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1df9d468-689e-4573-a243-63aed96da386 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:06.277313 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1df9d468-689e-4573-a243-63aed96da386 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:06.277748 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1df9d468-689e-4573-a243-63aed96da386 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:06.278449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1df9d468-689e-4573-a243-63aed96da386 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:06.278894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1df9d468-689e-4573-a243-63aed96da386 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:06.279316 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1df9d468-689e-4573-a243-63aed96da386 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:06.414246 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1df9d468-689e-4573-a243-63aed96da386 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:06.426488 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1df9d468-689e-4573-a243-63aed96da386 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2" status: 200 len: 1366 microversion: 2.1 time: 0.161277 Aug 28 01:26:06.426878 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1427/3076] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:06 2022] GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2 => generated 1366 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:06.585667 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ab598f58-00e3-44db-ac67-c6be3586fa20 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:06.586392 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ab598f58-00e3-44db-ac67-c6be3586fa20 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:06.595435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab598f58-00e3-44db-ac67-c6be3586fa20 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:06.595912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab598f58-00e3-44db-ac67-c6be3586fa20 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:06.596340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab598f58-00e3-44db-ac67-c6be3586fa20 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:06.597098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab598f58-00e3-44db-ac67-c6be3586fa20 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:06.597496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab598f58-00e3-44db-ac67-c6be3586fa20 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:06.597912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ab598f58-00e3-44db-ac67-c6be3586fa20 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:06.738286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ab598f58-00e3-44db-ac67-c6be3586fa20 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:06.747057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ab598f58-00e3-44db-ac67-c6be3586fa20 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.167362 Aug 28 01:26:06.747630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1650/3077] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:06 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:07.450652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2ebc3b40-fc93-4e77-8647-67f322901de3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:07.452445 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2ebc3b40-fc93-4e77-8647-67f322901de3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:07.459434 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ebc3b40-fc93-4e77-8647-67f322901de3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:07.459925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ebc3b40-fc93-4e77-8647-67f322901de3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:07.460358 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ebc3b40-fc93-4e77-8647-67f322901de3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:07.461094 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ebc3b40-fc93-4e77-8647-67f322901de3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:07.461545 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ebc3b40-fc93-4e77-8647-67f322901de3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:07.461969 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ebc3b40-fc93-4e77-8647-67f322901de3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:07.610264 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2ebc3b40-fc93-4e77-8647-67f322901de3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:07.618696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2ebc3b40-fc93-4e77-8647-67f322901de3 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2" status: 200 len: 1366 microversion: 2.1 time: 0.172265 Aug 28 01:26:07.619244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1428/3078] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:07 2022] GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2 => generated 1366 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:07.767994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ebc68730-123f-4813-a804-1b4071db554f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:07.768629 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ebc68730-123f-4813-a804-1b4071db554f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:07.775266 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebc68730-123f-4813-a804-1b4071db554f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:07.775753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebc68730-123f-4813-a804-1b4071db554f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:07.776295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebc68730-123f-4813-a804-1b4071db554f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:07.777069 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebc68730-123f-4813-a804-1b4071db554f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:07.777614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebc68730-123f-4813-a804-1b4071db554f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:07.778073 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ebc68730-123f-4813-a804-1b4071db554f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:08.009209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ebc68730-123f-4813-a804-1b4071db554f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:08.020317 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ebc68730-123f-4813-a804-1b4071db554f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.256217 Aug 28 01:26:08.020937 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1651/3079] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:07 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:08.640996 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ea107368-f50c-470f-b73f-849bddac2f1f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:08.641662 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ea107368-f50c-470f-b73f-849bddac2f1f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:08.648424 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea107368-f50c-470f-b73f-849bddac2f1f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:08.648930 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea107368-f50c-470f-b73f-849bddac2f1f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:08.649382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea107368-f50c-470f-b73f-849bddac2f1f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:08.650075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea107368-f50c-470f-b73f-849bddac2f1f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:08.650520 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea107368-f50c-470f-b73f-849bddac2f1f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:08.650941 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ea107368-f50c-470f-b73f-849bddac2f1f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:08.792293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ea107368-f50c-470f-b73f-849bddac2f1f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:08.800993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ea107368-f50c-470f-b73f-849bddac2f1f tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2" status: 200 len: 1366 microversion: 2.1 time: 0.165372 Aug 28 01:26:08.801575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1429/3080] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:08 2022] GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2 => generated 1366 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:09.046126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-29d29780-21a1-45c8-92f0-0d5c8a2d7735 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:09.046994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-29d29780-21a1-45c8-92f0-0d5c8a2d7735 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:09.057282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29d29780-21a1-45c8-92f0-0d5c8a2d7735 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:09.057934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29d29780-21a1-45c8-92f0-0d5c8a2d7735 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:09.058518 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29d29780-21a1-45c8-92f0-0d5c8a2d7735 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:09.059530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29d29780-21a1-45c8-92f0-0d5c8a2d7735 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:09.060162 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29d29780-21a1-45c8-92f0-0d5c8a2d7735 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:09.060751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-29d29780-21a1-45c8-92f0-0d5c8a2d7735 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:09.202741 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-29d29780-21a1-45c8-92f0-0d5c8a2d7735 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:09.213069 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-29d29780-21a1-45c8-92f0-0d5c8a2d7735 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.173050 Aug 28 01:26:09.213692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1652/3081] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:09 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:09.826886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-df1104a9-638f-487e-a748-c407f2f4c256 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:09.827683 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-df1104a9-638f-487e-a748-c407f2f4c256 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:09.835823 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-df1104a9-638f-487e-a748-c407f2f4c256 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:09.836305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-df1104a9-638f-487e-a748-c407f2f4c256 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:09.836741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-df1104a9-638f-487e-a748-c407f2f4c256 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:09.837471 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-df1104a9-638f-487e-a748-c407f2f4c256 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:09.837922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-df1104a9-638f-487e-a748-c407f2f4c256 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:09.838343 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-df1104a9-638f-487e-a748-c407f2f4c256 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:09.979559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-df1104a9-638f-487e-a748-c407f2f4c256 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:09.988359 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-df1104a9-638f-487e-a748-c407f2f4c256 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2" status: 200 len: 1366 microversion: 2.1 time: 0.165942 Aug 28 01:26:09.989045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1430/3082] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:09 2022] GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2 => generated 1366 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:10.237880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e03f417a-787f-4232-88e9-dda755db6c77 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:10.239089 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e03f417a-787f-4232-88e9-dda755db6c77 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:10.249506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e03f417a-787f-4232-88e9-dda755db6c77 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:10.250124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e03f417a-787f-4232-88e9-dda755db6c77 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:10.250567 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e03f417a-787f-4232-88e9-dda755db6c77 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:10.251268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e03f417a-787f-4232-88e9-dda755db6c77 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:10.251726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e03f417a-787f-4232-88e9-dda755db6c77 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:10.252150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e03f417a-787f-4232-88e9-dda755db6c77 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:10.395133 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e03f417a-787f-4232-88e9-dda755db6c77 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:10.399365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:26:10.409025 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e03f417a-787f-4232-88e9-dda755db6c77 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.177619 Aug 28 01:26:10.409607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1653/3083] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:10 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:10.947491 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-26e695b9-b199-4488-9ffd-ae08ef92f2f8 req-2014c1d8-f7c5-4258-b8ae-4ce675dbf5d7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9d45bf39-4417-4af3-869d-9761326e9eb2", "name": "network-vif-plugged", "status": "completed", "tag": "9e2ae2aa-ec13-4a1a-a862-13d3bf69d93a"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:26:10.959230 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-26e695b9-b199-4488-9ffd-ae08ef92f2f8 req-2014c1d8-f7c5-4258-b8ae-4ce675dbf5d7 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:10.959762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-26e695b9-b199-4488-9ffd-ae08ef92f2f8 req-2014c1d8-f7c5-4258-b8ae-4ce675dbf5d7 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:10.960196 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-26e695b9-b199-4488-9ffd-ae08ef92f2f8 req-2014c1d8-f7c5-4258-b8ae-4ce675dbf5d7 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:10.971913 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-26e695b9-b199-4488-9ffd-ae08ef92f2f8 req-2014c1d8-f7c5-4258-b8ae-4ce675dbf5d7 service nova] Creating event network-vif-plugged:9e2ae2aa-ec13-4a1a-a862-13d3bf69d93a for instance 9d45bf39-4417-4af3-869d-9761326e9eb2 on hv-cinder-maste Aug 28 01:26:10.977066 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-26e695b9-b199-4488-9ffd-ae08ef92f2f8 req-2014c1d8-f7c5-4258-b8ae-4ce675dbf5d7 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033292 Aug 28 01:26:10.977665 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1431/3084] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:26:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:26:11.013463 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-061ae9d1-ac12-4abd-b7a5-d5010e04da70 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:11.014100 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-061ae9d1-ac12-4abd-b7a5-d5010e04da70 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:11.021792 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-061ae9d1-ac12-4abd-b7a5-d5010e04da70 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:11.022346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-061ae9d1-ac12-4abd-b7a5-d5010e04da70 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:11.022789 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-061ae9d1-ac12-4abd-b7a5-d5010e04da70 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:11.023707 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-061ae9d1-ac12-4abd-b7a5-d5010e04da70 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:11.024354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-061ae9d1-ac12-4abd-b7a5-d5010e04da70 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:11.024907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-061ae9d1-ac12-4abd-b7a5-d5010e04da70 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:11.170757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-061ae9d1-ac12-4abd-b7a5-d5010e04da70 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:11.179030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-061ae9d1-ac12-4abd-b7a5-d5010e04da70 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2" status: 200 len: 1366 microversion: 2.1 time: 0.170398 Aug 28 01:26:11.179614 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1654/3085] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:11 2022] GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2 => generated 1366 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:11.432893 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-fddf2996-4658-4112-89d5-ef856f3aa1aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:11.434069 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-fddf2996-4658-4112-89d5-ef856f3aa1aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:11.442219 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fddf2996-4658-4112-89d5-ef856f3aa1aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:11.442853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fddf2996-4658-4112-89d5-ef856f3aa1aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:11.442970 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fddf2996-4658-4112-89d5-ef856f3aa1aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:11.443595 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fddf2996-4658-4112-89d5-ef856f3aa1aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:11.444043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fddf2996-4658-4112-89d5-ef856f3aa1aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:11.444468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-fddf2996-4658-4112-89d5-ef856f3aa1aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:11.460089 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=2 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:26:11.477192 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=10 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:26:11.592078 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-fddf2996-4658-4112-89d5-ef856f3aa1aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:11.600756 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-fddf2996-4658-4112-89d5-ef856f3aa1aa tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.174256 Aug 28 01:26:11.601404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1432/3086] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:11 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:12.201276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f1c43682-7de8-43bb-afa4-31ecb67b335d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:12.202313 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f1c43682-7de8-43bb-afa4-31ecb67b335d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:12.205702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=2 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:26:12.213752 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f1c43682-7de8-43bb-afa4-31ecb67b335d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:12.214406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f1c43682-7de8-43bb-afa4-31ecb67b335d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:12.215039 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f1c43682-7de8-43bb-afa4-31ecb67b335d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:12.215990 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f1c43682-7de8-43bb-afa4-31ecb67b335d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:12.216576 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f1c43682-7de8-43bb-afa4-31ecb67b335d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:12.217117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f1c43682-7de8-43bb-afa4-31ecb67b335d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:12.355163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f1c43682-7de8-43bb-afa4-31ecb67b335d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:12.363642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f1c43682-7de8-43bb-afa4-31ecb67b335d tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2" status: 200 len: 1366 microversion: 2.1 time: 0.167108 Aug 28 01:26:12.364233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1655/3087] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:12 2022] GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2 => generated 1366 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:12.623956 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e92cdb1a-53d7-4d00-9905-8ce68b856d7b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:12.624587 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e92cdb1a-53d7-4d00-9905-8ce68b856d7b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:12.631445 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e92cdb1a-53d7-4d00-9905-8ce68b856d7b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:12.631905 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e92cdb1a-53d7-4d00-9905-8ce68b856d7b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:12.632337 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e92cdb1a-53d7-4d00-9905-8ce68b856d7b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:12.633171 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e92cdb1a-53d7-4d00-9905-8ce68b856d7b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:12.633619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e92cdb1a-53d7-4d00-9905-8ce68b856d7b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:12.634039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e92cdb1a-53d7-4d00-9905-8ce68b856d7b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:12.775984 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e92cdb1a-53d7-4d00-9905-8ce68b856d7b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:12.784546 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e92cdb1a-53d7-4d00-9905-8ce68b856d7b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.164504 Aug 28 01:26:12.785139 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1433/3088] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:12 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:13.384361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a2417243-aeae-49fd-91c3-025c073bec71 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:13.385004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a2417243-aeae-49fd-91c3-025c073bec71 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:13.391662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2417243-aeae-49fd-91c3-025c073bec71 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:13.392133 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2417243-aeae-49fd-91c3-025c073bec71 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:13.392557 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2417243-aeae-49fd-91c3-025c073bec71 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:13.393270 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2417243-aeae-49fd-91c3-025c073bec71 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:13.393708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2417243-aeae-49fd-91c3-025c073bec71 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:13.394126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a2417243-aeae-49fd-91c3-025c073bec71 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:13.528943 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a2417243-aeae-49fd-91c3-025c073bec71 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:13.536984 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a2417243-aeae-49fd-91c3-025c073bec71 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2" status: 200 len: 1557 microversion: 2.1 time: 0.156502 Aug 28 01:26:13.537553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1656/3089] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:13 2022] GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2 => generated 1557 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:13.565950 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2b5a595f-c7a8-4a22-bf82-24b016a7cd59 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:13.566444 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2b5a595f-c7a8-4a22-bf82-24b016a7cd59 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:13.572602 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b5a595f-c7a8-4a22-bf82-24b016a7cd59 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:13.573113 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b5a595f-c7a8-4a22-bf82-24b016a7cd59 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:13.573536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b5a595f-c7a8-4a22-bf82-24b016a7cd59 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:13.574181 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b5a595f-c7a8-4a22-bf82-24b016a7cd59 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:13.574657 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b5a595f-c7a8-4a22-bf82-24b016a7cd59 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:13.575067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2b5a595f-c7a8-4a22-bf82-24b016a7cd59 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:13.652400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2b5a595f-c7a8-4a22-bf82-24b016a7cd59 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.090659 Aug 28 01:26:13.653044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1434/3090] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Aug 28 01:26:13 2022] GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2/os-interface => generated 285 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:26:13.807954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6d13d7cd-5dd3-4e2b-bb42-4e4733cd3bcc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:13.808558 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6d13d7cd-5dd3-4e2b-bb42-4e4733cd3bcc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:13.815463 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d13d7cd-5dd3-4e2b-bb42-4e4733cd3bcc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:13.815957 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d13d7cd-5dd3-4e2b-bb42-4e4733cd3bcc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:13.816377 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d13d7cd-5dd3-4e2b-bb42-4e4733cd3bcc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:13.817153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d13d7cd-5dd3-4e2b-bb42-4e4733cd3bcc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:13.817610 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d13d7cd-5dd3-4e2b-bb42-4e4733cd3bcc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:13.818038 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d13d7cd-5dd3-4e2b-bb42-4e4733cd3bcc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:14.334260 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6d13d7cd-5dd3-4e2b-bb42-4e4733cd3bcc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:14.343377 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6d13d7cd-5dd3-4e2b-bb42-4e4733cd3bcc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.539836 Aug 28 01:26:14.343968 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1657/3091] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:13 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 541 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:14.970631 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-3aa28e90-9b3f-4e91-97a8-efcee66f05db req-e2fb2ae6-d2da-4850-b138-41d13fd36768 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9d45bf39-4417-4af3-869d-9761326e9eb2", "tag": "9e2ae2aa-ec13-4a1a-a862-13d3bf69d93a"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:26:14.980453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-3aa28e90-9b3f-4e91-97a8-efcee66f05db req-e2fb2ae6-d2da-4850-b138-41d13fd36768 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:14.980925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-3aa28e90-9b3f-4e91-97a8-efcee66f05db req-e2fb2ae6-d2da-4850-b138-41d13fd36768 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:14.981398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-3aa28e90-9b3f-4e91-97a8-efcee66f05db req-e2fb2ae6-d2da-4850-b138-41d13fd36768 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:14.993628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-3aa28e90-9b3f-4e91-97a8-efcee66f05db req-e2fb2ae6-d2da-4850-b138-41d13fd36768 service nova] Creating event network-changed:9e2ae2aa-ec13-4a1a-a862-13d3bf69d93a for instance 9d45bf39-4417-4af3-869d-9761326e9eb2 on hv-cinder-maste Aug 28 01:26:14.998977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-3aa28e90-9b3f-4e91-97a8-efcee66f05db req-e2fb2ae6-d2da-4850-b138-41d13fd36768 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032472 Aug 28 01:26:14.999606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1435/3092] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:26:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:26:15.364406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-21d6e8be-c0af-40bf-a98a-c95948d8f464 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:15.365067 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-21d6e8be-c0af-40bf-a98a-c95948d8f464 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:15.371743 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-21d6e8be-c0af-40bf-a98a-c95948d8f464 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:15.372237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-21d6e8be-c0af-40bf-a98a-c95948d8f464 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:15.372651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-21d6e8be-c0af-40bf-a98a-c95948d8f464 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:15.373509 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-21d6e8be-c0af-40bf-a98a-c95948d8f464 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:15.374178 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-21d6e8be-c0af-40bf-a98a-c95948d8f464 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:15.374703 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-21d6e8be-c0af-40bf-a98a-c95948d8f464 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:15.602586 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-21d6e8be-c0af-40bf-a98a-c95948d8f464 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:15.611162 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-21d6e8be-c0af-40bf-a98a-c95948d8f464 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.251126 Aug 28 01:26:15.617421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1658/3093] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:15 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:16.633833 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a29f29e3-976e-4098-9d14-9b86be5e0276 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:16.634534 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a29f29e3-976e-4098-9d14-9b86be5e0276 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:16.636530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=94,nova_api:UPDATE=2,nova_api:INSERT=4 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:26:16.642947 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a29f29e3-976e-4098-9d14-9b86be5e0276 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:16.643444 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a29f29e3-976e-4098-9d14-9b86be5e0276 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:16.643957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a29f29e3-976e-4098-9d14-9b86be5e0276 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:16.644651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a29f29e3-976e-4098-9d14-9b86be5e0276 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:16.645132 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a29f29e3-976e-4098-9d14-9b86be5e0276 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:16.645550 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a29f29e3-976e-4098-9d14-9b86be5e0276 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:16.789687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a29f29e3-976e-4098-9d14-9b86be5e0276 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:16.798673 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a29f29e3-976e-4098-9d14-9b86be5e0276 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.169722 Aug 28 01:26:16.799272 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1436/3094] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:16 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:17.821479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-506418c8-246d-483b-a5a1-34c7b083f198 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:17.822103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-506418c8-246d-483b-a5a1-34c7b083f198 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:17.829345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-506418c8-246d-483b-a5a1-34c7b083f198 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:17.829832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-506418c8-246d-483b-a5a1-34c7b083f198 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:17.830265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-506418c8-246d-483b-a5a1-34c7b083f198 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:17.830966 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-506418c8-246d-483b-a5a1-34c7b083f198 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:17.831417 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-506418c8-246d-483b-a5a1-34c7b083f198 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:17.831839 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-506418c8-246d-483b-a5a1-34c7b083f198 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:17.975803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-506418c8-246d-483b-a5a1-34c7b083f198 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:17.986108 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-506418c8-246d-483b-a5a1-34c7b083f198 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.169385 Aug 28 01:26:17.986898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1659/3095] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:17 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:19.007928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8b167360-d56f-4508-80e9-3d9c957baada tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:19.008667 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8b167360-d56f-4508-80e9-3d9c957baada tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:19.015865 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b167360-d56f-4508-80e9-3d9c957baada tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:19.016344 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b167360-d56f-4508-80e9-3d9c957baada tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:19.016781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b167360-d56f-4508-80e9-3d9c957baada tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:19.017568 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b167360-d56f-4508-80e9-3d9c957baada tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:19.018018 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b167360-d56f-4508-80e9-3d9c957baada tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:19.018444 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b167360-d56f-4508-80e9-3d9c957baada tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:19.180841 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8b167360-d56f-4508-80e9-3d9c957baada tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:19.190560 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8b167360-d56f-4508-80e9-3d9c957baada tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.186504 Aug 28 01:26:19.191319 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1437/3096] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:19 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:20.212095 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-badff8a4-8d45-49ca-bdd5-0e46ee5a20f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:20.212712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-badff8a4-8d45-49ca-bdd5-0e46ee5a20f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:20.219803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-badff8a4-8d45-49ca-bdd5-0e46ee5a20f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:20.220289 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-badff8a4-8d45-49ca-bdd5-0e46ee5a20f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:20.220733 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-badff8a4-8d45-49ca-bdd5-0e46ee5a20f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:20.221589 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-badff8a4-8d45-49ca-bdd5-0e46ee5a20f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:20.222012 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-badff8a4-8d45-49ca-bdd5-0e46ee5a20f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:20.222416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-badff8a4-8d45-49ca-bdd5-0e46ee5a20f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:20.359776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-badff8a4-8d45-49ca-bdd5-0e46ee5a20f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:20.372640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-badff8a4-8d45-49ca-bdd5-0e46ee5a20f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.164488 Aug 28 01:26:20.373232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1660/3097] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:20 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:21.394397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d6a203da-f6a5-44c8-a2a9-8941c3a25a14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:21.395040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d6a203da-f6a5-44c8-a2a9-8941c3a25a14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:21.402339 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6a203da-f6a5-44c8-a2a9-8941c3a25a14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:21.402813 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6a203da-f6a5-44c8-a2a9-8941c3a25a14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:21.403241 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6a203da-f6a5-44c8-a2a9-8941c3a25a14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:21.403933 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6a203da-f6a5-44c8-a2a9-8941c3a25a14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:21.404374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6a203da-f6a5-44c8-a2a9-8941c3a25a14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:21.404782 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6a203da-f6a5-44c8-a2a9-8941c3a25a14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:21.407486 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:26:21.540480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d6a203da-f6a5-44c8-a2a9-8941c3a25a14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:21.549150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d6a203da-f6a5-44c8-a2a9-8941c3a25a14 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.158654 Aug 28 01:26:21.549716 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1438/3098] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:21 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:22.571458 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-55130548-9e18-4bfb-a48b-d7a0be72a88a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:22.572081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-55130548-9e18-4bfb-a48b-d7a0be72a88a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:22.579006 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-55130548-9e18-4bfb-a48b-d7a0be72a88a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:22.579467 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-55130548-9e18-4bfb-a48b-d7a0be72a88a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:22.579926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-55130548-9e18-4bfb-a48b-d7a0be72a88a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:22.580609 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-55130548-9e18-4bfb-a48b-d7a0be72a88a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:22.581078 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-55130548-9e18-4bfb-a48b-d7a0be72a88a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:22.581496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-55130548-9e18-4bfb-a48b-d7a0be72a88a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:22.713397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-55130548-9e18-4bfb-a48b-d7a0be72a88a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:22.722025 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-55130548-9e18-4bfb-a48b-d7a0be72a88a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.154474 Aug 28 01:26:22.722585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1661/3099] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:22 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:23.745552 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-13ffc755-8700-48e5-af3d-75a45454f163 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:23.746155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-13ffc755-8700-48e5-af3d-75a45454f163 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:23.752773 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-13ffc755-8700-48e5-af3d-75a45454f163 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:23.753281 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-13ffc755-8700-48e5-af3d-75a45454f163 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:23.753719 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-13ffc755-8700-48e5-af3d-75a45454f163 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:23.754414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-13ffc755-8700-48e5-af3d-75a45454f163 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:23.754860 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-13ffc755-8700-48e5-af3d-75a45454f163 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:23.755285 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-13ffc755-8700-48e5-af3d-75a45454f163 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:23.888388 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-13ffc755-8700-48e5-af3d-75a45454f163 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:23.896868 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-13ffc755-8700-48e5-af3d-75a45454f163 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.155386 Aug 28 01:26:23.897477 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1439/3100] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:23 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:24.917858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-38518154-bb3e-42e8-a5e8-439ac02dba83 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:24.918502 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-38518154-bb3e-42e8-a5e8-439ac02dba83 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:24.925847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38518154-bb3e-42e8-a5e8-439ac02dba83 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:24.926382 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38518154-bb3e-42e8-a5e8-439ac02dba83 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:24.926835 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38518154-bb3e-42e8-a5e8-439ac02dba83 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:24.927566 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38518154-bb3e-42e8-a5e8-439ac02dba83 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:24.928018 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38518154-bb3e-42e8-a5e8-439ac02dba83 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:24.928471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38518154-bb3e-42e8-a5e8-439ac02dba83 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:25.079026 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-38518154-bb3e-42e8-a5e8-439ac02dba83 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:25.088407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-38518154-bb3e-42e8-a5e8-439ac02dba83 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.174365 Aug 28 01:26:25.089036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1662/3101] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:24 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:26.110425 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-01a32b12-586f-4e3c-a6b7-37eb19090314 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:26.111053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-01a32b12-586f-4e3c-a6b7-37eb19090314 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:26.118558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-01a32b12-586f-4e3c-a6b7-37eb19090314 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:26.119092 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-01a32b12-586f-4e3c-a6b7-37eb19090314 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:26.119526 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-01a32b12-586f-4e3c-a6b7-37eb19090314 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:26.120245 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-01a32b12-586f-4e3c-a6b7-37eb19090314 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:26.120698 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-01a32b12-586f-4e3c-a6b7-37eb19090314 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:26.121783 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-01a32b12-586f-4e3c-a6b7-37eb19090314 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:26.260842 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-01a32b12-586f-4e3c-a6b7-37eb19090314 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:26.270590 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-01a32b12-586f-4e3c-a6b7-37eb19090314 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.164155 Aug 28 01:26:26.271179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1440/3102] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:26 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:27.294714 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3ca3594f-d5a6-46bd-ab0c-82524e4fc3fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:27.295852 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3ca3594f-d5a6-46bd-ab0c-82524e4fc3fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:27.303834 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ca3594f-d5a6-46bd-ab0c-82524e4fc3fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:27.304314 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ca3594f-d5a6-46bd-ab0c-82524e4fc3fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:27.304740 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ca3594f-d5a6-46bd-ab0c-82524e4fc3fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:27.305505 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ca3594f-d5a6-46bd-ab0c-82524e4fc3fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:27.305949 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ca3594f-d5a6-46bd-ab0c-82524e4fc3fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:27.306370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ca3594f-d5a6-46bd-ab0c-82524e4fc3fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:27.462572 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3ca3594f-d5a6-46bd-ab0c-82524e4fc3fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:27.471313 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3ca3594f-d5a6-46bd-ab0c-82524e4fc3fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.180687 Aug 28 01:26:27.471908 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1663/3103] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:27 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:28.496375 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d04bdbe7-9510-47ce-81f3-d5af4644aff9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:28.497104 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d04bdbe7-9510-47ce-81f3-d5af4644aff9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:28.504315 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d04bdbe7-9510-47ce-81f3-d5af4644aff9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:28.504802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d04bdbe7-9510-47ce-81f3-d5af4644aff9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:28.505283 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d04bdbe7-9510-47ce-81f3-d5af4644aff9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:28.505992 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d04bdbe7-9510-47ce-81f3-d5af4644aff9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:28.506441 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d04bdbe7-9510-47ce-81f3-d5af4644aff9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:28.506844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d04bdbe7-9510-47ce-81f3-d5af4644aff9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:28.645887 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d04bdbe7-9510-47ce-81f3-d5af4644aff9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:28.654940 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d04bdbe7-9510-47ce-81f3-d5af4644aff9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.163840 Aug 28 01:26:28.655507 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1441/3104] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:28 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:29.686111 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4590afd0-fc93-4c4a-b637-fe9e324d4c38 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:29.686805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4590afd0-fc93-4c4a-b637-fe9e324d4c38 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:29.689517 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=56,nova_api:DELETE=2 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:26:29.697139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4590afd0-fc93-4c4a-b637-fe9e324d4c38 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:29.697811 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4590afd0-fc93-4c4a-b637-fe9e324d4c38 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:29.698522 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4590afd0-fc93-4c4a-b637-fe9e324d4c38 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:29.699644 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4590afd0-fc93-4c4a-b637-fe9e324d4c38 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:29.700373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4590afd0-fc93-4c4a-b637-fe9e324d4c38 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:29.701080 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4590afd0-fc93-4c4a-b637-fe9e324d4c38 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:29.862588 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4590afd0-fc93-4c4a-b637-fe9e324d4c38 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:29.871676 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4590afd0-fc93-4c4a-b637-fe9e324d4c38 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.189903 Aug 28 01:26:29.872269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1664/3105] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:29 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:30.893072 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4d15d6f8-6ad4-4cdf-8b5e-c0c25d99bb80 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:30.893717 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4d15d6f8-6ad4-4cdf-8b5e-c0c25d99bb80 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:30.901123 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d15d6f8-6ad4-4cdf-8b5e-c0c25d99bb80 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:30.901626 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d15d6f8-6ad4-4cdf-8b5e-c0c25d99bb80 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:30.902062 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d15d6f8-6ad4-4cdf-8b5e-c0c25d99bb80 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:30.902780 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d15d6f8-6ad4-4cdf-8b5e-c0c25d99bb80 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:30.903232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d15d6f8-6ad4-4cdf-8b5e-c0c25d99bb80 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:30.903658 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4d15d6f8-6ad4-4cdf-8b5e-c0c25d99bb80 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:31.049758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4d15d6f8-6ad4-4cdf-8b5e-c0c25d99bb80 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:31.058998 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4d15d6f8-6ad4-4cdf-8b5e-c0c25d99bb80 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.169780 Aug 28 01:26:31.059577 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1442/3106] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:30 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:32.081093 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c5ad4230-27f4-425c-a7e7-32f4e6c9bc5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:32.081748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c5ad4230-27f4-425c-a7e7-32f4e6c9bc5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:32.088400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5ad4230-27f4-425c-a7e7-32f4e6c9bc5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:32.088886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5ad4230-27f4-425c-a7e7-32f4e6c9bc5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:32.089338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5ad4230-27f4-425c-a7e7-32f4e6c9bc5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:32.090034 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5ad4230-27f4-425c-a7e7-32f4e6c9bc5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:32.090474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5ad4230-27f4-425c-a7e7-32f4e6c9bc5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:32.090890 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c5ad4230-27f4-425c-a7e7-32f4e6c9bc5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:32.248521 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c5ad4230-27f4-425c-a7e7-32f4e6c9bc5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:32.263388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c5ad4230-27f4-425c-a7e7-32f4e6c9bc5d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.186047 Aug 28 01:26:32.264010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1665/3107] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:32 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:33.289902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3a09cc9e-a255-4ab0-8757-bc30b28ab6d3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:33.290565 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3a09cc9e-a255-4ab0-8757-bc30b28ab6d3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:33.298555 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a09cc9e-a255-4ab0-8757-bc30b28ab6d3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:33.299277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a09cc9e-a255-4ab0-8757-bc30b28ab6d3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:33.299730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a09cc9e-a255-4ab0-8757-bc30b28ab6d3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:33.300453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a09cc9e-a255-4ab0-8757-bc30b28ab6d3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:33.300908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a09cc9e-a255-4ab0-8757-bc30b28ab6d3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:33.301357 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3a09cc9e-a255-4ab0-8757-bc30b28ab6d3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:33.441985 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3a09cc9e-a255-4ab0-8757-bc30b28ab6d3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:33.451095 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3a09cc9e-a255-4ab0-8757-bc30b28ab6d3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.165030 Aug 28 01:26:33.451726 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1443/3108] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:33 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:34.472512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-dc7c76e2-eea3-47d0-8008-7b2a0fdb4572 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:34.473174 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-dc7c76e2-eea3-47d0-8008-7b2a0fdb4572 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:34.479972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc7c76e2-eea3-47d0-8008-7b2a0fdb4572 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:34.480450 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc7c76e2-eea3-47d0-8008-7b2a0fdb4572 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:34.480872 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc7c76e2-eea3-47d0-8008-7b2a0fdb4572 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:34.481589 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc7c76e2-eea3-47d0-8008-7b2a0fdb4572 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:34.482027 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc7c76e2-eea3-47d0-8008-7b2a0fdb4572 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:34.482435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc7c76e2-eea3-47d0-8008-7b2a0fdb4572 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:34.626642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dc7c76e2-eea3-47d0-8008-7b2a0fdb4572 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:34.635707 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-dc7c76e2-eea3-47d0-8008-7b2a0fdb4572 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.167047 Aug 28 01:26:34.636332 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1666/3109] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:34 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:35.657352 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-714f65d8-0e2f-415f-b5d1-0102c9f83ff9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:35.658013 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-714f65d8-0e2f-415f-b5d1-0102c9f83ff9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:35.665045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-714f65d8-0e2f-415f-b5d1-0102c9f83ff9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:35.665540 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-714f65d8-0e2f-415f-b5d1-0102c9f83ff9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:35.665980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-714f65d8-0e2f-415f-b5d1-0102c9f83ff9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:35.666659 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-714f65d8-0e2f-415f-b5d1-0102c9f83ff9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:35.667123 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-714f65d8-0e2f-415f-b5d1-0102c9f83ff9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:35.667534 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-714f65d8-0e2f-415f-b5d1-0102c9f83ff9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:35.814606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-714f65d8-0e2f-415f-b5d1-0102c9f83ff9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:35.823576 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-714f65d8-0e2f-415f-b5d1-0102c9f83ff9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.170236 Aug 28 01:26:35.824191 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1444/3110] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:35 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:36.846585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d36b69c7-9cca-4033-958c-63bb183142e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:36.847650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d36b69c7-9cca-4033-958c-63bb183142e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:36.855343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d36b69c7-9cca-4033-958c-63bb183142e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:36.855843 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d36b69c7-9cca-4033-958c-63bb183142e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:36.856279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d36b69c7-9cca-4033-958c-63bb183142e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:36.857014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d36b69c7-9cca-4033-958c-63bb183142e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:36.857496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d36b69c7-9cca-4033-958c-63bb183142e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:36.857934 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d36b69c7-9cca-4033-958c-63bb183142e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:37.013969 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d36b69c7-9cca-4033-958c-63bb183142e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:37.022896 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d36b69c7-9cca-4033-958c-63bb183142e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.180546 Aug 28 01:26:37.023520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1667/3111] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:36 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:38.049266 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9b2ab4a3-866f-4344-98b4-50ecb6db16e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:38.050022 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9b2ab4a3-866f-4344-98b4-50ecb6db16e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:38.057255 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b2ab4a3-866f-4344-98b4-50ecb6db16e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:38.057757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b2ab4a3-866f-4344-98b4-50ecb6db16e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:38.058194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b2ab4a3-866f-4344-98b4-50ecb6db16e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:38.058924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b2ab4a3-866f-4344-98b4-50ecb6db16e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:38.059388 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b2ab4a3-866f-4344-98b4-50ecb6db16e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:38.059820 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9b2ab4a3-866f-4344-98b4-50ecb6db16e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:38.207105 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9b2ab4a3-866f-4344-98b4-50ecb6db16e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:38.215984 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9b2ab4a3-866f-4344-98b4-50ecb6db16e2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.171152 Aug 28 01:26:38.216581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1445/3112] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:38 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:39.239683 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fb38a210-db45-4fe1-b4c2-231eb04543f5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:39.240766 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fb38a210-db45-4fe1-b4c2-231eb04543f5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:39.249669 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fb38a210-db45-4fe1-b4c2-231eb04543f5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:39.250150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fb38a210-db45-4fe1-b4c2-231eb04543f5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:39.250592 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fb38a210-db45-4fe1-b4c2-231eb04543f5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:39.251291 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fb38a210-db45-4fe1-b4c2-231eb04543f5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:39.251740 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fb38a210-db45-4fe1-b4c2-231eb04543f5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:39.252170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fb38a210-db45-4fe1-b4c2-231eb04543f5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:39.389931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fb38a210-db45-4fe1-b4c2-231eb04543f5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:39.398296 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fb38a210-db45-4fe1-b4c2-231eb04543f5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.164713 Aug 28 01:26:39.398855 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1668/3113] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:39 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:40.420556 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7ea6c815-a05f-4f91-b965-85f74cc9c14b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:40.421250 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7ea6c815-a05f-4f91-b965-85f74cc9c14b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:40.428389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ea6c815-a05f-4f91-b965-85f74cc9c14b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:40.428880 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ea6c815-a05f-4f91-b965-85f74cc9c14b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:40.429344 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ea6c815-a05f-4f91-b965-85f74cc9c14b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:40.430040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ea6c815-a05f-4f91-b965-85f74cc9c14b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:40.430486 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ea6c815-a05f-4f91-b965-85f74cc9c14b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:40.430903 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7ea6c815-a05f-4f91-b965-85f74cc9c14b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:40.566370 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7ea6c815-a05f-4f91-b965-85f74cc9c14b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:40.575667 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7ea6c815-a05f-4f91-b965-85f74cc9c14b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.159729 Aug 28 01:26:40.576305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1446/3114] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:40 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:41.598896 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-982523a5-1fed-4d92-951d-adc956f9beb1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:41.599558 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-982523a5-1fed-4d92-951d-adc956f9beb1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:41.606673 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-982523a5-1fed-4d92-951d-adc956f9beb1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:41.607154 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-982523a5-1fed-4d92-951d-adc956f9beb1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:41.607580 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-982523a5-1fed-4d92-951d-adc956f9beb1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:41.608279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-982523a5-1fed-4d92-951d-adc956f9beb1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:41.608853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-982523a5-1fed-4d92-951d-adc956f9beb1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:41.609309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-982523a5-1fed-4d92-951d-adc956f9beb1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:41.612234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=92 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:26:41.758023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-982523a5-1fed-4d92-951d-adc956f9beb1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:41.766490 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-982523a5-1fed-4d92-951d-adc956f9beb1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.171490 Aug 28 01:26:41.767085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1669/3115] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:41 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:42.792418 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2d88d2fb-24d4-4a8a-b0cf-456bb0c40f58 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:42.793476 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2d88d2fb-24d4-4a8a-b0cf-456bb0c40f58 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:42.802059 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d88d2fb-24d4-4a8a-b0cf-456bb0c40f58 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:42.802560 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d88d2fb-24d4-4a8a-b0cf-456bb0c40f58 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:42.802998 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d88d2fb-24d4-4a8a-b0cf-456bb0c40f58 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:42.803723 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d88d2fb-24d4-4a8a-b0cf-456bb0c40f58 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:42.804181 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d88d2fb-24d4-4a8a-b0cf-456bb0c40f58 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:42.804606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2d88d2fb-24d4-4a8a-b0cf-456bb0c40f58 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:42.945497 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2d88d2fb-24d4-4a8a-b0cf-456bb0c40f58 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:42.954274 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2d88d2fb-24d4-4a8a-b0cf-456bb0c40f58 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.169162 Aug 28 01:26:42.954871 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1447/3116] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:42 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:43.976030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a924dc8d-1756-422e-820e-4e9b3f83812c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:43.976662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a924dc8d-1756-422e-820e-4e9b3f83812c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:43.983581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a924dc8d-1756-422e-820e-4e9b3f83812c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:43.984067 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a924dc8d-1756-422e-820e-4e9b3f83812c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:43.984499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a924dc8d-1756-422e-820e-4e9b3f83812c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:43.985236 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a924dc8d-1756-422e-820e-4e9b3f83812c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:43.985676 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a924dc8d-1756-422e-820e-4e9b3f83812c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:43.986096 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a924dc8d-1756-422e-820e-4e9b3f83812c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:44.130870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a924dc8d-1756-422e-820e-4e9b3f83812c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:44.139974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a924dc8d-1756-422e-820e-4e9b3f83812c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.167690 Aug 28 01:26:44.140562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1670/3117] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:43 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:45.163291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9beef31c-78f5-4e22-a1e8-814687875fa6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:45.163902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9beef31c-78f5-4e22-a1e8-814687875fa6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:45.171578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9beef31c-78f5-4e22-a1e8-814687875fa6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:45.172071 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9beef31c-78f5-4e22-a1e8-814687875fa6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:45.172503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9beef31c-78f5-4e22-a1e8-814687875fa6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:45.173251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9beef31c-78f5-4e22-a1e8-814687875fa6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:45.173702 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9beef31c-78f5-4e22-a1e8-814687875fa6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:45.174121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9beef31c-78f5-4e22-a1e8-814687875fa6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:45.315980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9beef31c-78f5-4e22-a1e8-814687875fa6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:45.329855 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9beef31c-78f5-4e22-a1e8-814687875fa6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.170154 Aug 28 01:26:45.330902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1448/3118] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:45 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:46.354434 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fa8a9064-3385-4515-9be6-48bcab1f5cd6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:46.355227 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fa8a9064-3385-4515-9be6-48bcab1f5cd6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:46.362896 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fa8a9064-3385-4515-9be6-48bcab1f5cd6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:46.363377 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fa8a9064-3385-4515-9be6-48bcab1f5cd6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:46.363805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fa8a9064-3385-4515-9be6-48bcab1f5cd6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:46.364499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fa8a9064-3385-4515-9be6-48bcab1f5cd6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:46.364929 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fa8a9064-3385-4515-9be6-48bcab1f5cd6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:46.365438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fa8a9064-3385-4515-9be6-48bcab1f5cd6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:46.509796 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fa8a9064-3385-4515-9be6-48bcab1f5cd6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:46.518382 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fa8a9064-3385-4515-9be6-48bcab1f5cd6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.167692 Aug 28 01:26:46.518971 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1671/3119] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:46 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:46.857271 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-09637c62-8e9a-48df-a9d6-a1876658ef3c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:46.857804 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-09637c62-8e9a-48df-a9d6-a1876658ef3c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:46.859786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=26 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:26:46.865502 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-09637c62-8e9a-48df-a9d6-a1876658ef3c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:46.866050 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-09637c62-8e9a-48df-a9d6-a1876658ef3c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:46.866314 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-09637c62-8e9a-48df-a9d6-a1876658ef3c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:46.867014 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-09637c62-8e9a-48df-a9d6-a1876658ef3c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:46.867459 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-09637c62-8e9a-48df-a9d6-a1876658ef3c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:46.867872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-09637c62-8e9a-48df-a9d6-a1876658ef3c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:46.893598 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-09637c62-8e9a-48df-a9d6-a1876658ef3c tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df/ips" status: 200 len: 145 microversion: 2.1 time: 0.099646 Aug 28 01:26:46.894190 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1449/3120] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Aug 28 01:26:46 2022] GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df/ips => generated 145 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:26:47.540650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d0c636a6-6394-4e08-91d2-9596848f6af9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:47.541327 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d0c636a6-6394-4e08-91d2-9596848f6af9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:47.548292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d0c636a6-6394-4e08-91d2-9596848f6af9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:47.548791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d0c636a6-6394-4e08-91d2-9596848f6af9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:47.549252 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d0c636a6-6394-4e08-91d2-9596848f6af9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:47.550129 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d0c636a6-6394-4e08-91d2-9596848f6af9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:47.550578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d0c636a6-6394-4e08-91d2-9596848f6af9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:47.550991 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d0c636a6-6394-4e08-91d2-9596848f6af9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:47.705860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d0c636a6-6394-4e08-91d2-9596848f6af9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:47.714926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d0c636a6-6394-4e08-91d2-9596848f6af9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.177977 Aug 28 01:26:47.715429 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1672/3121] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:47 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:26:48.737663 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b37bdd50-5d4e-4369-b5e6-4c0d854497ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:48.738322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b37bdd50-5d4e-4369-b5e6-4c0d854497ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:48.746284 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b37bdd50-5d4e-4369-b5e6-4c0d854497ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:48.746868 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b37bdd50-5d4e-4369-b5e6-4c0d854497ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:48.747350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b37bdd50-5d4e-4369-b5e6-4c0d854497ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:48.748128 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b37bdd50-5d4e-4369-b5e6-4c0d854497ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:48.748615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b37bdd50-5d4e-4369-b5e6-4c0d854497ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:48.749280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b37bdd50-5d4e-4369-b5e6-4c0d854497ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:48.897825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b37bdd50-5d4e-4369-b5e6-4c0d854497ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:48.907049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b37bdd50-5d4e-4369-b5e6-4c0d854497ff tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.174345 Aug 28 01:26:48.907648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1450/3122] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:48 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:49.929580 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cea18b5a-d561-42d4-814f-2b535ad20fcd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:49.930316 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cea18b5a-d561-42d4-814f-2b535ad20fcd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:49.938765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cea18b5a-d561-42d4-814f-2b535ad20fcd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:49.939449 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cea18b5a-d561-42d4-814f-2b535ad20fcd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:49.939982 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cea18b5a-d561-42d4-814f-2b535ad20fcd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:49.941082 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cea18b5a-d561-42d4-814f-2b535ad20fcd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:49.941642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cea18b5a-d561-42d4-814f-2b535ad20fcd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:49.942173 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cea18b5a-d561-42d4-814f-2b535ad20fcd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:50.104562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-cea18b5a-d561-42d4-814f-2b535ad20fcd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:50.114339 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cea18b5a-d561-42d4-814f-2b535ad20fcd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.188789 Aug 28 01:26:50.121221 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1673/3123] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:49 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:51.139324 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0a129cda-275f-486b-8ec5-46c39b9a9eb9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:51.140057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0a129cda-275f-486b-8ec5-46c39b9a9eb9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:51.146940 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0a129cda-275f-486b-8ec5-46c39b9a9eb9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:51.147430 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0a129cda-275f-486b-8ec5-46c39b9a9eb9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:51.147863 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0a129cda-275f-486b-8ec5-46c39b9a9eb9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:51.148582 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0a129cda-275f-486b-8ec5-46c39b9a9eb9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:51.149049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0a129cda-275f-486b-8ec5-46c39b9a9eb9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:51.149565 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0a129cda-275f-486b-8ec5-46c39b9a9eb9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:51.303759 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0a129cda-275f-486b-8ec5-46c39b9a9eb9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:51.314840 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0a129cda-275f-486b-8ec5-46c39b9a9eb9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.179641 Aug 28 01:26:51.315434 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1451/3124] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:51 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:52.337371 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-012f43c7-7883-4824-b4b8-48c3da5c2127 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:52.338159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-012f43c7-7883-4824-b4b8-48c3da5c2127 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:52.345657 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-012f43c7-7883-4824-b4b8-48c3da5c2127 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:52.346145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-012f43c7-7883-4824-b4b8-48c3da5c2127 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:52.346576 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-012f43c7-7883-4824-b4b8-48c3da5c2127 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:52.347283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-012f43c7-7883-4824-b4b8-48c3da5c2127 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:52.347732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-012f43c7-7883-4824-b4b8-48c3da5c2127 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:52.348150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-012f43c7-7883-4824-b4b8-48c3da5c2127 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:52.492040 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-012f43c7-7883-4824-b4b8-48c3da5c2127 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:52.502090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-012f43c7-7883-4824-b4b8-48c3da5c2127 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.168738 Aug 28 01:26:52.502857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1674/3125] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:52 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:53.524927 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-67db8020-24d8-4be7-9709-0b0aae9430b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:53.525699 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-67db8020-24d8-4be7-9709-0b0aae9430b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:53.532674 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67db8020-24d8-4be7-9709-0b0aae9430b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:53.533260 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67db8020-24d8-4be7-9709-0b0aae9430b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:53.533739 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67db8020-24d8-4be7-9709-0b0aae9430b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:53.534503 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67db8020-24d8-4be7-9709-0b0aae9430b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:53.535005 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67db8020-24d8-4be7-9709-0b0aae9430b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:53.535438 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-67db8020-24d8-4be7-9709-0b0aae9430b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:53.538821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=82 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:26:53.691480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-67db8020-24d8-4be7-9709-0b0aae9430b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:53.700899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-67db8020-24d8-4be7-9709-0b0aae9430b3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.180063 Aug 28 01:26:53.701528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1452/3126] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:53 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:26:54.723098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e5ddebc5-8626-4ea6-8d38-c1e5667d4bf5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:54.723757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e5ddebc5-8626-4ea6-8d38-c1e5667d4bf5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:54.731162 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e5ddebc5-8626-4ea6-8d38-c1e5667d4bf5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:54.731709 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e5ddebc5-8626-4ea6-8d38-c1e5667d4bf5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:54.732185 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e5ddebc5-8626-4ea6-8d38-c1e5667d4bf5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:54.733059 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e5ddebc5-8626-4ea6-8d38-c1e5667d4bf5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:54.733570 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e5ddebc5-8626-4ea6-8d38-c1e5667d4bf5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:54.734065 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e5ddebc5-8626-4ea6-8d38-c1e5667d4bf5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:54.872468 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e5ddebc5-8626-4ea6-8d38-c1e5667d4bf5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:54.885431 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e5ddebc5-8626-4ea6-8d38-c1e5667d4bf5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.166017 Aug 28 01:26:54.886255 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1675/3127] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:54 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:55.914083 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-01a36616-885a-4950-873a-b8c9a8ed9f96 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:55.914920 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-01a36616-885a-4950-873a-b8c9a8ed9f96 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:55.922733 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-01a36616-885a-4950-873a-b8c9a8ed9f96 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:55.923225 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-01a36616-885a-4950-873a-b8c9a8ed9f96 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:55.923661 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-01a36616-885a-4950-873a-b8c9a8ed9f96 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:55.924379 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-01a36616-885a-4950-873a-b8c9a8ed9f96 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:55.924831 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-01a36616-885a-4950-873a-b8c9a8ed9f96 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:55.925291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-01a36616-885a-4950-873a-b8c9a8ed9f96 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:56.072974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-01a36616-885a-4950-873a-b8c9a8ed9f96 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:56.082905 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-01a36616-885a-4950-873a-b8c9a8ed9f96 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.172600 Aug 28 01:26:56.083507 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1453/3128] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:55 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:57.110815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-692312d7-3c19-45d9-8bf1-51b17c6421f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:57.111427 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-692312d7-3c19-45d9-8bf1-51b17c6421f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:57.118792 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-692312d7-3c19-45d9-8bf1-51b17c6421f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:57.119300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-692312d7-3c19-45d9-8bf1-51b17c6421f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:57.119709 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-692312d7-3c19-45d9-8bf1-51b17c6421f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:57.120459 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-692312d7-3c19-45d9-8bf1-51b17c6421f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:57.120895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-692312d7-3c19-45d9-8bf1-51b17c6421f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:57.121340 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-692312d7-3c19-45d9-8bf1-51b17c6421f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:57.261370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-692312d7-3c19-45d9-8bf1-51b17c6421f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:57.272357 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-692312d7-3c19-45d9-8bf1-51b17c6421f8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.165478 Aug 28 01:26:57.272948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1676/3129] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:57 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:58.294168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bc2348f7-8595-4b63-a68c-940a35ab04f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:58.294835 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bc2348f7-8595-4b63-a68c-940a35ab04f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:58.301974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc2348f7-8595-4b63-a68c-940a35ab04f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:58.302457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc2348f7-8595-4b63-a68c-940a35ab04f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:58.302890 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc2348f7-8595-4b63-a68c-940a35ab04f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:58.303589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc2348f7-8595-4b63-a68c-940a35ab04f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:58.304034 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc2348f7-8595-4b63-a68c-940a35ab04f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:58.304457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bc2348f7-8595-4b63-a68c-940a35ab04f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:58.451839 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bc2348f7-8595-4b63-a68c-940a35ab04f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:58.461361 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bc2348f7-8595-4b63-a68c-940a35ab04f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.170929 Aug 28 01:26:58.462089 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1454/3130] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:58 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:26:59.484514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5866a0b1-b9c2-4b7b-be04-d74bd09ab4a5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:26:59.485206 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5866a0b1-b9c2-4b7b-be04-d74bd09ab4a5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:26:59.491920 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5866a0b1-b9c2-4b7b-be04-d74bd09ab4a5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:59.492406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5866a0b1-b9c2-4b7b-be04-d74bd09ab4a5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:59.492839 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5866a0b1-b9c2-4b7b-be04-d74bd09ab4a5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:59.493567 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5866a0b1-b9c2-4b7b-be04-d74bd09ab4a5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:26:59.494015 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5866a0b1-b9c2-4b7b-be04-d74bd09ab4a5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:26:59.494437 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5866a0b1-b9c2-4b7b-be04-d74bd09ab4a5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:26:59.634030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5866a0b1-b9c2-4b7b-be04-d74bd09ab4a5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:26:59.646332 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5866a0b1-b9c2-4b7b-be04-d74bd09ab4a5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.165855 Aug 28 01:26:59.647003 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1677/3131] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:26:59 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:27:00.670107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d1169ca2-5bc9-4ee3-8bf4-056fd8da7c0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:00.671159 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d1169ca2-5bc9-4ee3-8bf4-056fd8da7c0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:00.680461 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d1169ca2-5bc9-4ee3-8bf4-056fd8da7c0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:00.681171 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d1169ca2-5bc9-4ee3-8bf4-056fd8da7c0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:00.681741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d1169ca2-5bc9-4ee3-8bf4-056fd8da7c0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:00.682835 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d1169ca2-5bc9-4ee3-8bf4-056fd8da7c0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:00.683509 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d1169ca2-5bc9-4ee3-8bf4-056fd8da7c0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:00.684150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d1169ca2-5bc9-4ee3-8bf4-056fd8da7c0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:00.823774 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d1169ca2-5bc9-4ee3-8bf4-056fd8da7c0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:00.832440 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d1169ca2-5bc9-4ee3-8bf4-056fd8da7c0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.167913 Aug 28 01:27:00.833048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1455/3132] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:00 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:01.855243 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1a089d68-7c3b-4c4f-b3de-acda25af7ac5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:01.855864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1a089d68-7c3b-4c4f-b3de-acda25af7ac5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:01.858810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=28 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:27:01.864756 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a089d68-7c3b-4c4f-b3de-acda25af7ac5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:01.865265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a089d68-7c3b-4c4f-b3de-acda25af7ac5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:01.865701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a089d68-7c3b-4c4f-b3de-acda25af7ac5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:01.866386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a089d68-7c3b-4c4f-b3de-acda25af7ac5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:01.866830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a089d68-7c3b-4c4f-b3de-acda25af7ac5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:01.867249 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1a089d68-7c3b-4c4f-b3de-acda25af7ac5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:02.001346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1a089d68-7c3b-4c4f-b3de-acda25af7ac5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:02.009821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1a089d68-7c3b-4c4f-b3de-acda25af7ac5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.158998 Aug 28 01:27:02.010402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1678/3133] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:01 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:03.033054 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d4b2d71c-7baa-4194-8e62-4862912a8ff7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:03.033910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d4b2d71c-7baa-4194-8e62-4862912a8ff7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:03.041111 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d4b2d71c-7baa-4194-8e62-4862912a8ff7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:03.041629 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d4b2d71c-7baa-4194-8e62-4862912a8ff7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:03.042055 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d4b2d71c-7baa-4194-8e62-4862912a8ff7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:03.042750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d4b2d71c-7baa-4194-8e62-4862912a8ff7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:03.043195 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d4b2d71c-7baa-4194-8e62-4862912a8ff7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:03.043608 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d4b2d71c-7baa-4194-8e62-4862912a8ff7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:03.182316 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d4b2d71c-7baa-4194-8e62-4862912a8ff7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:03.191137 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d4b2d71c-7baa-4194-8e62-4862912a8ff7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.162663 Aug 28 01:27:03.191710 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1456/3134] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:03 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:04.211935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5807f7c1-e23e-4957-819a-af2c0c7df774 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:04.212565 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5807f7c1-e23e-4957-819a-af2c0c7df774 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:04.219889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5807f7c1-e23e-4957-819a-af2c0c7df774 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:04.220516 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5807f7c1-e23e-4957-819a-af2c0c7df774 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:04.220781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5807f7c1-e23e-4957-819a-af2c0c7df774 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:04.221965 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5807f7c1-e23e-4957-819a-af2c0c7df774 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:04.221965 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5807f7c1-e23e-4957-819a-af2c0c7df774 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:04.222351 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5807f7c1-e23e-4957-819a-af2c0c7df774 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:04.362059 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5807f7c1-e23e-4957-819a-af2c0c7df774 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:04.370470 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5807f7c1-e23e-4957-819a-af2c0c7df774 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.162409 Aug 28 01:27:04.371144 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1679/3135] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:04 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:05.395397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a99f9d83-ccd4-4c3a-a63a-4e3b0d339c67 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:05.396043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a99f9d83-ccd4-4c3a-a63a-4e3b0d339c67 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:05.402921 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a99f9d83-ccd4-4c3a-a63a-4e3b0d339c67 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:05.403696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a99f9d83-ccd4-4c3a-a63a-4e3b0d339c67 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:05.403870 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a99f9d83-ccd4-4c3a-a63a-4e3b0d339c67 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:05.404636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a99f9d83-ccd4-4c3a-a63a-4e3b0d339c67 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:05.405159 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a99f9d83-ccd4-4c3a-a63a-4e3b0d339c67 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:05.405589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a99f9d83-ccd4-4c3a-a63a-4e3b0d339c67 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:05.537249 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a99f9d83-ccd4-4c3a-a63a-4e3b0d339c67 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:05.545966 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a99f9d83-ccd4-4c3a-a63a-4e3b0d339c67 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.154501 Aug 28 01:27:05.546559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1457/3136] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:05 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:06.568177 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7e621716-5b01-4f28-8692-0d54bebe65af tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:06.568845 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7e621716-5b01-4f28-8692-0d54bebe65af tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:06.575594 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e621716-5b01-4f28-8692-0d54bebe65af tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:06.576075 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e621716-5b01-4f28-8692-0d54bebe65af tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:06.576640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e621716-5b01-4f28-8692-0d54bebe65af tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:06.577373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e621716-5b01-4f28-8692-0d54bebe65af tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:06.577808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e621716-5b01-4f28-8692-0d54bebe65af tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:06.578227 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7e621716-5b01-4f28-8692-0d54bebe65af tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:06.712367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7e621716-5b01-4f28-8692-0d54bebe65af tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:06.721177 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7e621716-5b01-4f28-8692-0d54bebe65af tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.156785 Aug 28 01:27:06.721758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1680/3137] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:06 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:07.743504 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5ce09c4f-b36d-4977-b7e2-cb33df7950ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:07.744193 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5ce09c4f-b36d-4977-b7e2-cb33df7950ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:07.751713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ce09c4f-b36d-4977-b7e2-cb33df7950ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:07.752202 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ce09c4f-b36d-4977-b7e2-cb33df7950ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:07.752664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ce09c4f-b36d-4977-b7e2-cb33df7950ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:07.753393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ce09c4f-b36d-4977-b7e2-cb33df7950ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:07.753843 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ce09c4f-b36d-4977-b7e2-cb33df7950ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:07.754268 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5ce09c4f-b36d-4977-b7e2-cb33df7950ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:07.900108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5ce09c4f-b36d-4977-b7e2-cb33df7950ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:07.909038 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5ce09c4f-b36d-4977-b7e2-cb33df7950ba tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.169908 Aug 28 01:27:07.909645 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1458/3138] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:07 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:08.933596 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a02ab8f6-f132-4195-a3f8-81b5cbea4a4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:08.934554 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a02ab8f6-f132-4195-a3f8-81b5cbea4a4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:08.942773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a02ab8f6-f132-4195-a3f8-81b5cbea4a4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:08.943126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a02ab8f6-f132-4195-a3f8-81b5cbea4a4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:08.943558 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a02ab8f6-f132-4195-a3f8-81b5cbea4a4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:08.944247 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a02ab8f6-f132-4195-a3f8-81b5cbea4a4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:08.944693 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a02ab8f6-f132-4195-a3f8-81b5cbea4a4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:08.945152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a02ab8f6-f132-4195-a3f8-81b5cbea4a4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:09.106359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a02ab8f6-f132-4195-a3f8-81b5cbea4a4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:09.116089 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a02ab8f6-f132-4195-a3f8-81b5cbea4a4f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.187891 Aug 28 01:27:09.116652 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1681/3139] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:08 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:10.140874 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bcfdcc5b-07c1-4f61-9174-4dc5582373e5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:10.141506 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bcfdcc5b-07c1-4f61-9174-4dc5582373e5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:10.148632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcfdcc5b-07c1-4f61-9174-4dc5582373e5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:10.149149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcfdcc5b-07c1-4f61-9174-4dc5582373e5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:10.149596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcfdcc5b-07c1-4f61-9174-4dc5582373e5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:10.150285 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcfdcc5b-07c1-4f61-9174-4dc5582373e5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:10.150730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcfdcc5b-07c1-4f61-9174-4dc5582373e5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:10.151173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bcfdcc5b-07c1-4f61-9174-4dc5582373e5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:10.294617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bcfdcc5b-07c1-4f61-9174-4dc5582373e5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:10.303297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bcfdcc5b-07c1-4f61-9174-4dc5582373e5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.167150 Aug 28 01:27:10.303876 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1459/3140] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:10 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:11.327117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a66d3ca2-0938-472d-a7be-532e0aff1cd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:11.327860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a66d3ca2-0938-472d-a7be-532e0aff1cd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:11.336186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a66d3ca2-0938-472d-a7be-532e0aff1cd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:11.336782 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a66d3ca2-0938-472d-a7be-532e0aff1cd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:11.337344 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a66d3ca2-0938-472d-a7be-532e0aff1cd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:11.338201 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a66d3ca2-0938-472d-a7be-532e0aff1cd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:11.338745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a66d3ca2-0938-472d-a7be-532e0aff1cd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:11.339305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a66d3ca2-0938-472d-a7be-532e0aff1cd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:11.476686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a66d3ca2-0938-472d-a7be-532e0aff1cd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:11.485228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a66d3ca2-0938-472d-a7be-532e0aff1cd2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.162852 Aug 28 01:27:11.485772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1682/3141] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:11 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:12.506717 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c9dc6ace-6a3c-4a20-bd0c-ddd0ba7206b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:12.507374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c9dc6ace-6a3c-4a20-bd0c-ddd0ba7206b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:12.514467 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9dc6ace-6a3c-4a20-bd0c-ddd0ba7206b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:12.514952 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9dc6ace-6a3c-4a20-bd0c-ddd0ba7206b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:12.515390 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9dc6ace-6a3c-4a20-bd0c-ddd0ba7206b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:12.516079 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9dc6ace-6a3c-4a20-bd0c-ddd0ba7206b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:12.516539 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9dc6ace-6a3c-4a20-bd0c-ddd0ba7206b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:12.516992 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9dc6ace-6a3c-4a20-bd0c-ddd0ba7206b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:12.654348 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c9dc6ace-6a3c-4a20-bd0c-ddd0ba7206b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:12.663943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c9dc6ace-6a3c-4a20-bd0c-ddd0ba7206b9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.161180 Aug 28 01:27:12.664546 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1460/3142] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:12 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:13.686187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-20324c2e-7617-416a-a948-539a198c5dfc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:13.686803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-20324c2e-7617-416a-a948-539a198c5dfc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:13.695619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-20324c2e-7617-416a-a948-539a198c5dfc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:13.696222 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-20324c2e-7617-416a-a948-539a198c5dfc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:13.696863 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-20324c2e-7617-416a-a948-539a198c5dfc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:13.697737 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-20324c2e-7617-416a-a948-539a198c5dfc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:13.698207 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-20324c2e-7617-416a-a948-539a198c5dfc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:13.698987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-20324c2e-7617-416a-a948-539a198c5dfc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:13.701870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=84 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:27:13.848841 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-20324c2e-7617-416a-a948-539a198c5dfc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:13.857710 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-20324c2e-7617-416a-a948-539a198c5dfc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.175398 Aug 28 01:27:13.858268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1683/3143] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:13 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:14.886805 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-727208a6-2c69-4faf-920f-a7decba815ab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:14.887491 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-727208a6-2c69-4faf-920f-a7decba815ab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:14.895392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-727208a6-2c69-4faf-920f-a7decba815ab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:14.895846 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-727208a6-2c69-4faf-920f-a7decba815ab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:14.896305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-727208a6-2c69-4faf-920f-a7decba815ab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:14.896987 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-727208a6-2c69-4faf-920f-a7decba815ab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:14.897472 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-727208a6-2c69-4faf-920f-a7decba815ab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:14.897892 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-727208a6-2c69-4faf-920f-a7decba815ab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:15.037099 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-727208a6-2c69-4faf-920f-a7decba815ab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:15.045703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-727208a6-2c69-4faf-920f-a7decba815ab tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.162968 Aug 28 01:27:15.046278 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1461/3144] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:14 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:16.068675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f9c5eab2-2750-40b9-b052-71630a676cb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:16.069334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f9c5eab2-2750-40b9-b052-71630a676cb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:16.076192 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f9c5eab2-2750-40b9-b052-71630a676cb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:16.077003 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f9c5eab2-2750-40b9-b052-71630a676cb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:16.077659 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f9c5eab2-2750-40b9-b052-71630a676cb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:16.078489 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f9c5eab2-2750-40b9-b052-71630a676cb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:16.079142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f9c5eab2-2750-40b9-b052-71630a676cb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:16.079816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f9c5eab2-2750-40b9-b052-71630a676cb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:16.265600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f9c5eab2-2750-40b9-b052-71630a676cb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:16.274431 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f9c5eab2-2750-40b9-b052-71630a676cb7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.209778 Aug 28 01:27:16.274987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1684/3145] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:16 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:17.296861 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-04f22d09-26c5-497e-b9e3-5d700234c995 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:17.297548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-04f22d09-26c5-497e-b9e3-5d700234c995 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:17.299331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=26 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:27:17.304951 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04f22d09-26c5-497e-b9e3-5d700234c995 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:17.305523 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04f22d09-26c5-497e-b9e3-5d700234c995 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:17.305958 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04f22d09-26c5-497e-b9e3-5d700234c995 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:17.306644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04f22d09-26c5-497e-b9e3-5d700234c995 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:17.307085 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04f22d09-26c5-497e-b9e3-5d700234c995 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:17.307501 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-04f22d09-26c5-497e-b9e3-5d700234c995 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:17.488406 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-04f22d09-26c5-497e-b9e3-5d700234c995 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:17.496800 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-04f22d09-26c5-497e-b9e3-5d700234c995 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.204390 Aug 28 01:27:17.497396 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1462/3146] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:17 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:18.518057 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e0d37cda-1941-43a8-9d93-ebf56764d3bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:18.518654 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e0d37cda-1941-43a8-9d93-ebf56764d3bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:18.525265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0d37cda-1941-43a8-9d93-ebf56764d3bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:18.525751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0d37cda-1941-43a8-9d93-ebf56764d3bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:18.526182 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0d37cda-1941-43a8-9d93-ebf56764d3bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:18.526906 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0d37cda-1941-43a8-9d93-ebf56764d3bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:18.527330 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0d37cda-1941-43a8-9d93-ebf56764d3bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:18.527753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e0d37cda-1941-43a8-9d93-ebf56764d3bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:18.660188 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e0d37cda-1941-43a8-9d93-ebf56764d3bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:18.668882 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e0d37cda-1941-43a8-9d93-ebf56764d3bd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.154820 Aug 28 01:27:18.669495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1685/3147] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:18 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:19.691601 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-377d2c95-0056-46d0-993f-4be5664a6099 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:19.692389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-377d2c95-0056-46d0-993f-4be5664a6099 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:19.699624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-377d2c95-0056-46d0-993f-4be5664a6099 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:19.700103 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-377d2c95-0056-46d0-993f-4be5664a6099 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:19.700534 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-377d2c95-0056-46d0-993f-4be5664a6099 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:19.701254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-377d2c95-0056-46d0-993f-4be5664a6099 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:19.701789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-377d2c95-0056-46d0-993f-4be5664a6099 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:19.702355 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-377d2c95-0056-46d0-993f-4be5664a6099 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:19.839598 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-377d2c95-0056-46d0-993f-4be5664a6099 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:19.848008 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-377d2c95-0056-46d0-993f-4be5664a6099 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.161353 Aug 28 01:27:19.848646 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1463/3148] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:19 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:20.869795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cf574b9f-8119-4c2f-a1b1-e3a12115522a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:20.870429 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cf574b9f-8119-4c2f-a1b1-e3a12115522a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:20.877270 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf574b9f-8119-4c2f-a1b1-e3a12115522a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:20.877754 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf574b9f-8119-4c2f-a1b1-e3a12115522a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:20.878172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf574b9f-8119-4c2f-a1b1-e3a12115522a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:20.878898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf574b9f-8119-4c2f-a1b1-e3a12115522a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:20.879305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf574b9f-8119-4c2f-a1b1-e3a12115522a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:20.879723 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cf574b9f-8119-4c2f-a1b1-e3a12115522a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:21.019473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-cf574b9f-8119-4c2f-a1b1-e3a12115522a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:21.031163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cf574b9f-8119-4c2f-a1b1-e3a12115522a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.165221 Aug 28 01:27:21.031746 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1686/3149] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:20 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:22.052055 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2fe74821-fd0d-4d20-842f-30679ff42cf2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:22.052731 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2fe74821-fd0d-4d20-842f-30679ff42cf2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:22.059664 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fe74821-fd0d-4d20-842f-30679ff42cf2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:22.060135 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fe74821-fd0d-4d20-842f-30679ff42cf2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:22.060589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fe74821-fd0d-4d20-842f-30679ff42cf2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:22.061329 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fe74821-fd0d-4d20-842f-30679ff42cf2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:22.061774 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fe74821-fd0d-4d20-842f-30679ff42cf2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:22.062166 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2fe74821-fd0d-4d20-842f-30679ff42cf2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:22.199205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2fe74821-fd0d-4d20-842f-30679ff42cf2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:22.207703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2fe74821-fd0d-4d20-842f-30679ff42cf2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.159581 Aug 28 01:27:22.208306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1464/3150] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:22 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:23.229799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5b55d41a-1ee3-469f-b18d-758e5de79e7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:23.230775 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5b55d41a-1ee3-469f-b18d-758e5de79e7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:23.238373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b55d41a-1ee3-469f-b18d-758e5de79e7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:23.238847 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b55d41a-1ee3-469f-b18d-758e5de79e7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:23.239274 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b55d41a-1ee3-469f-b18d-758e5de79e7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:23.239956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b55d41a-1ee3-469f-b18d-758e5de79e7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:23.240401 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b55d41a-1ee3-469f-b18d-758e5de79e7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:23.240945 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5b55d41a-1ee3-469f-b18d-758e5de79e7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:23.382619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5b55d41a-1ee3-469f-b18d-758e5de79e7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:23.391681 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5b55d41a-1ee3-469f-b18d-758e5de79e7f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.165666 Aug 28 01:27:23.392270 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1687/3151] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:23 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:24.413616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8b1bcb21-0745-48e4-8ef1-86c6ff6be7da tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:24.414255 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8b1bcb21-0745-48e4-8ef1-86c6ff6be7da tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:24.420798 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b1bcb21-0745-48e4-8ef1-86c6ff6be7da tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:24.421320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b1bcb21-0745-48e4-8ef1-86c6ff6be7da tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:24.421762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b1bcb21-0745-48e4-8ef1-86c6ff6be7da tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:24.422457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b1bcb21-0745-48e4-8ef1-86c6ff6be7da tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:24.423283 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b1bcb21-0745-48e4-8ef1-86c6ff6be7da tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:24.423700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b1bcb21-0745-48e4-8ef1-86c6ff6be7da tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:24.426305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=78 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:27:24.573163 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8b1bcb21-0745-48e4-8ef1-86c6ff6be7da tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:24.581809 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8b1bcb21-0745-48e4-8ef1-86c6ff6be7da tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.172186 Aug 28 01:27:24.582390 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1465/3152] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:24 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:25.604816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f95257ce-0f5f-44b3-8742-19047d9385e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:25.605490 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f95257ce-0f5f-44b3-8742-19047d9385e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:25.613511 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f95257ce-0f5f-44b3-8742-19047d9385e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:25.614052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f95257ce-0f5f-44b3-8742-19047d9385e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:25.614524 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f95257ce-0f5f-44b3-8742-19047d9385e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:25.615281 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f95257ce-0f5f-44b3-8742-19047d9385e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:25.615850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f95257ce-0f5f-44b3-8742-19047d9385e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:25.616281 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f95257ce-0f5f-44b3-8742-19047d9385e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:25.757904 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f95257ce-0f5f-44b3-8742-19047d9385e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:25.766597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f95257ce-0f5f-44b3-8742-19047d9385e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.166128 Aug 28 01:27:25.767192 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1688/3153] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:25 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:26.789598 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-546336be-ece0-4fc0-96df-a1bc036b7f74 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:26.790249 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-546336be-ece0-4fc0-96df-a1bc036b7f74 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:26.796987 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-546336be-ece0-4fc0-96df-a1bc036b7f74 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:26.797487 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-546336be-ece0-4fc0-96df-a1bc036b7f74 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:26.797951 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-546336be-ece0-4fc0-96df-a1bc036b7f74 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:26.798648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-546336be-ece0-4fc0-96df-a1bc036b7f74 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:26.799107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-546336be-ece0-4fc0-96df-a1bc036b7f74 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:26.799520 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-546336be-ece0-4fc0-96df-a1bc036b7f74 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:26.943578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-546336be-ece0-4fc0-96df-a1bc036b7f74 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:26.953572 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-546336be-ece0-4fc0-96df-a1bc036b7f74 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.168896 Aug 28 01:27:26.954477 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1466/3154] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:26 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:27.978245 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fc890f62-7fe3-4c5f-9e9e-da10bcd9fa99 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:27.978977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fc890f62-7fe3-4c5f-9e9e-da10bcd9fa99 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:27.988871 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc890f62-7fe3-4c5f-9e9e-da10bcd9fa99 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:27.989555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc890f62-7fe3-4c5f-9e9e-da10bcd9fa99 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:27.990117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc890f62-7fe3-4c5f-9e9e-da10bcd9fa99 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:27.991195 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc890f62-7fe3-4c5f-9e9e-da10bcd9fa99 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:27.991883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc890f62-7fe3-4c5f-9e9e-da10bcd9fa99 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:27.992512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fc890f62-7fe3-4c5f-9e9e-da10bcd9fa99 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:28.139907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fc890f62-7fe3-4c5f-9e9e-da10bcd9fa99 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:28.148448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fc890f62-7fe3-4c5f-9e9e-da10bcd9fa99 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.176031 Aug 28 01:27:28.149033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1689/3155] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:27 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:29.172972 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4ac332e3-5e81-48aa-a5e0-708745829e4d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:29.173591 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4ac332e3-5e81-48aa-a5e0-708745829e4d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:29.181854 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ac332e3-5e81-48aa-a5e0-708745829e4d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:29.182318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ac332e3-5e81-48aa-a5e0-708745829e4d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:29.182779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ac332e3-5e81-48aa-a5e0-708745829e4d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:29.183506 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ac332e3-5e81-48aa-a5e0-708745829e4d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:29.183958 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ac332e3-5e81-48aa-a5e0-708745829e4d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:29.184382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4ac332e3-5e81-48aa-a5e0-708745829e4d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:29.324168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4ac332e3-5e81-48aa-a5e0-708745829e4d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:29.332928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4ac332e3-5e81-48aa-a5e0-708745829e4d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.165519 Aug 28 01:27:29.333518 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1467/3156] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:29 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:30.085471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3eb9065f-0017-4a36-b11d-9880d25a9959 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:30.086118 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3eb9065f-0017-4a36-b11d-9880d25a9959 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:30.094211 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3eb9065f-0017-4a36-b11d-9880d25a9959 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:30.094975 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3eb9065f-0017-4a36-b11d-9880d25a9959 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:30.095638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3eb9065f-0017-4a36-b11d-9880d25a9959 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:30.096795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3eb9065f-0017-4a36-b11d-9880d25a9959 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:30.097348 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3eb9065f-0017-4a36-b11d-9880d25a9959 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:30.097939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3eb9065f-0017-4a36-b11d-9880d25a9959 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:30.123364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3eb9065f-0017-4a36-b11d-9880d25a9959 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2/ips" status: 200 len: 150 microversion: 2.1 time: 0.041775 Aug 28 01:27:30.123984 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1690/3157] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Aug 28 01:27:30 2022] GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2/ips => generated 150 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:27:30.353977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9189b0ef-fe73-491b-b829-24880e1c37fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:30.354584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9189b0ef-fe73-491b-b829-24880e1c37fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:30.362484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9189b0ef-fe73-491b-b829-24880e1c37fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:30.362972 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9189b0ef-fe73-491b-b829-24880e1c37fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:30.363401 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9189b0ef-fe73-491b-b829-24880e1c37fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:30.364105 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9189b0ef-fe73-491b-b829-24880e1c37fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:30.364552 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9189b0ef-fe73-491b-b829-24880e1c37fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:30.364989 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9189b0ef-fe73-491b-b829-24880e1c37fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:30.502159 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9189b0ef-fe73-491b-b829-24880e1c37fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:30.510845 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9189b0ef-fe73-491b-b829-24880e1c37fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.160770 Aug 28 01:27:30.511426 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1468/3158] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:30 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:31.538630 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-226a522a-5bc6-40ee-80bd-bb2a5c46d98f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:31.539234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-226a522a-5bc6-40ee-80bd-bb2a5c46d98f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:31.548402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-226a522a-5bc6-40ee-80bd-bb2a5c46d98f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:31.549029 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-226a522a-5bc6-40ee-80bd-bb2a5c46d98f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:31.549623 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-226a522a-5bc6-40ee-80bd-bb2a5c46d98f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:31.550404 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-226a522a-5bc6-40ee-80bd-bb2a5c46d98f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:31.550997 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-226a522a-5bc6-40ee-80bd-bb2a5c46d98f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:31.551496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-226a522a-5bc6-40ee-80bd-bb2a5c46d98f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:31.694381 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-226a522a-5bc6-40ee-80bd-bb2a5c46d98f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:31.706877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-226a522a-5bc6-40ee-80bd-bb2a5c46d98f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.173248 Aug 28 01:27:31.707708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1691/3159] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:31 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:32.729914 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-68708d4b-2836-4fbd-b153-05c00b58cddc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:32.730580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-68708d4b-2836-4fbd-b153-05c00b58cddc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:32.737954 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-68708d4b-2836-4fbd-b153-05c00b58cddc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:32.738420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-68708d4b-2836-4fbd-b153-05c00b58cddc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:32.738869 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-68708d4b-2836-4fbd-b153-05c00b58cddc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:32.739565 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-68708d4b-2836-4fbd-b153-05c00b58cddc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:32.740007 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-68708d4b-2836-4fbd-b153-05c00b58cddc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:32.740426 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-68708d4b-2836-4fbd-b153-05c00b58cddc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:32.880227 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-68708d4b-2836-4fbd-b153-05c00b58cddc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:32.890317 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-68708d4b-2836-4fbd-b153-05c00b58cddc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.164576 Aug 28 01:27:32.890922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1469/3160] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:32 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:27:33.913495 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-90286447-df75-4914-9dae-8dbf06aabc26 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:33.914155 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-90286447-df75-4914-9dae-8dbf06aabc26 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:33.917333 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=28 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:27:33.924568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90286447-df75-4914-9dae-8dbf06aabc26 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:33.925223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90286447-df75-4914-9dae-8dbf06aabc26 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:33.925753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90286447-df75-4914-9dae-8dbf06aabc26 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:33.926821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90286447-df75-4914-9dae-8dbf06aabc26 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:33.927451 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90286447-df75-4914-9dae-8dbf06aabc26 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:33.927868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-90286447-df75-4914-9dae-8dbf06aabc26 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:34.072123 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-90286447-df75-4914-9dae-8dbf06aabc26 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:34.081081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-90286447-df75-4914-9dae-8dbf06aabc26 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.172604 Aug 28 01:27:34.081663 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1692/3161] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:33 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:35.103810 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f76cef8d-b88a-446b-b4c9-5391f3ca6a9a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:35.104469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f76cef8d-b88a-446b-b4c9-5391f3ca6a9a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:35.111298 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f76cef8d-b88a-446b-b4c9-5391f3ca6a9a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:35.111772 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f76cef8d-b88a-446b-b4c9-5391f3ca6a9a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:35.112218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f76cef8d-b88a-446b-b4c9-5391f3ca6a9a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:35.112923 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f76cef8d-b88a-446b-b4c9-5391f3ca6a9a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:35.113389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f76cef8d-b88a-446b-b4c9-5391f3ca6a9a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:35.113823 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f76cef8d-b88a-446b-b4c9-5391f3ca6a9a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:35.253814 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f76cef8d-b88a-446b-b4c9-5391f3ca6a9a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:35.263080 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f76cef8d-b88a-446b-b4c9-5391f3ca6a9a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.163362 Aug 28 01:27:35.263654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1470/3162] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:35 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:36.284010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7d9c4631-7ec4-46ce-8f68-16576fb1c5c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:36.284600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7d9c4631-7ec4-46ce-8f68-16576fb1c5c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:36.292175 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7d9c4631-7ec4-46ce-8f68-16576fb1c5c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:36.292658 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7d9c4631-7ec4-46ce-8f68-16576fb1c5c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:36.293159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7d9c4631-7ec4-46ce-8f68-16576fb1c5c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:36.293853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7d9c4631-7ec4-46ce-8f68-16576fb1c5c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:36.294295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7d9c4631-7ec4-46ce-8f68-16576fb1c5c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:36.294722 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7d9c4631-7ec4-46ce-8f68-16576fb1c5c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:36.432139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7d9c4631-7ec4-46ce-8f68-16576fb1c5c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:36.441305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7d9c4631-7ec4-46ce-8f68-16576fb1c5c1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.161190 Aug 28 01:27:36.441887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1693/3163] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:36 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:37.462799 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-22a236e0-9228-4c72-aa38-02f0b890ad17 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:37.463473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-22a236e0-9228-4c72-aa38-02f0b890ad17 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:37.470355 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-22a236e0-9228-4c72-aa38-02f0b890ad17 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:37.470846 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-22a236e0-9228-4c72-aa38-02f0b890ad17 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:37.471282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-22a236e0-9228-4c72-aa38-02f0b890ad17 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:37.471975 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-22a236e0-9228-4c72-aa38-02f0b890ad17 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:37.472427 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-22a236e0-9228-4c72-aa38-02f0b890ad17 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:37.472861 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-22a236e0-9228-4c72-aa38-02f0b890ad17 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:37.609073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-22a236e0-9228-4c72-aa38-02f0b890ad17 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:37.618223 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-22a236e0-9228-4c72-aa38-02f0b890ad17 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.159340 Aug 28 01:27:37.618803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1471/3164] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:37 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:38.640008 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7c3abbe0-27e4-4529-99ea-eb84f084486f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:38.640767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7c3abbe0-27e4-4529-99ea-eb84f084486f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:38.647694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c3abbe0-27e4-4529-99ea-eb84f084486f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:38.648172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c3abbe0-27e4-4529-99ea-eb84f084486f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:38.648670 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c3abbe0-27e4-4529-99ea-eb84f084486f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:38.649431 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c3abbe0-27e4-4529-99ea-eb84f084486f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:38.649873 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c3abbe0-27e4-4529-99ea-eb84f084486f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:38.650291 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7c3abbe0-27e4-4529-99ea-eb84f084486f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:38.792050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-7c3abbe0-27e4-4529-99ea-eb84f084486f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:38.800763 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7c3abbe0-27e4-4529-99ea-eb84f084486f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.164751 Aug 28 01:27:38.801409 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1694/3165] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:38 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:39.823676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f283d0cc-7a98-457a-a689-708cebf0f7d9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:39.824280 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f283d0cc-7a98-457a-a689-708cebf0f7d9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:39.831206 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f283d0cc-7a98-457a-a689-708cebf0f7d9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:39.831693 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f283d0cc-7a98-457a-a689-708cebf0f7d9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:39.832127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f283d0cc-7a98-457a-a689-708cebf0f7d9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:39.832825 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f283d0cc-7a98-457a-a689-708cebf0f7d9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:39.833311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f283d0cc-7a98-457a-a689-708cebf0f7d9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:39.833793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f283d0cc-7a98-457a-a689-708cebf0f7d9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:39.969430 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f283d0cc-7a98-457a-a689-708cebf0f7d9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:39.978228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f283d0cc-7a98-457a-a689-708cebf0f7d9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.158520 Aug 28 01:27:39.978816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1472/3166] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:39 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:41.000547 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3e52d56e-bad0-4c18-bd4f-d59461fc020a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:41.001256 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3e52d56e-bad0-4c18-bd4f-d59461fc020a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:41.008236 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e52d56e-bad0-4c18-bd4f-d59461fc020a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:41.008720 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e52d56e-bad0-4c18-bd4f-d59461fc020a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:41.009181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e52d56e-bad0-4c18-bd4f-d59461fc020a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:41.009867 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e52d56e-bad0-4c18-bd4f-d59461fc020a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:41.010310 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e52d56e-bad0-4c18-bd4f-d59461fc020a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:41.010720 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3e52d56e-bad0-4c18-bd4f-d59461fc020a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:41.153730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3e52d56e-bad0-4c18-bd4f-d59461fc020a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:41.162394 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3e52d56e-bad0-4c18-bd4f-d59461fc020a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.165879 Aug 28 01:27:41.162980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1695/3167] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:40 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:42.184026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a1029520-74d6-43a9-aebc-06fcc52f6c31 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:42.184667 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a1029520-74d6-43a9-aebc-06fcc52f6c31 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:42.192484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a1029520-74d6-43a9-aebc-06fcc52f6c31 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:42.192992 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a1029520-74d6-43a9-aebc-06fcc52f6c31 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:42.193433 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a1029520-74d6-43a9-aebc-06fcc52f6c31 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:42.194157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a1029520-74d6-43a9-aebc-06fcc52f6c31 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:42.194610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a1029520-74d6-43a9-aebc-06fcc52f6c31 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:42.195044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a1029520-74d6-43a9-aebc-06fcc52f6c31 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:42.331786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a1029520-74d6-43a9-aebc-06fcc52f6c31 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:42.340731 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a1029520-74d6-43a9-aebc-06fcc52f6c31 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.160606 Aug 28 01:27:42.341346 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1473/3168] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:42 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:43.365014 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cea3ed12-69a7-4580-9f23-56a2f3d4ca2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:43.365652 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cea3ed12-69a7-4580-9f23-56a2f3d4ca2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:43.374756 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cea3ed12-69a7-4580-9f23-56a2f3d4ca2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:43.375156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cea3ed12-69a7-4580-9f23-56a2f3d4ca2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:43.375577 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cea3ed12-69a7-4580-9f23-56a2f3d4ca2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:43.376257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cea3ed12-69a7-4580-9f23-56a2f3d4ca2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:43.376713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cea3ed12-69a7-4580-9f23-56a2f3d4ca2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:43.377162 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cea3ed12-69a7-4580-9f23-56a2f3d4ca2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:43.520214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-cea3ed12-69a7-4580-9f23-56a2f3d4ca2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:43.531822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cea3ed12-69a7-4580-9f23-56a2f3d4ca2c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.171860 Aug 28 01:27:43.532617 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1696/3169] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:43 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:44.556136 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-74b8fb31-e055-4b59-9184-e21dfdfc5b24 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:44.556804 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-74b8fb31-e055-4b59-9184-e21dfdfc5b24 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:44.563975 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-74b8fb31-e055-4b59-9184-e21dfdfc5b24 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:44.564455 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-74b8fb31-e055-4b59-9184-e21dfdfc5b24 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:44.564883 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-74b8fb31-e055-4b59-9184-e21dfdfc5b24 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:44.565602 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-74b8fb31-e055-4b59-9184-e21dfdfc5b24 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:44.566046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-74b8fb31-e055-4b59-9184-e21dfdfc5b24 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:44.566462 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-74b8fb31-e055-4b59-9184-e21dfdfc5b24 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:44.705950 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-74b8fb31-e055-4b59-9184-e21dfdfc5b24 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:44.714867 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-74b8fb31-e055-4b59-9184-e21dfdfc5b24 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.162787 Aug 28 01:27:44.715432 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1474/3170] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:44 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:45.739499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8632c8f3-7120-4112-aea9-fbc02de97667 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:45.740355 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8632c8f3-7120-4112-aea9-fbc02de97667 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:45.748503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8632c8f3-7120-4112-aea9-fbc02de97667 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:45.748991 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8632c8f3-7120-4112-aea9-fbc02de97667 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:45.749438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8632c8f3-7120-4112-aea9-fbc02de97667 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:45.750124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8632c8f3-7120-4112-aea9-fbc02de97667 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:45.750565 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8632c8f3-7120-4112-aea9-fbc02de97667 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:45.750978 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8632c8f3-7120-4112-aea9-fbc02de97667 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:45.753520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=82 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:27:45.893317 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8632c8f3-7120-4112-aea9-fbc02de97667 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:45.901985 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8632c8f3-7120-4112-aea9-fbc02de97667 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.168157 Aug 28 01:27:45.902587 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1697/3171] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:45 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:46.924237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-15a97c93-d764-4383-a4c6-7951259ce40c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:46.924932 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-15a97c93-d764-4383-a4c6-7951259ce40c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:46.932741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-15a97c93-d764-4383-a4c6-7951259ce40c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:46.933288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-15a97c93-d764-4383-a4c6-7951259ce40c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:46.933813 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-15a97c93-d764-4383-a4c6-7951259ce40c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:46.934565 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-15a97c93-d764-4383-a4c6-7951259ce40c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:46.935060 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-15a97c93-d764-4383-a4c6-7951259ce40c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:46.935524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-15a97c93-d764-4383-a4c6-7951259ce40c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:46.967424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-26312faa-37b6-4a69-97d6-fa7e75a8f96e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:46.968068 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-26312faa-37b6-4a69-97d6-fa7e75a8f96e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:46.975288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26312faa-37b6-4a69-97d6-fa7e75a8f96e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:46.975765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26312faa-37b6-4a69-97d6-fa7e75a8f96e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:46.976223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26312faa-37b6-4a69-97d6-fa7e75a8f96e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:46.976918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26312faa-37b6-4a69-97d6-fa7e75a8f96e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:46.977403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26312faa-37b6-4a69-97d6-fa7e75a8f96e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:46.977822 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-26312faa-37b6-4a69-97d6-fa7e75a8f96e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:47.004223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-26312faa-37b6-4a69-97d6-fa7e75a8f96e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Going to try to terminate instance {{(pid=120525) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 28 01:27:47.074318 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-15a97c93-d764-4383-a4c6-7951259ce40c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:47.083033 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-15a97c93-d764-4383-a4c6-7951259ce40c tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.163093 Aug 28 01:27:47.083831 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1475/3172] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:46 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:47.093582 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-26312faa-37b6-4a69-97d6-fa7e75a8f96e tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "DELETE /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df" status: 204 len: 0 microversion: 2.1 time: 0.129648 Aug 28 01:27:47.094351 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1698/3173] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 01:27:46 2022] DELETE /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df => generated 0 bytes in 131 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 28 01:27:47.104028 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2774890b-e713-4c5e-84f4-810fb049a895 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:47.104646 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2774890b-e713-4c5e-84f4-810fb049a895 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:47.111481 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2774890b-e713-4c5e-84f4-810fb049a895 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:47.112028 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2774890b-e713-4c5e-84f4-810fb049a895 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:47.112466 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2774890b-e713-4c5e-84f4-810fb049a895 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:47.113212 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2774890b-e713-4c5e-84f4-810fb049a895 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:47.113676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2774890b-e713-4c5e-84f4-810fb049a895 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:47.114085 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2774890b-e713-4c5e-84f4-810fb049a895 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:47.248443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2774890b-e713-4c5e-84f4-810fb049a895 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:47.256387 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2774890b-e713-4c5e-84f4-810fb049a895 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df" status: 200 len: 1672 microversion: 2.1 time: 0.156161 Aug 28 01:27:47.256975 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1476/3174] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:47 2022] GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df => generated 1672 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:48.108692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-05b5e2ce-6f2d-4e8e-824b-58f54ea10844 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:48.109414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-05b5e2ce-6f2d-4e8e-824b-58f54ea10844 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:48.117111 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05b5e2ce-6f2d-4e8e-824b-58f54ea10844 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:48.117597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05b5e2ce-6f2d-4e8e-824b-58f54ea10844 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:48.117997 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05b5e2ce-6f2d-4e8e-824b-58f54ea10844 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:48.118692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05b5e2ce-6f2d-4e8e-824b-58f54ea10844 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:48.119239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05b5e2ce-6f2d-4e8e-824b-58f54ea10844 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:48.119758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05b5e2ce-6f2d-4e8e-824b-58f54ea10844 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:48.271031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-05b5e2ce-6f2d-4e8e-824b-58f54ea10844 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:48.279061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6782a4db-cf18-4160-b3dc-583bb30fdcce tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:48.279701 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6782a4db-cf18-4160-b3dc-583bb30fdcce tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:48.280449 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-05b5e2ce-6f2d-4e8e-824b-58f54ea10844 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.178367 Aug 28 01:27:48.280994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1699/3175] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:48 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:48.282463 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=30 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:27:48.288137 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6782a4db-cf18-4160-b3dc-583bb30fdcce tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:48.288633 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6782a4db-cf18-4160-b3dc-583bb30fdcce tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:48.289093 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6782a4db-cf18-4160-b3dc-583bb30fdcce tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:48.289805 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6782a4db-cf18-4160-b3dc-583bb30fdcce tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:48.290259 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6782a4db-cf18-4160-b3dc-583bb30fdcce tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:48.290688 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6782a4db-cf18-4160-b3dc-583bb30fdcce tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:48.429127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6782a4db-cf18-4160-b3dc-583bb30fdcce tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:48.441705 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6782a4db-cf18-4160-b3dc-583bb30fdcce tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df" status: 200 len: 1672 microversion: 2.1 time: 0.166570 Aug 28 01:27:48.442549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1477/3176] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:48 2022] GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df => generated 1672 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:49.302192 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9b50e01e-347c-4d66-85cb-e0b15d56d118 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:49.302843 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9b50e01e-347c-4d66-85cb-e0b15d56d118 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:49.309974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b50e01e-347c-4d66-85cb-e0b15d56d118 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:49.310469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b50e01e-347c-4d66-85cb-e0b15d56d118 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:49.310907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b50e01e-347c-4d66-85cb-e0b15d56d118 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:49.311688 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b50e01e-347c-4d66-85cb-e0b15d56d118 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:49.312421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b50e01e-347c-4d66-85cb-e0b15d56d118 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:49.312838 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b50e01e-347c-4d66-85cb-e0b15d56d118 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:49.466145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1314b9ca-22f6-4a56-b4de-7f6f92fd1484 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:49.466879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1314b9ca-22f6-4a56-b4de-7f6f92fd1484 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:49.473741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1314b9ca-22f6-4a56-b4de-7f6f92fd1484 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:49.474227 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1314b9ca-22f6-4a56-b4de-7f6f92fd1484 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:49.474730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1314b9ca-22f6-4a56-b4de-7f6f92fd1484 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:49.475456 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1314b9ca-22f6-4a56-b4de-7f6f92fd1484 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:49.476110 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1314b9ca-22f6-4a56-b4de-7f6f92fd1484 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:49.476731 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1314b9ca-22f6-4a56-b4de-7f6f92fd1484 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:49.550378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9b50e01e-347c-4d66-85cb-e0b15d56d118 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:49.558962 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9b50e01e-347c-4d66-85cb-e0b15d56d118 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.260590 Aug 28 01:27:49.559540 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1700/3177] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:49 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:49.581804 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1314b9ca-22f6-4a56-b4de-7f6f92fd1484 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:49.589423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1314b9ca-22f6-4a56-b4de-7f6f92fd1484 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df" status: 200 len: 1605 microversion: 2.1 time: 0.128297 Aug 28 01:27:49.597212 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1478/3178] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:49 2022] GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df => generated 1605 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:49.863550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-e0e1f120-ef42-4e7e-99d2-b2c3db7c016b req-ff2347a1-4844-45d2-833f-2d99816cff1d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "004da63e-f25e-4dcc-a519-d253e534d6df", "name": "network-vif-deleted", "tag": "417c327c-0418-496e-87db-69ace013e663"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:27:49.874888 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-e0e1f120-ef42-4e7e-99d2-b2c3db7c016b req-ff2347a1-4844-45d2-833f-2d99816cff1d service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:49.875503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-e0e1f120-ef42-4e7e-99d2-b2c3db7c016b req-ff2347a1-4844-45d2-833f-2d99816cff1d service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:49.876024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-e0e1f120-ef42-4e7e-99d2-b2c3db7c016b req-ff2347a1-4844-45d2-833f-2d99816cff1d service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:49.887815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-e0e1f120-ef42-4e7e-99d2-b2c3db7c016b req-ff2347a1-4844-45d2-833f-2d99816cff1d service nova] Creating event network-vif-deleted:417c327c-0418-496e-87db-69ace013e663 for instance 004da63e-f25e-4dcc-a519-d253e534d6df on hv-cinder-maste Aug 28 01:27:49.893832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-e0e1f120-ef42-4e7e-99d2-b2c3db7c016b req-ff2347a1-4844-45d2-833f-2d99816cff1d service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034092 Aug 28 01:27:49.894666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1701/3179] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:27:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:27:50.581184 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4c6d9f6c-7676-4500-9a3e-c30d8dce6549 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:50.581803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4c6d9f6c-7676-4500-9a3e-c30d8dce6549 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:50.588493 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c6d9f6c-7676-4500-9a3e-c30d8dce6549 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:50.589055 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c6d9f6c-7676-4500-9a3e-c30d8dce6549 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:50.589456 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c6d9f6c-7676-4500-9a3e-c30d8dce6549 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:50.590305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c6d9f6c-7676-4500-9a3e-c30d8dce6549 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:50.590758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c6d9f6c-7676-4500-9a3e-c30d8dce6549 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:50.591216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4c6d9f6c-7676-4500-9a3e-c30d8dce6549 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:50.614302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-19b931ad-c81c-4449-a7f2-37046c0a49f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:50.615743 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-19b931ad-c81c-4449-a7f2-37046c0a49f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:50.623456 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-19b931ad-c81c-4449-a7f2-37046c0a49f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:50.624227 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-19b931ad-c81c-4449-a7f2-37046c0a49f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:50.624877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-19b931ad-c81c-4449-a7f2-37046c0a49f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:50.625811 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-19b931ad-c81c-4449-a7f2-37046c0a49f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:50.626436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-19b931ad-c81c-4449-a7f2-37046c0a49f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:50.627028 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-19b931ad-c81c-4449-a7f2-37046c0a49f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:50.674365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-19b931ad-c81c-4449-a7f2-37046c0a49f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:50.678795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-19b931ad-c81c-4449-a7f2-37046c0a49f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:50.679507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-19b931ad-c81c-4449-a7f2-37046c0a49f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:50.686101 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.objects.instance [None req-19b931ad-c81c-4449-a7f2-37046c0a49f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lazy-loading 'fault' on Instance uuid 004da63e-f25e-4dcc-a519-d253e534d6df {{(pid=120525) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:27:50.743443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-19b931ad-c81c-4449-a7f2-37046c0a49f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3de5d68f67784e33bdcd19a9cdfed744', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2348c1ddbec1476bacb194214794015d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:50.752827 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-19b931ad-c81c-4449-a7f2-37046c0a49f1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df" status: 200 len: 1327 microversion: 2.1 time: 0.144110 Aug 28 01:27:50.765183 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1702/3180] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:50 2022] GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df => generated 1327 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:27:50.782512 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4c6d9f6c-7676-4500-9a3e-c30d8dce6549 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:50.792194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4c6d9f6c-7676-4500-9a3e-c30d8dce6549 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.214856 Aug 28 01:27:50.793325 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1479/3181] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:50 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:51.775124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e6104cf6-4267-474d-bd9c-5e02fa770c28 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:51.775834 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e6104cf6-4267-474d-bd9c-5e02fa770c28 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:51.782870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6104cf6-4267-474d-bd9c-5e02fa770c28 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:51.783357 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6104cf6-4267-474d-bd9c-5e02fa770c28 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:51.783783 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6104cf6-4267-474d-bd9c-5e02fa770c28 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:51.784483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6104cf6-4267-474d-bd9c-5e02fa770c28 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:51.784928 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6104cf6-4267-474d-bd9c-5e02fa770c28 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:51.785380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e6104cf6-4267-474d-bd9c-5e02fa770c28 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:51.799779 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-e6104cf6-4267-474d-bd9c-5e02fa770c28 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] HTTP exception thrown: Instance 004da63e-f25e-4dcc-a519-d253e534d6df could not be found. Aug 28 01:27:51.801483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e6104cf6-4267-474d-bd9c-5e02fa770c28 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Request method failure captured: Aug 28 01:27:51.801483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df HTTP/1.1 Aug 28 01:27:51.801483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:27:51.801483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:27:51.801483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:27:51.801483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:27:51.801483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:27:51.801483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:27:51.801483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:27:51.801483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:27:51.801483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCrzmE0DQ9Mzy06OSRTD7Zc_5zbHJmgF9lrNWigauhfXdKgrk3t1iKQIvc72yKH8l7izSVSee0mW2vlsUYm08FWc-PX6YtIoZHTUEF5-2Kw4nOfJw6h6qrTA-JveSh59xEFSntY-alln8RdPrN4NOCHaeIkj1safHIIs1D9ZkiRfDOzw Aug 28 01:27:51.801483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:27:51.801483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:27:51.801483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:27:51.801483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:27:51.801483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:27:51.801483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:27:51.801483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 2348c1ddbec1476bacb194214794015d Aug 28 01:27:51.801483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-AttachVolumeTestJSON-192914243 Aug 28 01:27:51.801483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:27:51.801483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:27:51.803388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_2348c1ddbec1476bacb194214794015d", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2348c1ddbec1476bacb194214794015d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2348c1ddbec1476bacb194214794015d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/2348c1ddbec1476bacb194214794015d"}]}] Aug 28 01:27:51.803388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-AttachVolumeTestJSON-192914243 Aug 28 01:27:51.803388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 2348c1ddbec1476bacb194214794015d Aug 28 01:27:51.803388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-AttachVolumeTestJSON-192914243 Aug 28 01:27:51.803388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-AttachVolumeTestJSON-192914243-project Aug 28 01:27:51.803388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:27:51.803388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:27:51.803388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: 3de5d68f67784e33bdcd19a9cdfed744 Aug 28 01:27:51.803388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-AttachVolumeTestJSON-192914243-project Aug 28 01:27:51.803388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 01:27:51.803388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Instance 004da63e-f25e-4dcc-a519-d253e534d6df could not be found. Aug 28 01:27:51.803388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'id': '004da63e-f25e-4dcc-a519-d253e534d6df'} Aug 28 01:27:51.804686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:27:51.804686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:27:51.804686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:27:51.804686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:27:51.804686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:27:51.804686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:27:51.804686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:27:51.804686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:27:51.804686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:27:51.804686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:27:51.804686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 004da63e-f25e-4dcc-a519-d253e534d6df could not be found. Aug 28 01:27:51.804686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:27:51.804686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:27:51.804686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:27:51.804686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:27:51.804686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:27:51.804686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:27:51.804686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:27:51.804686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:27:51.804686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:27:51.806691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:27:51.806691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 01:27:51.806691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 01:27:51.806691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:27:51.806691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:27:51.806691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:27:51.806691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:27:51.806691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 004da63e-f25e-4dcc-a519-d253e534d6df could not be found. Aug 28 01:27:51.806691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:27:51.806691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:27:51.806691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:27:51.806691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:27:51.806691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:27:51.806691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:27:51.806691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:27:51.806691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:27:51.806691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 004da63e-f25e-4dcc-a519-d253e534d6df could not be found. Aug 28 01:27:51.806691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:27:51.808647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e6104cf6-4267-474d-bd9c-5e02fa770c28 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Returning 404 to user: Instance 004da63e-f25e-4dcc-a519-d253e534d6df could not be found. {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:27:51.808647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e6104cf6-4267-474d-bd9c-5e02fa770c28 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df" status: 404 len: 111 microversion: 2.1 time: 0.031862 Aug 28 01:27:51.808647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1703/3182] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:51 2022] GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:27:51.813830 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1dfe4916-c768-4132-a830-282ddda692ed tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '.version_select at 0x7f0e6e814b80>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:51.815964 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1beb7eed-eec2-4eec-8351-270f26e38f85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:51.816550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1beb7eed-eec2-4eec-8351-270f26e38f85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:51.822831 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1beb7eed-eec2-4eec-8351-270f26e38f85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:51.823305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1beb7eed-eec2-4eec-8351-270f26e38f85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:51.823742 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1beb7eed-eec2-4eec-8351-270f26e38f85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:51.824463 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1beb7eed-eec2-4eec-8351-270f26e38f85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:51.824887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1beb7eed-eec2-4eec-8351-270f26e38f85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:51.825336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1beb7eed-eec2-4eec-8351-270f26e38f85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:51.836574 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1dfe4916-c768-4132-a830-282ddda692ed tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1248497303" status: 202 len: 0 microversion: 2.1 time: 0.027235 Aug 28 01:27:51.837408 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1480/3183] 10.222.0.42 () {58 vars in 1221 bytes} [Sun Aug 28 01:27:51 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1248497303 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 01:27:52.017686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1beb7eed-eec2-4eec-8351-270f26e38f85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:52.026767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1beb7eed-eec2-4eec-8351-270f26e38f85 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.214682 Aug 28 01:27:52.027521 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1704/3184] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:51 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:52.584698 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7f59c881-686d-444f-9f9f-8e9cf00dc8b1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:52.585320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7f59c881-686d-444f-9f9f-8e9cf00dc8b1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:52.593389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7f59c881-686d-444f-9f9f-8e9cf00dc8b1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:52.593873 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7f59c881-686d-444f-9f9f-8e9cf00dc8b1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:52.594301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7f59c881-686d-444f-9f9f-8e9cf00dc8b1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:52.595000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7f59c881-686d-444f-9f9f-8e9cf00dc8b1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:52.595452 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7f59c881-686d-444f-9f9f-8e9cf00dc8b1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:52.595868 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7f59c881-686d-444f-9f9f-8e9cf00dc8b1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:52.610096 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-7f59c881-686d-444f-9f9f-8e9cf00dc8b1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] HTTP exception thrown: Instance 004da63e-f25e-4dcc-a519-d253e534d6df could not be found. Aug 28 01:27:52.611811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7f59c881-686d-444f-9f9f-8e9cf00dc8b1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Request method failure captured: Aug 28 01:27:52.611811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: DELETE /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df HTTP/1.1 Aug 28 01:27:52.611811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:27:52.611811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:27:52.611811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:27:52.611811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:27:52.611811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:27:52.611811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:27:52.611811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:27:52.611811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:27:52.611811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCrzmE0DQ9Mzy06OSRTD7Zc_5zbHJmgF9lrNWigauhfXdKgrk3t1iKQIvc72yKH8l7izSVSee0mW2vlsUYm08FWc-PX6YtIoZHTUEF5-2Kw4nOfJw6h6qrTA-JveSh59xEFSntY-alln8RdPrN4NOCHaeIkj1safHIIs1D9ZkiRfDOzw Aug 28 01:27:52.611811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:27:52.611811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:27:52.611811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:27:52.611811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:27:52.611811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:27:52.611811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:27:52.611811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 2348c1ddbec1476bacb194214794015d Aug 28 01:27:52.611811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-AttachVolumeTestJSON-192914243 Aug 28 01:27:52.611811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:27:52.611811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:27:52.613681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_2348c1ddbec1476bacb194214794015d", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2348c1ddbec1476bacb194214794015d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2348c1ddbec1476bacb194214794015d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/2348c1ddbec1476bacb194214794015d"}]}] Aug 28 01:27:52.613681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-AttachVolumeTestJSON-192914243 Aug 28 01:27:52.613681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 2348c1ddbec1476bacb194214794015d Aug 28 01:27:52.613681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-AttachVolumeTestJSON-192914243 Aug 28 01:27:52.613681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-AttachVolumeTestJSON-192914243-project Aug 28 01:27:52.613681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:27:52.613681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:27:52.613681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: 3de5d68f67784e33bdcd19a9cdfed744 Aug 28 01:27:52.613681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-AttachVolumeTestJSON-192914243-project Aug 28 01:27:52.613681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:27:52.613681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Instance 004da63e-f25e-4dcc-a519-d253e534d6df could not be found. Aug 28 01:27:52.613681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'id': '004da63e-f25e-4dcc-a519-d253e534d6df'} Aug 28 01:27:52.614670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:27:52.614670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:27:52.614670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:27:52.614670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:27:52.614670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:27:52.614670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:27:52.614670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:27:52.614670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:27:52.614670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:27:52.614670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:27:52.614670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 004da63e-f25e-4dcc-a519-d253e534d6df could not be found. Aug 28 01:27:52.614670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:27:52.614670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:27:52.614670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:27:52.614670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:27:52.614670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:27:52.614670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:27:52.614670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:27:52.614670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:27:52.614670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:27:52.616237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:27:52.616237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 28 01:27:52.616237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 28 01:27:52.616237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 28 01:27:52.616237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 28 01:27:52.616237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:27:52.616237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:27:52.616237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:27:52.616237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:27:52.616237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 004da63e-f25e-4dcc-a519-d253e534d6df could not be found. Aug 28 01:27:52.616237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:27:52.616237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:27:52.616237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:27:52.616237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:27:52.616237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:27:52.616237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:27:52.616237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:27:52.616237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:27:52.616237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 004da63e-f25e-4dcc-a519-d253e534d6df could not be found. Aug 28 01:27:52.616237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:27:52.617753 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7f59c881-686d-444f-9f9f-8e9cf00dc8b1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Returning 404 to user: Instance 004da63e-f25e-4dcc-a519-d253e534d6df could not be found. {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:27:52.617753 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7f59c881-686d-444f-9f9f-8e9cf00dc8b1 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "DELETE /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df" status: 404 len: 111 microversion: 2.1 time: 0.033621 Aug 28 01:27:52.617753 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1481/3185] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 01:27:52 2022] DELETE /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:27:52.622517 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-14174e18-899f-4116-b32d-f595ab9eac0a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:52.623128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-14174e18-899f-4116-b32d-f595ab9eac0a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: 004da63e-f25e-4dcc-a519-d253e534d6df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:52.629701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14174e18-899f-4116-b32d-f595ab9eac0a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:52.630191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14174e18-899f-4116-b32d-f595ab9eac0a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:52.630623 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14174e18-899f-4116-b32d-f595ab9eac0a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:52.631319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14174e18-899f-4116-b32d-f595ab9eac0a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:52.631767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14174e18-899f-4116-b32d-f595ab9eac0a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:52.632191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-14174e18-899f-4116-b32d-f595ab9eac0a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:52.646516 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-14174e18-899f-4116-b32d-f595ab9eac0a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] HTTP exception thrown: Instance 004da63e-f25e-4dcc-a519-d253e534d6df could not be found. Aug 28 01:27:52.648064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-14174e18-899f-4116-b32d-f595ab9eac0a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Request method failure captured: Aug 28 01:27:52.648064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df HTTP/1.1 Aug 28 01:27:52.648064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:27:52.648064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:27:52.648064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:27:52.648064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:27:52.648064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:27:52.648064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:27:52.648064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:27:52.648064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:27:52.648064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCrzmE0DQ9Mzy06OSRTD7Zc_5zbHJmgF9lrNWigauhfXdKgrk3t1iKQIvc72yKH8l7izSVSee0mW2vlsUYm08FWc-PX6YtIoZHTUEF5-2Kw4nOfJw6h6qrTA-JveSh59xEFSntY-alln8RdPrN4NOCHaeIkj1safHIIs1D9ZkiRfDOzw Aug 28 01:27:52.648064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:27:52.648064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:27:52.648064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:27:52.648064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:27:52.648064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:27:52.648064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:27:52.648064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 2348c1ddbec1476bacb194214794015d Aug 28 01:27:52.648064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-AttachVolumeTestJSON-192914243 Aug 28 01:27:52.648064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:27:52.648064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:27:52.649974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_2348c1ddbec1476bacb194214794015d", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2348c1ddbec1476bacb194214794015d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2348c1ddbec1476bacb194214794015d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/2348c1ddbec1476bacb194214794015d"}]}] Aug 28 01:27:52.649974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-AttachVolumeTestJSON-192914243 Aug 28 01:27:52.649974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 2348c1ddbec1476bacb194214794015d Aug 28 01:27:52.649974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-AttachVolumeTestJSON-192914243 Aug 28 01:27:52.649974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-AttachVolumeTestJSON-192914243-project Aug 28 01:27:52.649974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:27:52.649974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:27:52.649974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: 3de5d68f67784e33bdcd19a9cdfed744 Aug 28 01:27:52.649974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-AttachVolumeTestJSON-192914243-project Aug 28 01:27:52.649974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 01:27:52.649974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Instance 004da63e-f25e-4dcc-a519-d253e534d6df could not be found. Aug 28 01:27:52.649974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'id': '004da63e-f25e-4dcc-a519-d253e534d6df'} Aug 28 01:27:52.650862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:27:52.650862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:27:52.650862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:27:52.650862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:27:52.650862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:27:52.650862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:27:52.650862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:27:52.650862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:27:52.650862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:27:52.650862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:27:52.650862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 004da63e-f25e-4dcc-a519-d253e534d6df could not be found. Aug 28 01:27:52.650862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:27:52.650862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:27:52.650862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:27:52.650862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:27:52.650862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:27:52.650862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:27:52.650862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:27:52.650862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:27:52.650862 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:27:52.652302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:27:52.652302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 01:27:52.652302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 01:27:52.652302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:27:52.652302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:27:52.652302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:27:52.652302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:27:52.652302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 004da63e-f25e-4dcc-a519-d253e534d6df could not be found. Aug 28 01:27:52.652302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:27:52.652302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:27:52.652302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:27:52.652302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:27:52.652302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:27:52.652302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:27:52.652302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:27:52.652302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:27:52.652302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 004da63e-f25e-4dcc-a519-d253e534d6df could not be found. Aug 28 01:27:52.652302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:27:52.653830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-14174e18-899f-4116-b32d-f595ab9eac0a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Returning 404 to user: Instance 004da63e-f25e-4dcc-a519-d253e534d6df could not be found. {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:27:52.653830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-14174e18-899f-4116-b32d-f595ab9eac0a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df" status: 404 len: 111 microversion: 2.1 time: 0.031078 Aug 28 01:27:52.653830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1705/3186] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:52 2022] GET /compute/v2.1/servers/004da63e-f25e-4dcc-a519-d253e534d6df => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 28 01:27:53.048767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f9346599-911b-411a-afe5-410b37c58e6e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:53.049465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f9346599-911b-411a-afe5-410b37c58e6e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:53.056218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9346599-911b-411a-afe5-410b37c58e6e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:53.056589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9346599-911b-411a-afe5-410b37c58e6e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:53.057072 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9346599-911b-411a-afe5-410b37c58e6e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:53.057775 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9346599-911b-411a-afe5-410b37c58e6e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:53.058219 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9346599-911b-411a-afe5-410b37c58e6e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:53.058677 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f9346599-911b-411a-afe5-410b37c58e6e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:53.198942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f9346599-911b-411a-afe5-410b37c58e6e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:53.207684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f9346599-911b-411a-afe5-410b37c58e6e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.162965 Aug 28 01:27:53.208254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1482/3187] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:53 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:54.229882 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0dbbd81b-c900-4a7e-b867-c0f9e550c840 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:54.230642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0dbbd81b-c900-4a7e-b867-c0f9e550c840 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:54.237728 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0dbbd81b-c900-4a7e-b867-c0f9e550c840 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:54.238204 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0dbbd81b-c900-4a7e-b867-c0f9e550c840 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:54.238634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0dbbd81b-c900-4a7e-b867-c0f9e550c840 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:54.239321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0dbbd81b-c900-4a7e-b867-c0f9e550c840 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:54.239764 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0dbbd81b-c900-4a7e-b867-c0f9e550c840 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:54.240180 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0dbbd81b-c900-4a7e-b867-c0f9e550c840 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:54.381833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0dbbd81b-c900-4a7e-b867-c0f9e550c840 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:54.392181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0dbbd81b-c900-4a7e-b867-c0f9e550c840 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.166186 Aug 28 01:27:54.392758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1706/3188] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:54 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:55.420090 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-236a0efb-8211-4bbf-82c2-a2233cb67fd0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:55.420744 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-236a0efb-8211-4bbf-82c2-a2233cb67fd0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:55.428026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-236a0efb-8211-4bbf-82c2-a2233cb67fd0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:55.428504 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-236a0efb-8211-4bbf-82c2-a2233cb67fd0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:55.428924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-236a0efb-8211-4bbf-82c2-a2233cb67fd0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:55.429653 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-236a0efb-8211-4bbf-82c2-a2233cb67fd0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:55.430101 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-236a0efb-8211-4bbf-82c2-a2233cb67fd0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:55.430519 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-236a0efb-8211-4bbf-82c2-a2233cb67fd0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:55.433094 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=98 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:27:55.569072 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-236a0efb-8211-4bbf-82c2-a2233cb67fd0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:55.581784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-236a0efb-8211-4bbf-82c2-a2233cb67fd0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.166902 Aug 28 01:27:55.582605 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1483/3189] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:55 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:56.605659 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-658b6dbe-9afa-4ba2-a0a7-a1ca2127a26f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:56.606321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-658b6dbe-9afa-4ba2-a0a7-a1ca2127a26f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:56.613067 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-658b6dbe-9afa-4ba2-a0a7-a1ca2127a26f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:56.613581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-658b6dbe-9afa-4ba2-a0a7-a1ca2127a26f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:56.614017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-658b6dbe-9afa-4ba2-a0a7-a1ca2127a26f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:56.614706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-658b6dbe-9afa-4ba2-a0a7-a1ca2127a26f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:56.615153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-658b6dbe-9afa-4ba2-a0a7-a1ca2127a26f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:56.615571 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-658b6dbe-9afa-4ba2-a0a7-a1ca2127a26f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:56.756811 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-658b6dbe-9afa-4ba2-a0a7-a1ca2127a26f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:56.765518 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-658b6dbe-9afa-4ba2-a0a7-a1ca2127a26f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.163769 Aug 28 01:27:56.766306 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1707/3190] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:56 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:57.787291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ef28f474-4a23-4c8c-ac5c-8042a7d15649 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:57.787950 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ef28f474-4a23-4c8c-ac5c-8042a7d15649 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:57.794871 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ef28f474-4a23-4c8c-ac5c-8042a7d15649 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:57.795363 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ef28f474-4a23-4c8c-ac5c-8042a7d15649 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:57.795796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ef28f474-4a23-4c8c-ac5c-8042a7d15649 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:57.796507 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ef28f474-4a23-4c8c-ac5c-8042a7d15649 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:57.796951 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ef28f474-4a23-4c8c-ac5c-8042a7d15649 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:57.797397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ef28f474-4a23-4c8c-ac5c-8042a7d15649 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:57.930176 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ef28f474-4a23-4c8c-ac5c-8042a7d15649 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:57.939316 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ef28f474-4a23-4c8c-ac5c-8042a7d15649 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.156020 Aug 28 01:27:57.939899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1484/3191] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:57 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:27:58.961186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0c978db2-5b03-48cc-b3b3-8d590975cfe2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:27:58.961824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0c978db2-5b03-48cc-b3b3-8d590975cfe2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:27:58.969939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c978db2-5b03-48cc-b3b3-8d590975cfe2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:58.970423 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c978db2-5b03-48cc-b3b3-8d590975cfe2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:58.970857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c978db2-5b03-48cc-b3b3-8d590975cfe2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:58.971565 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c978db2-5b03-48cc-b3b3-8d590975cfe2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:27:58.971991 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c978db2-5b03-48cc-b3b3-8d590975cfe2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:27:58.972408 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0c978db2-5b03-48cc-b3b3-8d590975cfe2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:27:59.113964 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0c978db2-5b03-48cc-b3b3-8d590975cfe2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:27:59.121307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0c978db2-5b03-48cc-b3b3-8d590975cfe2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.164187 Aug 28 01:27:59.121931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1708/3192] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:27:58 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:00.143065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ed9cce6c-a8ff-4ddb-b644-af4bf884dab9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:00.143678 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ed9cce6c-a8ff-4ddb-b644-af4bf884dab9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:00.150561 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ed9cce6c-a8ff-4ddb-b644-af4bf884dab9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:00.151062 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ed9cce6c-a8ff-4ddb-b644-af4bf884dab9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:00.151487 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ed9cce6c-a8ff-4ddb-b644-af4bf884dab9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:00.152197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ed9cce6c-a8ff-4ddb-b644-af4bf884dab9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:00.152741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ed9cce6c-a8ff-4ddb-b644-af4bf884dab9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:00.153475 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ed9cce6c-a8ff-4ddb-b644-af4bf884dab9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:00.303039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ed9cce6c-a8ff-4ddb-b644-af4bf884dab9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:00.315073 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ed9cce6c-a8ff-4ddb-b644-af4bf884dab9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.175706 Aug 28 01:28:00.315792 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1485/3193] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:00 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:01.342230 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-b5af7609-2207-49be-86b7-c9dc3e4e635b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:01.342887 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-b5af7609-2207-49be-86b7-c9dc3e4e635b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:01.353292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5af7609-2207-49be-86b7-c9dc3e4e635b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:01.353978 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5af7609-2207-49be-86b7-c9dc3e4e635b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:01.354540 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5af7609-2207-49be-86b7-c9dc3e4e635b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:01.355546 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5af7609-2207-49be-86b7-c9dc3e4e635b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:01.356184 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5af7609-2207-49be-86b7-c9dc3e4e635b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:01.356708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-b5af7609-2207-49be-86b7-c9dc3e4e635b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:01.499176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-b5af7609-2207-49be-86b7-c9dc3e4e635b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:01.507735 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-b5af7609-2207-49be-86b7-c9dc3e4e635b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.171469 Aug 28 01:28:01.508319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1709/3194] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:01 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:02.536171 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e6a51a32-e613-441e-99fe-4b15d806a9ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:02.537084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e6a51a32-e613-441e-99fe-4b15d806a9ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:02.545984 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6a51a32-e613-441e-99fe-4b15d806a9ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:02.546491 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6a51a32-e613-441e-99fe-4b15d806a9ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:02.546920 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6a51a32-e613-441e-99fe-4b15d806a9ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:02.547612 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6a51a32-e613-441e-99fe-4b15d806a9ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:02.548053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6a51a32-e613-441e-99fe-4b15d806a9ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:02.548460 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e6a51a32-e613-441e-99fe-4b15d806a9ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:02.678453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e6a51a32-e613-441e-99fe-4b15d806a9ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:02.687929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e6a51a32-e613-441e-99fe-4b15d806a9ee tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.157110 Aug 28 01:28:02.688496 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1486/3195] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:02 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:03.714235 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e98a1024-58af-4a85-928c-d02e61c2e790 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:03.714875 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e98a1024-58af-4a85-928c-d02e61c2e790 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:03.722002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e98a1024-58af-4a85-928c-d02e61c2e790 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:03.722499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e98a1024-58af-4a85-928c-d02e61c2e790 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:03.722927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e98a1024-58af-4a85-928c-d02e61c2e790 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:03.723639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e98a1024-58af-4a85-928c-d02e61c2e790 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:03.724088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e98a1024-58af-4a85-928c-d02e61c2e790 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:03.724514 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e98a1024-58af-4a85-928c-d02e61c2e790 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:03.862173 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e98a1024-58af-4a85-928c-d02e61c2e790 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:03.871249 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e98a1024-58af-4a85-928c-d02e61c2e790 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.160896 Aug 28 01:28:03.871833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1710/3196] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:03 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:04.892124 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9766bc66-a3b0-4df0-86f4-a415bc1ad483 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:04.892805 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9766bc66-a3b0-4df0-86f4-a415bc1ad483 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:04.900046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9766bc66-a3b0-4df0-86f4-a415bc1ad483 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:04.900528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9766bc66-a3b0-4df0-86f4-a415bc1ad483 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:04.900988 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9766bc66-a3b0-4df0-86f4-a415bc1ad483 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:04.901672 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9766bc66-a3b0-4df0-86f4-a415bc1ad483 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:04.902115 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9766bc66-a3b0-4df0-86f4-a415bc1ad483 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:04.902560 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9766bc66-a3b0-4df0-86f4-a415bc1ad483 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:05.049522 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9766bc66-a3b0-4df0-86f4-a415bc1ad483 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:05.058617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9766bc66-a3b0-4df0-86f4-a415bc1ad483 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.170438 Aug 28 01:28:05.059214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1487/3197] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:04 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:06.081474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-45b8ae7a-e603-4900-aa82-0fd65eddd703 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:06.082512 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-45b8ae7a-e603-4900-aa82-0fd65eddd703 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:06.085401 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=45,nova_api:UPDATE=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:28:06.091056 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45b8ae7a-e603-4900-aa82-0fd65eddd703 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:06.091535 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45b8ae7a-e603-4900-aa82-0fd65eddd703 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:06.092047 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45b8ae7a-e603-4900-aa82-0fd65eddd703 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:06.092764 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45b8ae7a-e603-4900-aa82-0fd65eddd703 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:06.093253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45b8ae7a-e603-4900-aa82-0fd65eddd703 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:06.093680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-45b8ae7a-e603-4900-aa82-0fd65eddd703 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:06.233752 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-45b8ae7a-e603-4900-aa82-0fd65eddd703 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:06.242766 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-45b8ae7a-e603-4900-aa82-0fd65eddd703 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.165376 Aug 28 01:28:06.243360 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1711/3198] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:06 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:07.266677 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c03e333a-8411-4b30-93b0-4468aef968f3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:07.267306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c03e333a-8411-4b30-93b0-4468aef968f3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:07.275366 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c03e333a-8411-4b30-93b0-4468aef968f3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:07.275855 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c03e333a-8411-4b30-93b0-4468aef968f3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:07.276289 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c03e333a-8411-4b30-93b0-4468aef968f3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:07.276991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c03e333a-8411-4b30-93b0-4468aef968f3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:07.277462 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c03e333a-8411-4b30-93b0-4468aef968f3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:07.277897 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c03e333a-8411-4b30-93b0-4468aef968f3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:07.423518 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c03e333a-8411-4b30-93b0-4468aef968f3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:07.436184 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c03e333a-8411-4b30-93b0-4468aef968f3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.175257 Aug 28 01:28:07.436748 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1488/3199] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:07 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:28:08.467628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-28eba4da-72a8-4f7b-b893-4e5fc5387b2a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:08.468315 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-28eba4da-72a8-4f7b-b893-4e5fc5387b2a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:08.475166 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-28eba4da-72a8-4f7b-b893-4e5fc5387b2a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:08.475646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-28eba4da-72a8-4f7b-b893-4e5fc5387b2a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:08.476078 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-28eba4da-72a8-4f7b-b893-4e5fc5387b2a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:08.476769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-28eba4da-72a8-4f7b-b893-4e5fc5387b2a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:08.477255 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-28eba4da-72a8-4f7b-b893-4e5fc5387b2a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:08.477661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-28eba4da-72a8-4f7b-b893-4e5fc5387b2a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:08.612111 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-28eba4da-72a8-4f7b-b893-4e5fc5387b2a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:08.620918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-28eba4da-72a8-4f7b-b893-4e5fc5387b2a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.158763 Aug 28 01:28:08.621501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1712/3200] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:08 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:09.642591 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b1fd0096-aec9-4d3a-9e16-f427c3c8a5dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:09.643185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b1fd0096-aec9-4d3a-9e16-f427c3c8a5dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:09.650077 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b1fd0096-aec9-4d3a-9e16-f427c3c8a5dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:09.650536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b1fd0096-aec9-4d3a-9e16-f427c3c8a5dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:09.650969 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b1fd0096-aec9-4d3a-9e16-f427c3c8a5dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:09.651686 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b1fd0096-aec9-4d3a-9e16-f427c3c8a5dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:09.652188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b1fd0096-aec9-4d3a-9e16-f427c3c8a5dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:09.652598 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b1fd0096-aec9-4d3a-9e16-f427c3c8a5dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:09.784372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b1fd0096-aec9-4d3a-9e16-f427c3c8a5dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:09.792719 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b1fd0096-aec9-4d3a-9e16-f427c3c8a5dc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.154275 Aug 28 01:28:09.793317 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1489/3201] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:09 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:10.819931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-38ab1b25-143b-46c4-ace0-2f2631ed2b9f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:10.820930 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-38ab1b25-143b-46c4-ace0-2f2631ed2b9f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:10.830233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38ab1b25-143b-46c4-ace0-2f2631ed2b9f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:10.830751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38ab1b25-143b-46c4-ace0-2f2631ed2b9f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:10.831185 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38ab1b25-143b-46c4-ace0-2f2631ed2b9f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:10.831872 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38ab1b25-143b-46c4-ace0-2f2631ed2b9f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:10.832320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38ab1b25-143b-46c4-ace0-2f2631ed2b9f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:10.832745 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-38ab1b25-143b-46c4-ace0-2f2631ed2b9f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:10.974259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-38ab1b25-143b-46c4-ace0-2f2631ed2b9f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:10.976697 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:28:10.984490 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-38ab1b25-143b-46c4-ace0-2f2631ed2b9f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.171363 Aug 28 01:28:10.985090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1713/3202] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:10 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:12.006405 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-32f863d8-caa8-41d9-be32-58f05c9bb3a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:12.007062 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-32f863d8-caa8-41d9-be32-58f05c9bb3a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:12.013929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-32f863d8-caa8-41d9-be32-58f05c9bb3a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:12.014404 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-32f863d8-caa8-41d9-be32-58f05c9bb3a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:12.014835 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-32f863d8-caa8-41d9-be32-58f05c9bb3a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:12.015521 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-32f863d8-caa8-41d9-be32-58f05c9bb3a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:12.015972 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-32f863d8-caa8-41d9-be32-58f05c9bb3a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:12.016391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-32f863d8-caa8-41d9-be32-58f05c9bb3a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:12.152668 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-32f863d8-caa8-41d9-be32-58f05c9bb3a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:12.161734 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-32f863d8-caa8-41d9-be32-58f05c9bb3a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.159413 Aug 28 01:28:12.162303 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1490/3203] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:12 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:13.190168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-579c05cd-463f-4c67-9928-9b2292e35af8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:13.190805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-579c05cd-463f-4c67-9928-9b2292e35af8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:13.198335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-579c05cd-463f-4c67-9928-9b2292e35af8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:13.198899 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-579c05cd-463f-4c67-9928-9b2292e35af8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:13.199372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-579c05cd-463f-4c67-9928-9b2292e35af8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:13.200094 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-579c05cd-463f-4c67-9928-9b2292e35af8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:13.200548 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-579c05cd-463f-4c67-9928-9b2292e35af8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:13.201090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-579c05cd-463f-4c67-9928-9b2292e35af8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:13.343598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-579c05cd-463f-4c67-9928-9b2292e35af8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:13.354419 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-579c05cd-463f-4c67-9928-9b2292e35af8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.168974 Aug 28 01:28:13.355077 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1714/3204] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:13 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:14.376466 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a5cfbdf6-4e27-45be-ab5a-22789e828e0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:14.377067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a5cfbdf6-4e27-45be-ab5a-22789e828e0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:14.384531 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5cfbdf6-4e27-45be-ab5a-22789e828e0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:14.385045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5cfbdf6-4e27-45be-ab5a-22789e828e0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:14.385480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5cfbdf6-4e27-45be-ab5a-22789e828e0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:14.386188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5cfbdf6-4e27-45be-ab5a-22789e828e0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:14.386637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5cfbdf6-4e27-45be-ab5a-22789e828e0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:14.387061 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a5cfbdf6-4e27-45be-ab5a-22789e828e0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:14.532740 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a5cfbdf6-4e27-45be-ab5a-22789e828e0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:14.541813 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a5cfbdf6-4e27-45be-ab5a-22789e828e0a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.169645 Aug 28 01:28:14.542384 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1491/3205] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:14 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:15.564438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0ad2495b-0fac-4065-acd7-05b59fe419a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:15.565056 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0ad2495b-0fac-4065-acd7-05b59fe419a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:15.571956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ad2495b-0fac-4065-acd7-05b59fe419a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:15.572444 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ad2495b-0fac-4065-acd7-05b59fe419a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:15.572875 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ad2495b-0fac-4065-acd7-05b59fe419a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:15.573603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ad2495b-0fac-4065-acd7-05b59fe419a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:15.574056 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ad2495b-0fac-4065-acd7-05b59fe419a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:15.574474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0ad2495b-0fac-4065-acd7-05b59fe419a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:15.712816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0ad2495b-0fac-4065-acd7-05b59fe419a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:15.722519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0ad2495b-0fac-4065-acd7-05b59fe419a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.162305 Aug 28 01:28:15.723092 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1715/3206] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:15 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:16.744660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9195548d-312a-4dd0-a1b6-e16ea39e041f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:16.745343 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9195548d-312a-4dd0-a1b6-e16ea39e041f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:16.752477 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9195548d-312a-4dd0-a1b6-e16ea39e041f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:16.752997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9195548d-312a-4dd0-a1b6-e16ea39e041f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:16.753420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9195548d-312a-4dd0-a1b6-e16ea39e041f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:16.754149 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9195548d-312a-4dd0-a1b6-e16ea39e041f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:16.754604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9195548d-312a-4dd0-a1b6-e16ea39e041f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:16.755029 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9195548d-312a-4dd0-a1b6-e16ea39e041f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:16.892630 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9195548d-312a-4dd0-a1b6-e16ea39e041f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:16.903660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9195548d-312a-4dd0-a1b6-e16ea39e041f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.162654 Aug 28 01:28:16.904421 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1492/3207] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:16 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:17.925545 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-786ab3c9-cef7-485a-b949-366844d2296a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:17.926208 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-786ab3c9-cef7-485a-b949-366844d2296a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:17.932607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-786ab3c9-cef7-485a-b949-366844d2296a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:17.933117 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-786ab3c9-cef7-485a-b949-366844d2296a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:17.933545 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-786ab3c9-cef7-485a-b949-366844d2296a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:17.934232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-786ab3c9-cef7-485a-b949-366844d2296a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:17.934700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-786ab3c9-cef7-485a-b949-366844d2296a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:17.935225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-786ab3c9-cef7-485a-b949-366844d2296a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:18.070827 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-786ab3c9-cef7-485a-b949-366844d2296a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:18.079375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-786ab3c9-cef7-485a-b949-366844d2296a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.157768 Aug 28 01:28:18.079942 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1716/3208] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:17 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:19.100999 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a2967d3d-cc2d-490e-81a4-5b1c5a976a10 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:19.101681 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a2967d3d-cc2d-490e-81a4-5b1c5a976a10 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:19.103970 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=33,nova_api:DELETE=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:28:19.109674 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a2967d3d-cc2d-490e-81a4-5b1c5a976a10 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:19.110160 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a2967d3d-cc2d-490e-81a4-5b1c5a976a10 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:19.110603 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a2967d3d-cc2d-490e-81a4-5b1c5a976a10 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:19.111307 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a2967d3d-cc2d-490e-81a4-5b1c5a976a10 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:19.111760 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a2967d3d-cc2d-490e-81a4-5b1c5a976a10 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:19.112189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a2967d3d-cc2d-490e-81a4-5b1c5a976a10 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:19.254737 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a2967d3d-cc2d-490e-81a4-5b1c5a976a10 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:19.264497 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a2967d3d-cc2d-490e-81a4-5b1c5a976a10 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.167305 Aug 28 01:28:19.265128 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1493/3209] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:19 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:20.288494 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0328429a-4c8c-4292-9d5e-47b134c27bd8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:20.289148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0328429a-4c8c-4292-9d5e-47b134c27bd8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:20.297793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0328429a-4c8c-4292-9d5e-47b134c27bd8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:20.298471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0328429a-4c8c-4292-9d5e-47b134c27bd8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:20.298903 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0328429a-4c8c-4292-9d5e-47b134c27bd8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:20.299585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0328429a-4c8c-4292-9d5e-47b134c27bd8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:20.300030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0328429a-4c8c-4292-9d5e-47b134c27bd8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:20.300443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0328429a-4c8c-4292-9d5e-47b134c27bd8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:20.439690 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0328429a-4c8c-4292-9d5e-47b134c27bd8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:20.452873 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0328429a-4c8c-4292-9d5e-47b134c27bd8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.169272 Aug 28 01:28:20.453902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1717/3210] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:20 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:21.475142 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2ac770b2-fceb-44e0-8417-e7774a87656a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:21.475758 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2ac770b2-fceb-44e0-8417-e7774a87656a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:21.482484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ac770b2-fceb-44e0-8417-e7774a87656a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:21.482972 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ac770b2-fceb-44e0-8417-e7774a87656a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:21.483405 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ac770b2-fceb-44e0-8417-e7774a87656a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:21.484103 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ac770b2-fceb-44e0-8417-e7774a87656a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:21.484536 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ac770b2-fceb-44e0-8417-e7774a87656a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:21.484953 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ac770b2-fceb-44e0-8417-e7774a87656a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:21.697473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2ac770b2-fceb-44e0-8417-e7774a87656a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:21.706183 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2ac770b2-fceb-44e0-8417-e7774a87656a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.234926 Aug 28 01:28:21.706806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1494/3211] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:21 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:22.730794 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3ee53d65-94b6-4f96-b94c-6b2914d1b3b4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:22.731446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3ee53d65-94b6-4f96-b94c-6b2914d1b3b4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:22.741955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ee53d65-94b6-4f96-b94c-6b2914d1b3b4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:22.742621 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ee53d65-94b6-4f96-b94c-6b2914d1b3b4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:22.743188 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ee53d65-94b6-4f96-b94c-6b2914d1b3b4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:22.744107 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ee53d65-94b6-4f96-b94c-6b2914d1b3b4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:22.744706 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ee53d65-94b6-4f96-b94c-6b2914d1b3b4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:22.745357 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3ee53d65-94b6-4f96-b94c-6b2914d1b3b4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:22.896884 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3ee53d65-94b6-4f96-b94c-6b2914d1b3b4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:22.906171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3ee53d65-94b6-4f96-b94c-6b2914d1b3b4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.180883 Aug 28 01:28:22.906767 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1718/3212] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:22 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:23.932151 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cd0c846c-1e2c-4e9d-9d6c-1dc60c212f50 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:23.932788 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cd0c846c-1e2c-4e9d-9d6c-1dc60c212f50 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:23.939839 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cd0c846c-1e2c-4e9d-9d6c-1dc60c212f50 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:23.940326 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cd0c846c-1e2c-4e9d-9d6c-1dc60c212f50 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:23.940761 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cd0c846c-1e2c-4e9d-9d6c-1dc60c212f50 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:23.941491 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cd0c846c-1e2c-4e9d-9d6c-1dc60c212f50 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:23.941941 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cd0c846c-1e2c-4e9d-9d6c-1dc60c212f50 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:23.942372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cd0c846c-1e2c-4e9d-9d6c-1dc60c212f50 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:24.084103 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-cd0c846c-1e2c-4e9d-9d6c-1dc60c212f50 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:24.095050 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cd0c846c-1e2c-4e9d-9d6c-1dc60c212f50 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.166978 Aug 28 01:28:24.095606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1495/3213] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:23 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:25.121825 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-dfaee9f5-b67b-4c25-8810-7ead736f3b72 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:25.122454 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-dfaee9f5-b67b-4c25-8810-7ead736f3b72 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:25.130808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dfaee9f5-b67b-4c25-8810-7ead736f3b72 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:25.131447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dfaee9f5-b67b-4c25-8810-7ead736f3b72 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:25.132085 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dfaee9f5-b67b-4c25-8810-7ead736f3b72 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:25.133152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dfaee9f5-b67b-4c25-8810-7ead736f3b72 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:25.133780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dfaee9f5-b67b-4c25-8810-7ead736f3b72 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:25.134395 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dfaee9f5-b67b-4c25-8810-7ead736f3b72 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:25.281116 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dfaee9f5-b67b-4c25-8810-7ead736f3b72 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:25.289900 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-dfaee9f5-b67b-4c25-8810-7ead736f3b72 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.172701 Aug 28 01:28:25.290484 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1719/3214] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:25 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:26.312026 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9d60d3b7-3f70-42b8-a980-abcf14c4fdd5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:26.312669 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9d60d3b7-3f70-42b8-a980-abcf14c4fdd5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:26.320288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d60d3b7-3f70-42b8-a980-abcf14c4fdd5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:26.320778 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d60d3b7-3f70-42b8-a980-abcf14c4fdd5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:26.321243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d60d3b7-3f70-42b8-a980-abcf14c4fdd5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:26.321942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d60d3b7-3f70-42b8-a980-abcf14c4fdd5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:26.322388 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d60d3b7-3f70-42b8-a980-abcf14c4fdd5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:26.322826 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9d60d3b7-3f70-42b8-a980-abcf14c4fdd5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:26.325735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=78 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:28:26.467208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9d60d3b7-3f70-42b8-a980-abcf14c4fdd5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:26.477367 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9d60d3b7-3f70-42b8-a980-abcf14c4fdd5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.169626 Aug 28 01:28:26.477931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1496/3215] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:26 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:27.499327 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3d860ce8-881b-49ce-8ba1-b1e2f364a869 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:27.499980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3d860ce8-881b-49ce-8ba1-b1e2f364a869 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:27.506693 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d860ce8-881b-49ce-8ba1-b1e2f364a869 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:27.507167 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d860ce8-881b-49ce-8ba1-b1e2f364a869 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:27.507616 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d860ce8-881b-49ce-8ba1-b1e2f364a869 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:27.508313 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d860ce8-881b-49ce-8ba1-b1e2f364a869 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:27.508759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d860ce8-881b-49ce-8ba1-b1e2f364a869 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:27.509223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d860ce8-881b-49ce-8ba1-b1e2f364a869 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:27.651056 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3d860ce8-881b-49ce-8ba1-b1e2f364a869 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:27.659793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3d860ce8-881b-49ce-8ba1-b1e2f364a869 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.164603 Aug 28 01:28:27.660486 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1720/3216] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:27 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:28.684624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-45b2d818-7fd1-4f32-bf73-034c0d0e5dd8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:28.685381 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-45b2d818-7fd1-4f32-bf73-034c0d0e5dd8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:28.692555 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-45b2d818-7fd1-4f32-bf73-034c0d0e5dd8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:28.693071 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-45b2d818-7fd1-4f32-bf73-034c0d0e5dd8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:28.693500 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-45b2d818-7fd1-4f32-bf73-034c0d0e5dd8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:28.694189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-45b2d818-7fd1-4f32-bf73-034c0d0e5dd8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:28.694637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-45b2d818-7fd1-4f32-bf73-034c0d0e5dd8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:28.695062 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-45b2d818-7fd1-4f32-bf73-034c0d0e5dd8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:28.834307 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-45b2d818-7fd1-4f32-bf73-034c0d0e5dd8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:28.843021 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-45b2d818-7fd1-4f32-bf73-034c0d0e5dd8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.162238 Aug 28 01:28:28.843625 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1497/3217] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:28 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:29.865042 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9c8383cc-0d6a-4edc-a890-43a9aefb09f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:29.865737 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9c8383cc-0d6a-4edc-a890-43a9aefb09f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:29.872694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c8383cc-0d6a-4edc-a890-43a9aefb09f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:29.873223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c8383cc-0d6a-4edc-a890-43a9aefb09f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:29.873651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c8383cc-0d6a-4edc-a890-43a9aefb09f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:29.874383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c8383cc-0d6a-4edc-a890-43a9aefb09f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:29.874832 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c8383cc-0d6a-4edc-a890-43a9aefb09f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:29.875253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c8383cc-0d6a-4edc-a890-43a9aefb09f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:30.011496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9c8383cc-0d6a-4edc-a890-43a9aefb09f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:30.020324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9c8383cc-0d6a-4edc-a890-43a9aefb09f4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.159296 Aug 28 01:28:30.020907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1721/3218] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:29 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:30.198009 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5920352d-1ab3-4e79-bfba-b356ebf8b399 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:30.198663 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5920352d-1ab3-4e79-bfba-b356ebf8b399 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:30.205561 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5920352d-1ab3-4e79-bfba-b356ebf8b399 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:30.206138 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5920352d-1ab3-4e79-bfba-b356ebf8b399 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:30.206485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5920352d-1ab3-4e79-bfba-b356ebf8b399 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:30.207190 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5920352d-1ab3-4e79-bfba-b356ebf8b399 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:30.207639 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5920352d-1ab3-4e79-bfba-b356ebf8b399 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:30.208044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5920352d-1ab3-4e79-bfba-b356ebf8b399 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:30.234068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5920352d-1ab3-4e79-bfba-b356ebf8b399 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] Going to try to terminate instance {{(pid=120524) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 28 01:28:30.323096 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5920352d-1ab3-4e79-bfba-b356ebf8b399 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "DELETE /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2" status: 204 len: 0 microversion: 2.1 time: 0.128703 Aug 28 01:28:30.323713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1498/3219] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 01:28:30 2022] DELETE /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 28 01:28:30.331959 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-52565b10-6a18-4a2b-b91b-93f809594519 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:30.332583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-52565b10-6a18-4a2b-b91b-93f809594519 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:30.338802 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52565b10-6a18-4a2b-b91b-93f809594519 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:30.339284 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52565b10-6a18-4a2b-b91b-93f809594519 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:30.339774 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52565b10-6a18-4a2b-b91b-93f809594519 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:30.340414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52565b10-6a18-4a2b-b91b-93f809594519 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:30.340868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52565b10-6a18-4a2b-b91b-93f809594519 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:30.341328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52565b10-6a18-4a2b-b91b-93f809594519 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:30.477004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-52565b10-6a18-4a2b-b91b-93f809594519 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:30.485371 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-52565b10-6a18-4a2b-b91b-93f809594519 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2" status: 200 len: 1681 microversion: 2.1 time: 0.157098 Aug 28 01:28:30.485974 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1722/3220] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:30 2022] GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2 => generated 1681 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:31.042075 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-78be1d21-4763-4d13-99b4-39882729808f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:31.042783 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-78be1d21-4763-4d13-99b4-39882729808f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:31.049468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78be1d21-4763-4d13-99b4-39882729808f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:31.049959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78be1d21-4763-4d13-99b4-39882729808f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:31.050399 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78be1d21-4763-4d13-99b4-39882729808f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:31.051104 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78be1d21-4763-4d13-99b4-39882729808f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:31.051557 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78be1d21-4763-4d13-99b4-39882729808f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:31.051983 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-78be1d21-4763-4d13-99b4-39882729808f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:31.189546 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-78be1d21-4763-4d13-99b4-39882729808f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:31.199532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-78be1d21-4763-4d13-99b4-39882729808f tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.161676 Aug 28 01:28:31.200271 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1499/3221] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:31 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:31.510518 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-56a9f259-f3bc-4759-9a4c-be16b55037c2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:31.511347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-56a9f259-f3bc-4759-9a4c-be16b55037c2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:31.519713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56a9f259-f3bc-4759-9a4c-be16b55037c2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:31.520388 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56a9f259-f3bc-4759-9a4c-be16b55037c2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:31.521027 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56a9f259-f3bc-4759-9a4c-be16b55037c2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:31.521787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56a9f259-f3bc-4759-9a4c-be16b55037c2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:31.522384 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56a9f259-f3bc-4759-9a4c-be16b55037c2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:31.523002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-56a9f259-f3bc-4759-9a4c-be16b55037c2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:31.655027 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-56a9f259-f3bc-4759-9a4c-be16b55037c2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:31.662867 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-56a9f259-f3bc-4759-9a4c-be16b55037c2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2" status: 200 len: 1681 microversion: 2.1 time: 0.158131 Aug 28 01:28:31.663430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1723/3222] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:31 2022] GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2 => generated 1681 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:32.221661 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b229159c-2f9a-4fc3-9f83-6a9649bdcebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:32.222316 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b229159c-2f9a-4fc3-9f83-6a9649bdcebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:32.229978 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b229159c-2f9a-4fc3-9f83-6a9649bdcebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:32.230451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b229159c-2f9a-4fc3-9f83-6a9649bdcebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:32.230921 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b229159c-2f9a-4fc3-9f83-6a9649bdcebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:32.231637 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b229159c-2f9a-4fc3-9f83-6a9649bdcebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:32.232094 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b229159c-2f9a-4fc3-9f83-6a9649bdcebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:32.232523 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b229159c-2f9a-4fc3-9f83-6a9649bdcebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:32.478217 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b229159c-2f9a-4fc3-9f83-6a9649bdcebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:32.486795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b229159c-2f9a-4fc3-9f83-6a9649bdcebf tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.269067 Aug 28 01:28:32.487741 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1500/3223] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:32 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:32.687132 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-96098a70-61cf-499e-865b-63f298b7b31e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:32.687977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-96098a70-61cf-499e-865b-63f298b7b31e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:32.695310 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-96098a70-61cf-499e-865b-63f298b7b31e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:32.695799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-96098a70-61cf-499e-865b-63f298b7b31e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:32.696228 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-96098a70-61cf-499e-865b-63f298b7b31e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:32.696935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-96098a70-61cf-499e-865b-63f298b7b31e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:32.697538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-96098a70-61cf-499e-865b-63f298b7b31e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:32.697848 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-96098a70-61cf-499e-865b-63f298b7b31e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:33.077955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-96098a70-61cf-499e-865b-63f298b7b31e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:33.087002 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-96098a70-61cf-499e-865b-63f298b7b31e tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2" status: 200 len: 1681 microversion: 2.1 time: 0.405854 Aug 28 01:28:33.087717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1724/3224] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:32 2022] GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2 => generated 1681 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:33.470583 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-a71a5278-39ae-4526-b53b-d7ee0a2e34a0 req-20948ca0-2c40-408f-b4a3-48ffe2f41f43 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9d45bf39-4417-4af3-869d-9761326e9eb2", "name": "network-vif-deleted", "tag": "9e2ae2aa-ec13-4a1a-a862-13d3bf69d93a"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:28:33.480896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-a71a5278-39ae-4526-b53b-d7ee0a2e34a0 req-20948ca0-2c40-408f-b4a3-48ffe2f41f43 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:33.481385 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-a71a5278-39ae-4526-b53b-d7ee0a2e34a0 req-20948ca0-2c40-408f-b4a3-48ffe2f41f43 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:33.481817 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-a71a5278-39ae-4526-b53b-d7ee0a2e34a0 req-20948ca0-2c40-408f-b4a3-48ffe2f41f43 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:33.494067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-a71a5278-39ae-4526-b53b-d7ee0a2e34a0 req-20948ca0-2c40-408f-b4a3-48ffe2f41f43 service nova] Creating event network-vif-deleted:9e2ae2aa-ec13-4a1a-a862-13d3bf69d93a for instance 9d45bf39-4417-4af3-869d-9761326e9eb2 on hv-cinder-maste Aug 28 01:28:33.500141 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-a71a5278-39ae-4526-b53b-d7ee0a2e34a0 req-20948ca0-2c40-408f-b4a3-48ffe2f41f43 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032820 Aug 28 01:28:33.505586 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1501/3225] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:28:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:28:33.508642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9b4fd2af-226f-4634-87ac-97d144abf6ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:33.509413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9b4fd2af-226f-4634-87ac-97d144abf6ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:33.519817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b4fd2af-226f-4634-87ac-97d144abf6ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:33.520186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b4fd2af-226f-4634-87ac-97d144abf6ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:33.520637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b4fd2af-226f-4634-87ac-97d144abf6ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:33.521367 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b4fd2af-226f-4634-87ac-97d144abf6ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:33.521821 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b4fd2af-226f-4634-87ac-97d144abf6ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:33.522241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b4fd2af-226f-4634-87ac-97d144abf6ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:33.706322 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9b4fd2af-226f-4634-87ac-97d144abf6ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:33.717121 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9b4fd2af-226f-4634-87ac-97d144abf6ae tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.212677 Aug 28 01:28:33.717697 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1725/3226] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:33 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:34.110558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e43b7d23-936b-4af8-aca0-ed61a29c0503 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:34.111288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e43b7d23-936b-4af8-aca0-ed61a29c0503 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:34.121759 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e43b7d23-936b-4af8-aca0-ed61a29c0503 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:34.122251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e43b7d23-936b-4af8-aca0-ed61a29c0503 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:34.122692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e43b7d23-936b-4af8-aca0-ed61a29c0503 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:34.123395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e43b7d23-936b-4af8-aca0-ed61a29c0503 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:34.123844 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e43b7d23-936b-4af8-aca0-ed61a29c0503 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:34.124258 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e43b7d23-936b-4af8-aca0-ed61a29c0503 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:34.161051 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e43b7d23-936b-4af8-aca0-ed61a29c0503 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:34.161618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e43b7d23-936b-4af8-aca0-ed61a29c0503 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:34.161953 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e43b7d23-936b-4af8-aca0-ed61a29c0503 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:34.162642 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.objects.instance [None req-e43b7d23-936b-4af8-aca0-ed61a29c0503 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lazy-loading 'fault' on Instance uuid 9d45bf39-4417-4af3-869d-9761326e9eb2 {{(pid=120524) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:28:34.208226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e43b7d23-936b-4af8-aca0-ed61a29c0503 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd209174d0db04b3584556824c6d0a3d4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '95df81e14a8549fca0851234f4a00b89', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:34.224032 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e43b7d23-936b-4af8-aca0-ed61a29c0503 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2" status: 200 len: 1331 microversion: 2.1 time: 0.119702 Aug 28 01:28:34.225144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1502/3227] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:34 2022] GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2 => generated 1331 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:34.740612 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-52c14ade-e35e-49dd-8f3f-373ef7068b58 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:34.741283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-52c14ade-e35e-49dd-8f3f-373ef7068b58 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:34.748151 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52c14ade-e35e-49dd-8f3f-373ef7068b58 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:34.748640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52c14ade-e35e-49dd-8f3f-373ef7068b58 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:34.749093 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52c14ade-e35e-49dd-8f3f-373ef7068b58 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:34.749889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52c14ade-e35e-49dd-8f3f-373ef7068b58 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:34.750336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52c14ade-e35e-49dd-8f3f-373ef7068b58 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:34.750788 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-52c14ade-e35e-49dd-8f3f-373ef7068b58 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:34.893324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-52c14ade-e35e-49dd-8f3f-373ef7068b58 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:34.903977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-52c14ade-e35e-49dd-8f3f-373ef7068b58 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.168106 Aug 28 01:28:34.904786 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1726/3228] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:34 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:35.245700 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0827fa61-6adc-4229-9a33-60f5d669d3ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:35.246302 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0827fa61-6adc-4229-9a33-60f5d669d3ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:35.253307 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0827fa61-6adc-4229-9a33-60f5d669d3ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:35.253792 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0827fa61-6adc-4229-9a33-60f5d669d3ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:35.254220 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0827fa61-6adc-4229-9a33-60f5d669d3ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:35.254919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0827fa61-6adc-4229-9a33-60f5d669d3ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:35.255366 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0827fa61-6adc-4229-9a33-60f5d669d3ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:35.255781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0827fa61-6adc-4229-9a33-60f5d669d3ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:35.269779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-0827fa61-6adc-4229-9a33-60f5d669d3ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] HTTP exception thrown: Instance 9d45bf39-4417-4af3-869d-9761326e9eb2 could not be found. Aug 28 01:28:35.271450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0827fa61-6adc-4229-9a33-60f5d669d3ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Request method failure captured: Aug 28 01:28:35.271450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2 HTTP/1.1 Aug 28 01:28:35.271450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:28:35.271450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:28:35.271450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:28:35.271450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:28:35.271450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:28:35.271450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:28:35.271450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:28:35.271450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:28:35.271450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCrzduxHoOEhvI8P-Mde9cL1oxkmc4fMm9FmGsHxLSNCBjncK-ctZ1RF74JsMq7QMGP1Qh6DECNOIg5DjWuRWqO-CpXkeFE6YBKwQT8SoQT23SwG4DtFzbguCYZajUyKFut9Dn943LOjL5V3-apkT_NK3VlDjeI9Rm1KYQSsNW0BOiQg Aug 28 01:28:35.271450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:28:35.271450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:28:35.271450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:28:35.271450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:28:35.271450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:28:35.271450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:28:35.271450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:28:35.271450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:28:35.271450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:28:35.271450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:28:35.273423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_95df81e14a8549fca0851234f4a00b89", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/95df81e14a8549fca0851234f4a00b89"}]}] Aug 28 01:28:35.273423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:28:35.273423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:28:35.273423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:28:35.273423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:28:35.273423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:28:35.273423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:28:35.273423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: d209174d0db04b3584556824c6d0a3d4 Aug 28 01:28:35.273423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:28:35.273423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:28:35.273423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Instance 9d45bf39-4417-4af3-869d-9761326e9eb2 could not be found. Aug 28 01:28:35.273423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'id': '9d45bf39-4417-4af3-869d-9761326e9eb2'} Aug 28 01:28:35.274579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:28:35.274579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:28:35.274579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:28:35.274579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:28:35.274579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:28:35.274579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:28:35.274579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:28:35.274579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:28:35.274579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:28:35.274579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:28:35.274579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9d45bf39-4417-4af3-869d-9761326e9eb2 could not be found. Aug 28 01:28:35.274579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:28:35.274579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:28:35.274579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:28:35.274579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:28:35.274579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:28:35.274579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:28:35.274579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:28:35.274579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:28:35.274579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:28:35.276395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:28:35.276395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 01:28:35.276395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 01:28:35.276395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:28:35.276395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:28:35.276395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:28:35.276395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:28:35.276395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9d45bf39-4417-4af3-869d-9761326e9eb2 could not be found. Aug 28 01:28:35.276395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:28:35.276395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:28:35.276395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:28:35.276395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:28:35.276395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:28:35.276395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:28:35.276395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:28:35.276395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:28:35.276395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9d45bf39-4417-4af3-869d-9761326e9eb2 could not be found. Aug 28 01:28:35.276395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:28:35.278167 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0827fa61-6adc-4229-9a33-60f5d669d3ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Returning 404 to user: Instance 9d45bf39-4417-4af3-869d-9761326e9eb2 could not be found. {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:28:35.278167 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0827fa61-6adc-4229-9a33-60f5d669d3ae tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2" status: 404 len: 111 microversion: 2.1 time: 0.031082 Aug 28 01:28:35.278167 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1503/3229] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:35 2022] GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:28:35.282226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-066fcef8-4095-4326-a3d8-17cbdf835dd2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '.version_select at 0x7f0e71ff9af0>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:35.304729 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-066fcef8-4095-4326-a3d8-17cbdf835dd2 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-221832629" status: 202 len: 0 microversion: 2.1 time: 0.026453 Aug 28 01:28:35.306723 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1727/3230] 10.222.0.42 () {58 vars in 1218 bytes} [Sun Aug 28 01:28:35 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-221832629 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Aug 28 01:28:35.927042 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-df282451-d69b-4697-894b-ba3d4836aed1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:35.927647 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-df282451-d69b-4697-894b-ba3d4836aed1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:35.934648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-df282451-d69b-4697-894b-ba3d4836aed1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:35.935108 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-df282451-d69b-4697-894b-ba3d4836aed1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:35.935582 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-df282451-d69b-4697-894b-ba3d4836aed1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:35.936407 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-df282451-d69b-4697-894b-ba3d4836aed1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:35.937346 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-df282451-d69b-4697-894b-ba3d4836aed1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:35.937774 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-df282451-d69b-4697-894b-ba3d4836aed1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:35.986661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d85b87b5-0e30-4c38-80e8-129470c51091 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:35.987269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d85b87b5-0e30-4c38-80e8-129470c51091 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:35.993460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d85b87b5-0e30-4c38-80e8-129470c51091 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:35.993935 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d85b87b5-0e30-4c38-80e8-129470c51091 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:35.994393 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d85b87b5-0e30-4c38-80e8-129470c51091 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:35.995083 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d85b87b5-0e30-4c38-80e8-129470c51091 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:35.995532 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d85b87b5-0e30-4c38-80e8-129470c51091 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:35.995953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d85b87b5-0e30-4c38-80e8-129470c51091 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:36.009817 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-d85b87b5-0e30-4c38-80e8-129470c51091 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] HTTP exception thrown: Instance 9d45bf39-4417-4af3-869d-9761326e9eb2 could not be found. Aug 28 01:28:36.011428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d85b87b5-0e30-4c38-80e8-129470c51091 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Request method failure captured: Aug 28 01:28:36.011428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: DELETE /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2 HTTP/1.1 Aug 28 01:28:36.011428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:28:36.011428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:28:36.011428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:28:36.011428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:28:36.011428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:28:36.011428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:28:36.011428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:28:36.011428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:28:36.011428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCrzduxHoOEhvI8P-Mde9cL1oxkmc4fMm9FmGsHxLSNCBjncK-ctZ1RF74JsMq7QMGP1Qh6DECNOIg5DjWuRWqO-CpXkeFE6YBKwQT8SoQT23SwG4DtFzbguCYZajUyKFut9Dn943LOjL5V3-apkT_NK3VlDjeI9Rm1KYQSsNW0BOiQg Aug 28 01:28:36.011428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:28:36.011428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:28:36.011428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:28:36.011428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:28:36.011428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:28:36.011428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:28:36.011428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:28:36.011428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:28:36.011428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:28:36.011428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:28:36.013361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_95df81e14a8549fca0851234f4a00b89", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/95df81e14a8549fca0851234f4a00b89"}]}] Aug 28 01:28:36.013361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:28:36.013361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:28:36.013361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:28:36.013361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:28:36.013361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:28:36.013361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:28:36.013361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: d209174d0db04b3584556824c6d0a3d4 Aug 28 01:28:36.013361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:28:36.013361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 01:28:36.013361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Instance 9d45bf39-4417-4af3-869d-9761326e9eb2 could not be found. Aug 28 01:28:36.013361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'id': '9d45bf39-4417-4af3-869d-9761326e9eb2'} Aug 28 01:28:36.014428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:28:36.014428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:28:36.014428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:28:36.014428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:28:36.014428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:28:36.014428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:28:36.014428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:28:36.014428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:28:36.014428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:28:36.014428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:28:36.014428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9d45bf39-4417-4af3-869d-9761326e9eb2 could not be found. Aug 28 01:28:36.014428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.014428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:28:36.014428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.014428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:28:36.014428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:28:36.014428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:28:36.014428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:28:36.014428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:28:36.014428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:28:36.015905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:28:36.015905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 28 01:28:36.015905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 28 01:28:36.015905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 28 01:28:36.015905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 28 01:28:36.015905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:28:36.015905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:28:36.015905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:28:36.015905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:28:36.015905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9d45bf39-4417-4af3-869d-9761326e9eb2 could not be found. Aug 28 01:28:36.015905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.015905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:28:36.015905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.015905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:28:36.015905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:28:36.015905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:28:36.015905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:28:36.015905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:28:36.015905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9d45bf39-4417-4af3-869d-9761326e9eb2 could not be found. Aug 28 01:28:36.015905 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.017840 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d85b87b5-0e30-4c38-80e8-129470c51091 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Returning 404 to user: Instance 9d45bf39-4417-4af3-869d-9761326e9eb2 could not be found. {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:28:36.017840 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d85b87b5-0e30-4c38-80e8-129470c51091 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "DELETE /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2" status: 404 len: 111 microversion: 2.1 time: 0.030639 Aug 28 01:28:36.017840 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1728/3231] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 01:28:35 2022] DELETE /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:28:36.021686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7d35a260-6e0c-473c-b4f1-c7fcff5f3fe8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:36.022393 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-7d35a260-6e0c-473c-b4f1-c7fcff5f3fe8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 9d45bf39-4417-4af3-869d-9761326e9eb2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:36.029397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7d35a260-6e0c-473c-b4f1-c7fcff5f3fe8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:36.029886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7d35a260-6e0c-473c-b4f1-c7fcff5f3fe8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:36.030316 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7d35a260-6e0c-473c-b4f1-c7fcff5f3fe8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:36.031164 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7d35a260-6e0c-473c-b4f1-c7fcff5f3fe8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:36.031616 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7d35a260-6e0c-473c-b4f1-c7fcff5f3fe8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:36.032040 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-7d35a260-6e0c-473c-b4f1-c7fcff5f3fe8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:36.046465 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-7d35a260-6e0c-473c-b4f1-c7fcff5f3fe8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] HTTP exception thrown: Instance 9d45bf39-4417-4af3-869d-9761326e9eb2 could not be found. Aug 28 01:28:36.048055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7d35a260-6e0c-473c-b4f1-c7fcff5f3fe8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Request method failure captured: Aug 28 01:28:36.048055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2 HTTP/1.1 Aug 28 01:28:36.048055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:28:36.048055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:28:36.048055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:28:36.048055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:28:36.048055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:28:36.048055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:28:36.048055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:28:36.048055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:28:36.048055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCrzduxHoOEhvI8P-Mde9cL1oxkmc4fMm9FmGsHxLSNCBjncK-ctZ1RF74JsMq7QMGP1Qh6DECNOIg5DjWuRWqO-CpXkeFE6YBKwQT8SoQT23SwG4DtFzbguCYZajUyKFut9Dn943LOjL5V3-apkT_NK3VlDjeI9Rm1KYQSsNW0BOiQg Aug 28 01:28:36.048055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:28:36.048055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:28:36.048055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:28:36.048055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:28:36.048055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:28:36.048055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:28:36.048055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:28:36.048055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:28:36.048055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:28:36.048055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:28:36.049939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_95df81e14a8549fca0851234f4a00b89", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/95df81e14a8549fca0851234f4a00b89"}]}] Aug 28 01:28:36.049939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:28:36.049939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:28:36.049939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:28:36.049939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:28:36.049939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:28:36.049939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:28:36.049939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: d209174d0db04b3584556824c6d0a3d4 Aug 28 01:28:36.049939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:28:36.049939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 01:28:36.049939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Instance 9d45bf39-4417-4af3-869d-9761326e9eb2 could not be found. Aug 28 01:28:36.049939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'id': '9d45bf39-4417-4af3-869d-9761326e9eb2'} Aug 28 01:28:36.053972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:28:36.053972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:28:36.053972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:28:36.053972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:28:36.053972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:28:36.053972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:28:36.053972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:28:36.053972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:28:36.053972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:28:36.053972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:28:36.053972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9d45bf39-4417-4af3-869d-9761326e9eb2 could not be found. Aug 28 01:28:36.053972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.053972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:28:36.053972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.053972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:28:36.053972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:28:36.053972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:28:36.053972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:28:36.053972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:28:36.053972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:28:36.055649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:28:36.055649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 01:28:36.055649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 01:28:36.055649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:28:36.055649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:28:36.055649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:28:36.055649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:28:36.055649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9d45bf39-4417-4af3-869d-9761326e9eb2 could not be found. Aug 28 01:28:36.055649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.055649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:28:36.055649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.055649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:28:36.055649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:28:36.055649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:28:36.055649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:28:36.055649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:28:36.055649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9d45bf39-4417-4af3-869d-9761326e9eb2 could not be found. Aug 28 01:28:36.055649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.057292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-7d35a260-6e0c-473c-b4f1-c7fcff5f3fe8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Returning 404 to user: Instance 9d45bf39-4417-4af3-869d-9761326e9eb2 could not be found. {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:28:36.057292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-7d35a260-6e0c-473c-b4f1-c7fcff5f3fe8 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2" status: 404 len: 111 microversion: 2.1 time: 0.032110 Aug 28 01:28:36.057292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1729/3232] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:36 2022] GET /compute/v2.1/servers/9d45bf39-4417-4af3-869d-9761326e9eb2 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:28:36.057698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d04b33b1-ca66-40c7-9023-845a80c310a6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:36.058188 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d04b33b1-ca66-40c7-9023-845a80c310a6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:36.064684 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d04b33b1-ca66-40c7-9023-845a80c310a6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:36.065194 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d04b33b1-ca66-40c7-9023-845a80c310a6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:36.065621 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d04b33b1-ca66-40c7-9023-845a80c310a6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:36.066307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d04b33b1-ca66-40c7-9023-845a80c310a6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:36.066749 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d04b33b1-ca66-40c7-9023-845a80c310a6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:36.067157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d04b33b1-ca66-40c7-9023-845a80c310a6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:36.077574 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-df282451-d69b-4697-894b-ba3d4836aed1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:36.081009 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-d04b33b1-ca66-40c7-9023-845a80c310a6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] HTTP exception thrown: Instance 2d365998-141e-4842-a129-31469afdb5cc could not be found. Aug 28 01:28:36.082568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d04b33b1-ca66-40c7-9023-845a80c310a6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Request method failure captured: Aug 28 01:28:36.082568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: DELETE /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc HTTP/1.1 Aug 28 01:28:36.082568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:28:36.082568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:28:36.082568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:28:36.082568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:28:36.082568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:28:36.082568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:28:36.082568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:28:36.082568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:28:36.082568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCrzduxHoOEhvI8P-Mde9cL1oxkmc4fMm9FmGsHxLSNCBjncK-ctZ1RF74JsMq7QMGP1Qh6DECNOIg5DjWuRWqO-CpXkeFE6YBKwQT8SoQT23SwG4DtFzbguCYZajUyKFut9Dn943LOjL5V3-apkT_NK3VlDjeI9Rm1KYQSsNW0BOiQg Aug 28 01:28:36.082568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:28:36.082568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:28:36.082568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:28:36.082568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:28:36.082568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:28:36.082568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:28:36.082568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:28:36.082568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:28:36.082568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:28:36.082568 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:28:36.084410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_95df81e14a8549fca0851234f4a00b89", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/95df81e14a8549fca0851234f4a00b89"}]}] Aug 28 01:28:36.084410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:28:36.084410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:28:36.084410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:28:36.084410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:28:36.084410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:28:36.084410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:28:36.084410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: d209174d0db04b3584556824c6d0a3d4 Aug 28 01:28:36.084410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:28:36.084410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 01:28:36.084410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Instance 2d365998-141e-4842-a129-31469afdb5cc could not be found. Aug 28 01:28:36.084410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'id': '2d365998-141e-4842-a129-31469afdb5cc'} Aug 28 01:28:36.085649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:28:36.085649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:28:36.085649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:28:36.085649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:28:36.085649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:28:36.085649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:28:36.085649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:28:36.085649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:28:36.085649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:28:36.085649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:28:36.085649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2d365998-141e-4842-a129-31469afdb5cc could not be found. Aug 28 01:28:36.085649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.085649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:28:36.085649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.085649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:28:36.085649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:28:36.085649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:28:36.085649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:28:36.085649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:28:36.085649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:28:36.091780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:28:36.091780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 28 01:28:36.091780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 28 01:28:36.091780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 28 01:28:36.091780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 28 01:28:36.091780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:28:36.091780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:28:36.091780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:28:36.091780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:28:36.091780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2d365998-141e-4842-a129-31469afdb5cc could not be found. Aug 28 01:28:36.091780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.091780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:28:36.091780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.091780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:28:36.091780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:28:36.091780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:28:36.091780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:28:36.091780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:28:36.091780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2d365998-141e-4842-a129-31469afdb5cc could not be found. Aug 28 01:28:36.091780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.093631 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d04b33b1-ca66-40c7-9023-845a80c310a6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Returning 404 to user: Instance 2d365998-141e-4842-a129-31469afdb5cc could not be found. {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:28:36.093631 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d04b33b1-ca66-40c7-9023-845a80c310a6 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "DELETE /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc" status: 404 len: 111 microversion: 2.1 time: 0.029898 Aug 28 01:28:36.093631 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1730/3233] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 01:28:36 2022] DELETE /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 28 01:28:36.093918 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-df282451-d69b-4697-894b-ba3d4836aed1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.163282 Aug 28 01:28:36.093918 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1504/3234] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:35 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:36.099063 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-85f62fb9-3982-4349-9d7d-f6b08697c3a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:36.099691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-85f62fb9-3982-4349-9d7d-f6b08697c3a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 2d365998-141e-4842-a129-31469afdb5cc] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:36.101550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=41,nova_api:DELETE=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:28:36.106965 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85f62fb9-3982-4349-9d7d-f6b08697c3a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:36.107435 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85f62fb9-3982-4349-9d7d-f6b08697c3a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:36.107860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85f62fb9-3982-4349-9d7d-f6b08697c3a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:36.108539 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85f62fb9-3982-4349-9d7d-f6b08697c3a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:36.109100 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85f62fb9-3982-4349-9d7d-f6b08697c3a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:36.109553 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85f62fb9-3982-4349-9d7d-f6b08697c3a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:36.123909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-85f62fb9-3982-4349-9d7d-f6b08697c3a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] HTTP exception thrown: Instance 2d365998-141e-4842-a129-31469afdb5cc could not be found. Aug 28 01:28:36.125543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-85f62fb9-3982-4349-9d7d-f6b08697c3a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Request method failure captured: Aug 28 01:28:36.125543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc HTTP/1.1 Aug 28 01:28:36.125543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:28:36.125543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:28:36.125543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:28:36.125543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:28:36.125543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:28:36.125543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:28:36.125543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:28:36.125543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:28:36.125543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCrzduxHoOEhvI8P-Mde9cL1oxkmc4fMm9FmGsHxLSNCBjncK-ctZ1RF74JsMq7QMGP1Qh6DECNOIg5DjWuRWqO-CpXkeFE6YBKwQT8SoQT23SwG4DtFzbguCYZajUyKFut9Dn943LOjL5V3-apkT_NK3VlDjeI9Rm1KYQSsNW0BOiQg Aug 28 01:28:36.125543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:28:36.125543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:28:36.125543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:28:36.125543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:28:36.125543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:28:36.125543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:28:36.125543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:28:36.125543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:28:36.125543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:28:36.125543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:28:36.127469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_95df81e14a8549fca0851234f4a00b89", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/95df81e14a8549fca0851234f4a00b89"}]}] Aug 28 01:28:36.127469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:28:36.127469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:28:36.127469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:28:36.127469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:28:36.127469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:28:36.127469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:28:36.127469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: d209174d0db04b3584556824c6d0a3d4 Aug 28 01:28:36.127469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:28:36.127469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 01:28:36.127469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Instance 2d365998-141e-4842-a129-31469afdb5cc could not be found. Aug 28 01:28:36.127469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'id': '2d365998-141e-4842-a129-31469afdb5cc'} Aug 28 01:28:36.128406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:28:36.128406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:28:36.128406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:28:36.128406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:28:36.128406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:28:36.128406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:28:36.128406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:28:36.128406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:28:36.128406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:28:36.128406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:28:36.128406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2d365998-141e-4842-a129-31469afdb5cc could not be found. Aug 28 01:28:36.128406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.128406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:28:36.128406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.128406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:28:36.128406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:28:36.128406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:28:36.128406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:28:36.128406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:28:36.128406 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:28:36.129915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:28:36.129915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 01:28:36.129915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 01:28:36.129915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:28:36.129915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:28:36.129915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:28:36.129915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:28:36.129915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2d365998-141e-4842-a129-31469afdb5cc could not be found. Aug 28 01:28:36.129915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.129915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:28:36.129915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.129915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:28:36.129915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:28:36.129915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:28:36.129915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:28:36.129915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:28:36.129915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2d365998-141e-4842-a129-31469afdb5cc could not be found. Aug 28 01:28:36.129915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.131421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-85f62fb9-3982-4349-9d7d-f6b08697c3a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Returning 404 to user: Instance 2d365998-141e-4842-a129-31469afdb5cc could not be found. {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:28:36.131421 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-85f62fb9-3982-4349-9d7d-f6b08697c3a5 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc" status: 404 len: 111 microversion: 2.1 time: 0.031598 Aug 28 01:28:36.131748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1731/3235] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:36 2022] GET /compute/v2.1/servers/2d365998-141e-4842-a129-31469afdb5cc => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:28:36.136659 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-24b079e7-55c0-4d81-a1e5-e900f288fa02 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:36.137265 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-24b079e7-55c0-4d81-a1e5-e900f288fa02 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:36.143798 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24b079e7-55c0-4d81-a1e5-e900f288fa02 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:36.144393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24b079e7-55c0-4d81-a1e5-e900f288fa02 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:36.144893 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24b079e7-55c0-4d81-a1e5-e900f288fa02 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:36.145763 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24b079e7-55c0-4d81-a1e5-e900f288fa02 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:36.146336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24b079e7-55c0-4d81-a1e5-e900f288fa02 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:36.147302 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-24b079e7-55c0-4d81-a1e5-e900f288fa02 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:36.162296 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-24b079e7-55c0-4d81-a1e5-e900f288fa02 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] HTTP exception thrown: Instance 27028458-f872-49aa-8d37-1a886d566c00 could not be found. Aug 28 01:28:36.163891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-24b079e7-55c0-4d81-a1e5-e900f288fa02 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Request method failure captured: Aug 28 01:28:36.163891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: DELETE /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 HTTP/1.1 Aug 28 01:28:36.163891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:28:36.163891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:28:36.163891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:28:36.163891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:28:36.163891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:28:36.163891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:28:36.163891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:28:36.163891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:28:36.163891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCrzduxHoOEhvI8P-Mde9cL1oxkmc4fMm9FmGsHxLSNCBjncK-ctZ1RF74JsMq7QMGP1Qh6DECNOIg5DjWuRWqO-CpXkeFE6YBKwQT8SoQT23SwG4DtFzbguCYZajUyKFut9Dn943LOjL5V3-apkT_NK3VlDjeI9Rm1KYQSsNW0BOiQg Aug 28 01:28:36.163891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:28:36.163891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:28:36.163891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:28:36.163891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:28:36.163891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:28:36.163891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:28:36.163891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:28:36.163891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:28:36.163891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:28:36.163891 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:28:36.165795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_95df81e14a8549fca0851234f4a00b89", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/95df81e14a8549fca0851234f4a00b89"}]}] Aug 28 01:28:36.165795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:28:36.165795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:28:36.165795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:28:36.165795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:28:36.165795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:28:36.165795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:28:36.165795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: d209174d0db04b3584556824c6d0a3d4 Aug 28 01:28:36.165795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:28:36.165795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:28:36.165795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Instance 27028458-f872-49aa-8d37-1a886d566c00 could not be found. Aug 28 01:28:36.165795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'id': '27028458-f872-49aa-8d37-1a886d566c00'} Aug 28 01:28:36.166850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:28:36.166850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:28:36.166850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:28:36.166850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:28:36.166850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:28:36.166850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:28:36.166850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:28:36.166850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:28:36.166850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:28:36.166850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:28:36.166850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 27028458-f872-49aa-8d37-1a886d566c00 could not be found. Aug 28 01:28:36.166850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.166850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:28:36.166850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.166850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:28:36.166850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:28:36.166850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:28:36.166850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:28:36.166850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:28:36.166850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:28:36.168532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:28:36.168532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 28 01:28:36.168532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 28 01:28:36.168532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 28 01:28:36.168532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 28 01:28:36.168532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:28:36.168532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:28:36.168532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:28:36.168532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:28:36.168532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 27028458-f872-49aa-8d37-1a886d566c00 could not be found. Aug 28 01:28:36.168532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.168532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:28:36.168532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.168532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:28:36.168532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:28:36.168532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:28:36.168532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:28:36.168532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:28:36.168532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 27028458-f872-49aa-8d37-1a886d566c00 could not be found. Aug 28 01:28:36.168532 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.170269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-24b079e7-55c0-4d81-a1e5-e900f288fa02 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Returning 404 to user: Instance 27028458-f872-49aa-8d37-1a886d566c00 could not be found. {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:28:36.170269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-24b079e7-55c0-4d81-a1e5-e900f288fa02 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "DELETE /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 404 len: 111 microversion: 2.1 time: 0.032568 Aug 28 01:28:36.170269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1505/3236] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 01:28:36 2022] DELETE /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:28:36.174238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4a3f4157-712f-4e8b-810f-0b6a43c766ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:36.174848 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4a3f4157-712f-4e8b-810f-0b6a43c766ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: 27028458-f872-49aa-8d37-1a886d566c00] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:36.181108 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a3f4157-712f-4e8b-810f-0b6a43c766ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:36.181598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a3f4157-712f-4e8b-810f-0b6a43c766ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:36.182028 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a3f4157-712f-4e8b-810f-0b6a43c766ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:36.182730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a3f4157-712f-4e8b-810f-0b6a43c766ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:36.183181 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a3f4157-712f-4e8b-810f-0b6a43c766ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:36.183599 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4a3f4157-712f-4e8b-810f-0b6a43c766ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:36.198731 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-4a3f4157-712f-4e8b-810f-0b6a43c766ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] HTTP exception thrown: Instance 27028458-f872-49aa-8d37-1a886d566c00 could not be found. Aug 28 01:28:36.205126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4a3f4157-712f-4e8b-810f-0b6a43c766ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Request method failure captured: Aug 28 01:28:36.205126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 HTTP/1.1 Aug 28 01:28:36.205126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:28:36.205126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:28:36.205126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:28:36.205126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:28:36.205126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:28:36.205126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:28:36.205126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:28:36.205126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:28:36.205126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCrzduxHoOEhvI8P-Mde9cL1oxkmc4fMm9FmGsHxLSNCBjncK-ctZ1RF74JsMq7QMGP1Qh6DECNOIg5DjWuRWqO-CpXkeFE6YBKwQT8SoQT23SwG4DtFzbguCYZajUyKFut9Dn943LOjL5V3-apkT_NK3VlDjeI9Rm1KYQSsNW0BOiQg Aug 28 01:28:36.205126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:28:36.205126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:28:36.205126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:28:36.205126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:28:36.205126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:28:36.205126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:28:36.205126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:28:36.205126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:28:36.205126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:28:36.205126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:28:36.209992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_95df81e14a8549fca0851234f4a00b89", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/95df81e14a8549fca0851234f4a00b89"}]}] Aug 28 01:28:36.209992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:28:36.209992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:28:36.209992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:28:36.209992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:28:36.209992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:28:36.209992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:28:36.209992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: d209174d0db04b3584556824c6d0a3d4 Aug 28 01:28:36.209992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:28:36.209992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 01:28:36.209992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Instance 27028458-f872-49aa-8d37-1a886d566c00 could not be found. Aug 28 01:28:36.209992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'id': '27028458-f872-49aa-8d37-1a886d566c00'} Aug 28 01:28:36.211196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:28:36.211196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:28:36.211196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:28:36.211196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:28:36.211196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:28:36.211196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:28:36.211196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:28:36.211196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:28:36.211196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:28:36.211196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:28:36.211196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 27028458-f872-49aa-8d37-1a886d566c00 could not be found. Aug 28 01:28:36.211196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.211196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:28:36.211196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.211196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:28:36.211196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:28:36.211196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:28:36.211196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:28:36.211196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:28:36.211196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:28:36.212987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:28:36.212987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 01:28:36.212987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 01:28:36.212987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:28:36.212987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:28:36.212987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:28:36.212987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:28:36.212987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 27028458-f872-49aa-8d37-1a886d566c00 could not be found. Aug 28 01:28:36.212987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.212987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:28:36.212987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.212987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:28:36.212987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:28:36.212987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:28:36.212987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:28:36.212987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:28:36.212987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 27028458-f872-49aa-8d37-1a886d566c00 could not be found. Aug 28 01:28:36.212987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:28:36.214684 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4a3f4157-712f-4e8b-810f-0b6a43c766ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Returning 404 to user: Instance 27028458-f872-49aa-8d37-1a886d566c00 could not be found. {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:28:36.214684 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4a3f4157-712f-4e8b-810f-0b6a43c766ed tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00" status: 404 len: 111 microversion: 2.1 time: 0.031453 Aug 28 01:28:36.214684 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1732/3237] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:36 2022] GET /compute/v2.1/servers/27028458-f872-49aa-8d37-1a886d566c00 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:28:37.110450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-23126920-11d3-4a6d-ba77-c97857faed63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:37.111058 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-23126920-11d3-4a6d-ba77-c97857faed63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:37.118743 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23126920-11d3-4a6d-ba77-c97857faed63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:37.119239 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23126920-11d3-4a6d-ba77-c97857faed63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:37.120158 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23126920-11d3-4a6d-ba77-c97857faed63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:37.120367 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23126920-11d3-4a6d-ba77-c97857faed63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:37.120819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23126920-11d3-4a6d-ba77-c97857faed63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:37.121275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-23126920-11d3-4a6d-ba77-c97857faed63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:37.263821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-23126920-11d3-4a6d-ba77-c97857faed63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:37.272834 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-23126920-11d3-4a6d-ba77-c97857faed63 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.167881 Aug 28 01:28:37.273423 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1506/3238] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:37 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:38.294492 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-dbc06dfd-384e-49c3-b9a9-1db78ed240b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:38.295153 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-dbc06dfd-384e-49c3-b9a9-1db78ed240b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:38.302126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbc06dfd-384e-49c3-b9a9-1db78ed240b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:38.302618 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbc06dfd-384e-49c3-b9a9-1db78ed240b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:38.303042 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbc06dfd-384e-49c3-b9a9-1db78ed240b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:38.303743 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbc06dfd-384e-49c3-b9a9-1db78ed240b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:38.304190 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbc06dfd-384e-49c3-b9a9-1db78ed240b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:38.304625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dbc06dfd-384e-49c3-b9a9-1db78ed240b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:38.449233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dbc06dfd-384e-49c3-b9a9-1db78ed240b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:38.461416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-dbc06dfd-384e-49c3-b9a9-1db78ed240b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.170812 Aug 28 01:28:38.462110 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1733/3239] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:38 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:39.483507 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-423ece72-145d-4cf6-9325-ed5d2c466074 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:39.484265 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-423ece72-145d-4cf6-9325-ed5d2c466074 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:39.491278 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-423ece72-145d-4cf6-9325-ed5d2c466074 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:39.491788 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-423ece72-145d-4cf6-9325-ed5d2c466074 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:39.492196 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-423ece72-145d-4cf6-9325-ed5d2c466074 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:39.492917 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-423ece72-145d-4cf6-9325-ed5d2c466074 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:39.493399 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-423ece72-145d-4cf6-9325-ed5d2c466074 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:39.493955 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-423ece72-145d-4cf6-9325-ed5d2c466074 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:39.638043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-423ece72-145d-4cf6-9325-ed5d2c466074 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:39.646686 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-423ece72-145d-4cf6-9325-ed5d2c466074 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.167050 Aug 28 01:28:39.647263 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1507/3240] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:39 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:40.669546 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-987ce8cc-b1cb-41e8-840c-58c682aa3153 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:40.670188 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-987ce8cc-b1cb-41e8-840c-58c682aa3153 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:40.677387 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-987ce8cc-b1cb-41e8-840c-58c682aa3153 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:40.677869 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-987ce8cc-b1cb-41e8-840c-58c682aa3153 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:40.678299 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-987ce8cc-b1cb-41e8-840c-58c682aa3153 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:40.678983 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-987ce8cc-b1cb-41e8-840c-58c682aa3153 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:40.679465 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-987ce8cc-b1cb-41e8-840c-58c682aa3153 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:40.679872 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-987ce8cc-b1cb-41e8-840c-58c682aa3153 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:40.823844 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-987ce8cc-b1cb-41e8-840c-58c682aa3153 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:40.826286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=100 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:28:40.834475 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-987ce8cc-b1cb-41e8-840c-58c682aa3153 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.169953 Aug 28 01:28:40.835071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1734/3241] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:40 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:41.856757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ce7d7e8c-02c2-4946-a9a5-b4bde799672b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:41.857390 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ce7d7e8c-02c2-4946-a9a5-b4bde799672b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:41.864528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce7d7e8c-02c2-4946-a9a5-b4bde799672b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:41.865016 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce7d7e8c-02c2-4946-a9a5-b4bde799672b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:41.865470 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce7d7e8c-02c2-4946-a9a5-b4bde799672b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:41.866173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce7d7e8c-02c2-4946-a9a5-b4bde799672b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:41.866596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce7d7e8c-02c2-4946-a9a5-b4bde799672b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:41.867015 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce7d7e8c-02c2-4946-a9a5-b4bde799672b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:42.012336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ce7d7e8c-02c2-4946-a9a5-b4bde799672b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:42.021567 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ce7d7e8c-02c2-4946-a9a5-b4bde799672b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.168736 Aug 28 01:28:42.022158 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1508/3242] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:41 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:43.046114 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-07a054d2-d6bf-4fe1-9da2-32f1a43c2889 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:43.046874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-07a054d2-d6bf-4fe1-9da2-32f1a43c2889 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:43.054562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07a054d2-d6bf-4fe1-9da2-32f1a43c2889 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:43.055075 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07a054d2-d6bf-4fe1-9da2-32f1a43c2889 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:43.055507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07a054d2-d6bf-4fe1-9da2-32f1a43c2889 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:43.056216 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07a054d2-d6bf-4fe1-9da2-32f1a43c2889 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:43.056666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07a054d2-d6bf-4fe1-9da2-32f1a43c2889 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:43.057128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-07a054d2-d6bf-4fe1-9da2-32f1a43c2889 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:43.205004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-07a054d2-d6bf-4fe1-9da2-32f1a43c2889 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:43.215399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-07a054d2-d6bf-4fe1-9da2-32f1a43c2889 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.173104 Aug 28 01:28:43.215990 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1735/3243] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:43 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:44.237552 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bb9faffb-fea6-4928-8d65-992d49fdd277 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:44.238198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bb9faffb-fea6-4928-8d65-992d49fdd277 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:44.245754 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb9faffb-fea6-4928-8d65-992d49fdd277 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:44.246232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb9faffb-fea6-4928-8d65-992d49fdd277 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:44.246668 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb9faffb-fea6-4928-8d65-992d49fdd277 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:44.247373 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb9faffb-fea6-4928-8d65-992d49fdd277 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:44.247821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb9faffb-fea6-4928-8d65-992d49fdd277 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:44.248240 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bb9faffb-fea6-4928-8d65-992d49fdd277 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:44.400973 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bb9faffb-fea6-4928-8d65-992d49fdd277 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:44.409663 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bb9faffb-fea6-4928-8d65-992d49fdd277 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.176560 Aug 28 01:28:44.410232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1509/3244] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:44 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:28:45.431023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e525515b-f222-4348-9e2a-66d8a108101d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:45.431647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e525515b-f222-4348-9e2a-66d8a108101d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:45.438860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e525515b-f222-4348-9e2a-66d8a108101d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:45.439349 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e525515b-f222-4348-9e2a-66d8a108101d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:45.439785 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e525515b-f222-4348-9e2a-66d8a108101d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:45.440477 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e525515b-f222-4348-9e2a-66d8a108101d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:45.440927 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e525515b-f222-4348-9e2a-66d8a108101d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:45.441378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e525515b-f222-4348-9e2a-66d8a108101d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:45.588981 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e525515b-f222-4348-9e2a-66d8a108101d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:45.598307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e525515b-f222-4348-9e2a-66d8a108101d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.171144 Aug 28 01:28:45.598928 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1736/3245] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:45 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:46.620558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c54625d6-b6a5-4f28-89db-c1e802269061 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:46.621214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c54625d6-b6a5-4f28-89db-c1e802269061 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:46.628200 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c54625d6-b6a5-4f28-89db-c1e802269061 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:46.628693 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c54625d6-b6a5-4f28-89db-c1e802269061 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:46.629163 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c54625d6-b6a5-4f28-89db-c1e802269061 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:46.629874 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c54625d6-b6a5-4f28-89db-c1e802269061 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:46.630321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c54625d6-b6a5-4f28-89db-c1e802269061 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:46.630819 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c54625d6-b6a5-4f28-89db-c1e802269061 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:46.775048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c54625d6-b6a5-4f28-89db-c1e802269061 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:46.783952 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c54625d6-b6a5-4f28-89db-c1e802269061 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.167300 Aug 28 01:28:46.784529 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1510/3246] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:46 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:47.807578 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-bb4ca193-ef8b-4b37-b00b-9dd89b7db4f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:47.808632 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-bb4ca193-ef8b-4b37-b00b-9dd89b7db4f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:47.820015 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bb4ca193-ef8b-4b37-b00b-9dd89b7db4f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:47.820898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bb4ca193-ef8b-4b37-b00b-9dd89b7db4f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:47.821705 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bb4ca193-ef8b-4b37-b00b-9dd89b7db4f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:47.822967 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bb4ca193-ef8b-4b37-b00b-9dd89b7db4f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:47.823781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bb4ca193-ef8b-4b37-b00b-9dd89b7db4f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:47.824550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-bb4ca193-ef8b-4b37-b00b-9dd89b7db4f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:47.995434 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-bb4ca193-ef8b-4b37-b00b-9dd89b7db4f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:48.006506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-bb4ca193-ef8b-4b37-b00b-9dd89b7db4f7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.204129 Aug 28 01:28:48.007071 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1737/3247] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:47 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:28:49.028804 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b4a67204-4c90-4efa-888f-fd1666c9c015 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:49.029502 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b4a67204-4c90-4efa-888f-fd1666c9c015 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:49.036937 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4a67204-4c90-4efa-888f-fd1666c9c015 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:49.037464 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4a67204-4c90-4efa-888f-fd1666c9c015 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:49.037884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4a67204-4c90-4efa-888f-fd1666c9c015 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:49.038591 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4a67204-4c90-4efa-888f-fd1666c9c015 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:49.039048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4a67204-4c90-4efa-888f-fd1666c9c015 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:49.039452 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4a67204-4c90-4efa-888f-fd1666c9c015 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:49.191686 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b4a67204-4c90-4efa-888f-fd1666c9c015 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:49.201567 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b4a67204-4c90-4efa-888f-fd1666c9c015 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.177040 Aug 28 01:28:49.202174 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1511/3248] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:49 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:50.223453 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-55141810-fc50-40a0-9706-045b9149213d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:50.224093 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-55141810-fc50-40a0-9706-045b9149213d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:50.231583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-55141810-fc50-40a0-9706-045b9149213d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:50.232077 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-55141810-fc50-40a0-9706-045b9149213d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:50.232507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-55141810-fc50-40a0-9706-045b9149213d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:50.233308 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-55141810-fc50-40a0-9706-045b9149213d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:50.233784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-55141810-fc50-40a0-9706-045b9149213d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:50.234205 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-55141810-fc50-40a0-9706-045b9149213d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:50.386634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-55141810-fc50-40a0-9706-045b9149213d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:50.396193 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-55141810-fc50-40a0-9706-045b9149213d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.176725 Aug 28 01:28:50.396807 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1738/3249] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:50 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:51.419615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ded15e1d-f736-4eb5-9690-560e7b8600f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:51.420480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ded15e1d-f736-4eb5-9690-560e7b8600f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:51.423351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=45,nova_api:UPDATE=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:28:51.429827 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ded15e1d-f736-4eb5-9690-560e7b8600f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:51.430579 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ded15e1d-f736-4eb5-9690-560e7b8600f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:51.431080 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ded15e1d-f736-4eb5-9690-560e7b8600f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:51.431802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ded15e1d-f736-4eb5-9690-560e7b8600f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:51.432254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ded15e1d-f736-4eb5-9690-560e7b8600f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:51.432682 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ded15e1d-f736-4eb5-9690-560e7b8600f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:51.599492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ded15e1d-f736-4eb5-9690-560e7b8600f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:51.603132 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:28:51.615603 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ded15e1d-f736-4eb5-9690-560e7b8600f9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.200140 Aug 28 01:28:51.616505 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1512/3250] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:51 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:52.638396 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3931695d-b4fd-427e-892e-02eec0bac440 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:52.639162 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3931695d-b4fd-427e-892e-02eec0bac440 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:52.646412 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3931695d-b4fd-427e-892e-02eec0bac440 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:52.646925 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3931695d-b4fd-427e-892e-02eec0bac440 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:52.647357 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3931695d-b4fd-427e-892e-02eec0bac440 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:52.648053 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3931695d-b4fd-427e-892e-02eec0bac440 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:52.648502 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3931695d-b4fd-427e-892e-02eec0bac440 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:52.648923 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3931695d-b4fd-427e-892e-02eec0bac440 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:52.787425 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3931695d-b4fd-427e-892e-02eec0bac440 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:52.796805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3931695d-b4fd-427e-892e-02eec0bac440 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.162350 Aug 28 01:28:52.797676 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1739/3251] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:52 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:53.820514 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-70bdc1b8-5d45-46a8-ab50-672845d59441 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:53.821227 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-70bdc1b8-5d45-46a8-ab50-672845d59441 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:53.828601 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70bdc1b8-5d45-46a8-ab50-672845d59441 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:53.829267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70bdc1b8-5d45-46a8-ab50-672845d59441 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:53.829587 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70bdc1b8-5d45-46a8-ab50-672845d59441 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:53.830302 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70bdc1b8-5d45-46a8-ab50-672845d59441 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:53.830757 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70bdc1b8-5d45-46a8-ab50-672845d59441 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:53.831183 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-70bdc1b8-5d45-46a8-ab50-672845d59441 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:53.977185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-70bdc1b8-5d45-46a8-ab50-672845d59441 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:53.986179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-70bdc1b8-5d45-46a8-ab50-672845d59441 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.169810 Aug 28 01:28:53.986770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1513/3252] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:53 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:55.008222 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ce2a0753-0c63-4611-a561-22a33780da32 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:55.009084 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ce2a0753-0c63-4611-a561-22a33780da32 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:55.017234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce2a0753-0c63-4611-a561-22a33780da32 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:55.017780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce2a0753-0c63-4611-a561-22a33780da32 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:55.018251 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce2a0753-0c63-4611-a561-22a33780da32 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:55.019109 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce2a0753-0c63-4611-a561-22a33780da32 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:55.019597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce2a0753-0c63-4611-a561-22a33780da32 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:55.020150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce2a0753-0c63-4611-a561-22a33780da32 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:55.167525 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ce2a0753-0c63-4611-a561-22a33780da32 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:55.176826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ce2a0753-0c63-4611-a561-22a33780da32 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.172504 Aug 28 01:28:55.177439 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1740/3253] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:55 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:56.199145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-6499215a-8249-49d5-a939-76fcb9eef7a9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:56.199941 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-6499215a-8249-49d5-a939-76fcb9eef7a9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:56.207291 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6499215a-8249-49d5-a939-76fcb9eef7a9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:56.207782 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6499215a-8249-49d5-a939-76fcb9eef7a9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:56.208217 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6499215a-8249-49d5-a939-76fcb9eef7a9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:56.208950 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6499215a-8249-49d5-a939-76fcb9eef7a9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:56.209440 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6499215a-8249-49d5-a939-76fcb9eef7a9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:56.209862 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-6499215a-8249-49d5-a939-76fcb9eef7a9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:56.360293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-6499215a-8249-49d5-a939-76fcb9eef7a9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:56.370007 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-6499215a-8249-49d5-a939-76fcb9eef7a9 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.174881 Aug 28 01:28:56.370616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1514/3254] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:56 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:57.404082 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8d38ce6b-e078-4564-8c44-a8edf9cf5b34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:57.404734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8d38ce6b-e078-4564-8c44-a8edf9cf5b34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:57.411997 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d38ce6b-e078-4564-8c44-a8edf9cf5b34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:57.412642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d38ce6b-e078-4564-8c44-a8edf9cf5b34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:57.413380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d38ce6b-e078-4564-8c44-a8edf9cf5b34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:57.414225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d38ce6b-e078-4564-8c44-a8edf9cf5b34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:57.414676 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d38ce6b-e078-4564-8c44-a8edf9cf5b34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:57.415093 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8d38ce6b-e078-4564-8c44-a8edf9cf5b34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:57.551198 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8d38ce6b-e078-4564-8c44-a8edf9cf5b34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:57.559996 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8d38ce6b-e078-4564-8c44-a8edf9cf5b34 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.159903 Aug 28 01:28:57.560584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1741/3255] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:57 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:58.582606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-46e3416c-f6a2-47ad-af09-5df8d888306b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:58.583308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-46e3416c-f6a2-47ad-af09-5df8d888306b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:58.590543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-46e3416c-f6a2-47ad-af09-5df8d888306b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:58.591069 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-46e3416c-f6a2-47ad-af09-5df8d888306b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:58.591505 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-46e3416c-f6a2-47ad-af09-5df8d888306b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:58.592205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-46e3416c-f6a2-47ad-af09-5df8d888306b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:58.592801 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-46e3416c-f6a2-47ad-af09-5df8d888306b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:58.593269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-46e3416c-f6a2-47ad-af09-5df8d888306b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:58.742555 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-46e3416c-f6a2-47ad-af09-5df8d888306b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:58.751822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-46e3416c-f6a2-47ad-af09-5df8d888306b tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.173397 Aug 28 01:28:58.752483 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1515/3256] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:58 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:28:59.774319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4fa3929e-d606-4213-ad4a-ea3805b0bbd3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:28:59.774995 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4fa3929e-d606-4213-ad4a-ea3805b0bbd3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:28:59.782162 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fa3929e-d606-4213-ad4a-ea3805b0bbd3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:59.782556 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fa3929e-d606-4213-ad4a-ea3805b0bbd3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:59.782990 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fa3929e-d606-4213-ad4a-ea3805b0bbd3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:59.783693 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fa3929e-d606-4213-ad4a-ea3805b0bbd3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:28:59.784142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fa3929e-d606-4213-ad4a-ea3805b0bbd3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:28:59.784564 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4fa3929e-d606-4213-ad4a-ea3805b0bbd3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:28:59.942989 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4fa3929e-d606-4213-ad4a-ea3805b0bbd3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:28:59.955683 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4fa3929e-d606-4213-ad4a-ea3805b0bbd3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.185255 Aug 28 01:28:59.956283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1742/3257] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:28:59 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:29:00.977955 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-253e24cf-1663-4e2e-8027-25c096f38da0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:29:00.978682 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-253e24cf-1663-4e2e-8027-25c096f38da0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:29:00.988278 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-253e24cf-1663-4e2e-8027-25c096f38da0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:00.988881 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-253e24cf-1663-4e2e-8027-25c096f38da0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:00.989384 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-253e24cf-1663-4e2e-8027-25c096f38da0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:00.990121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-253e24cf-1663-4e2e-8027-25c096f38da0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:00.990577 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-253e24cf-1663-4e2e-8027-25c096f38da0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:00.991002 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-253e24cf-1663-4e2e-8027-25c096f38da0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:01.134132 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-253e24cf-1663-4e2e-8027-25c096f38da0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:29:01.143132 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-253e24cf-1663-4e2e-8027-25c096f38da0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.169178 Aug 28 01:29:01.143714 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1516/3258] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:29:00 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:29:02.164359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9a525f5e-ab63-48f9-affb-86fab8b26b39 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:29:02.165050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9a525f5e-ab63-48f9-affb-86fab8b26b39 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:29:02.172078 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a525f5e-ab63-48f9-affb-86fab8b26b39 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:02.172562 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a525f5e-ab63-48f9-affb-86fab8b26b39 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:02.173031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a525f5e-ab63-48f9-affb-86fab8b26b39 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:02.173749 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a525f5e-ab63-48f9-affb-86fab8b26b39 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:02.174213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a525f5e-ab63-48f9-affb-86fab8b26b39 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:02.174641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9a525f5e-ab63-48f9-affb-86fab8b26b39 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:02.317443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9a525f5e-ab63-48f9-affb-86fab8b26b39 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:29:02.326233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9a525f5e-ab63-48f9-affb-86fab8b26b39 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.165982 Aug 28 01:29:02.326819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1743/3259] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:29:02 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:29:03.349137 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-682f79b4-9986-4d70-9860-c2a0758bad04 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:29:03.349855 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-682f79b4-9986-4d70-9860-c2a0758bad04 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:29:03.357355 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-682f79b4-9986-4d70-9860-c2a0758bad04 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:03.357858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-682f79b4-9986-4d70-9860-c2a0758bad04 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:03.358420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-682f79b4-9986-4d70-9860-c2a0758bad04 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:03.359135 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-682f79b4-9986-4d70-9860-c2a0758bad04 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:03.359586 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-682f79b4-9986-4d70-9860-c2a0758bad04 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:03.360011 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-682f79b4-9986-4d70-9860-c2a0758bad04 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:03.514010 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-682f79b4-9986-4d70-9860-c2a0758bad04 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:29:03.523086 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-682f79b4-9986-4d70-9860-c2a0758bad04 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.177917 Aug 28 01:29:03.523669 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1517/3260] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:29:03 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:29:04.544583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-18e2d541-77f6-46bb-90d3-d20270e0c0e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:29:04.545254 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-18e2d541-77f6-46bb-90d3-d20270e0c0e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:29:04.552345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-18e2d541-77f6-46bb-90d3-d20270e0c0e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:04.552820 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-18e2d541-77f6-46bb-90d3-d20270e0c0e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:04.553278 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-18e2d541-77f6-46bb-90d3-d20270e0c0e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:04.553981 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-18e2d541-77f6-46bb-90d3-d20270e0c0e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:04.554427 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-18e2d541-77f6-46bb-90d3-d20270e0c0e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:04.554844 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-18e2d541-77f6-46bb-90d3-d20270e0c0e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:04.697506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-18e2d541-77f6-46bb-90d3-d20270e0c0e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:29:04.706417 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-18e2d541-77f6-46bb-90d3-d20270e0c0e0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.165662 Aug 28 01:29:04.706984 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1744/3261] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:29:04 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:29:05.728175 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9f13ae96-4dac-41f0-b773-7e4c4d2d4a8e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:29:05.728855 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9f13ae96-4dac-41f0-b773-7e4c4d2d4a8e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:29:05.735876 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f13ae96-4dac-41f0-b773-7e4c4d2d4a8e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:05.736368 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f13ae96-4dac-41f0-b773-7e4c4d2d4a8e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:05.736796 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f13ae96-4dac-41f0-b773-7e4c4d2d4a8e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:05.737548 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f13ae96-4dac-41f0-b773-7e4c4d2d4a8e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:05.737997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f13ae96-4dac-41f0-b773-7e4c4d2d4a8e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:05.739007 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9f13ae96-4dac-41f0-b773-7e4c4d2d4a8e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:05.893920 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9f13ae96-4dac-41f0-b773-7e4c4d2d4a8e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:29:05.904408 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9f13ae96-4dac-41f0-b773-7e4c4d2d4a8e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.180142 Aug 28 01:29:05.905175 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1518/3262] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:29:05 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:29:06.926381 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cef79b1a-5745-414a-9f7d-f3fa8a0d80a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:29:06.927038 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cef79b1a-5745-414a-9f7d-f3fa8a0d80a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:29:06.930072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=28 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:29:06.935492 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cef79b1a-5745-414a-9f7d-f3fa8a0d80a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:06.935970 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cef79b1a-5745-414a-9f7d-f3fa8a0d80a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:06.936827 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cef79b1a-5745-414a-9f7d-f3fa8a0d80a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:06.937549 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cef79b1a-5745-414a-9f7d-f3fa8a0d80a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:06.937992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cef79b1a-5745-414a-9f7d-f3fa8a0d80a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:06.938414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cef79b1a-5745-414a-9f7d-f3fa8a0d80a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:07.085909 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-cef79b1a-5745-414a-9f7d-f3fa8a0d80a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:29:07.094943 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cef79b1a-5745-414a-9f7d-f3fa8a0d80a3 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.172434 Aug 28 01:29:07.095535 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1745/3263] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:29:06 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:29:08.116533 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-aef0fc7f-441d-4b0e-bb6c-6b1afc626e88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:29:08.117210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-aef0fc7f-441d-4b0e-bb6c-6b1afc626e88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:29:08.125010 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aef0fc7f-441d-4b0e-bb6c-6b1afc626e88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:08.125507 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aef0fc7f-441d-4b0e-bb6c-6b1afc626e88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:08.125936 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aef0fc7f-441d-4b0e-bb6c-6b1afc626e88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:08.126647 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aef0fc7f-441d-4b0e-bb6c-6b1afc626e88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:08.127089 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aef0fc7f-441d-4b0e-bb6c-6b1afc626e88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:08.127506 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-aef0fc7f-441d-4b0e-bb6c-6b1afc626e88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:08.270753 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-aef0fc7f-441d-4b0e-bb6c-6b1afc626e88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:29:08.279429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-aef0fc7f-441d-4b0e-bb6c-6b1afc626e88 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.166892 Aug 28 01:29:08.280012 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1519/3264] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:29:08 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:29:09.300446 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5d6f18dc-20d6-4a36-8a57-dee54b610cfb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:29:09.301167 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5d6f18dc-20d6-4a36-8a57-dee54b610cfb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:29:09.308159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d6f18dc-20d6-4a36-8a57-dee54b610cfb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:09.308652 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d6f18dc-20d6-4a36-8a57-dee54b610cfb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:09.309120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d6f18dc-20d6-4a36-8a57-dee54b610cfb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:09.309834 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d6f18dc-20d6-4a36-8a57-dee54b610cfb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:09.310282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d6f18dc-20d6-4a36-8a57-dee54b610cfb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:09.310700 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5d6f18dc-20d6-4a36-8a57-dee54b610cfb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:09.473485 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5d6f18dc-20d6-4a36-8a57-dee54b610cfb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:29:09.482105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5d6f18dc-20d6-4a36-8a57-dee54b610cfb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.185678 Aug 28 01:29:09.482718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1746/3265] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:29:09 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:29:10.503969 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-40d9c00e-eb5e-4764-ad34-def92bd7fbdc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:29:10.504581 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-40d9c00e-eb5e-4764-ad34-def92bd7fbdc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:29:10.513382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-40d9c00e-eb5e-4764-ad34-def92bd7fbdc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:10.513968 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-40d9c00e-eb5e-4764-ad34-def92bd7fbdc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:10.514553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-40d9c00e-eb5e-4764-ad34-def92bd7fbdc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:10.515480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-40d9c00e-eb5e-4764-ad34-def92bd7fbdc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:10.516100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-40d9c00e-eb5e-4764-ad34-def92bd7fbdc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:10.516519 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-40d9c00e-eb5e-4764-ad34-def92bd7fbdc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:10.661882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-40d9c00e-eb5e-4764-ad34-def92bd7fbdc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:29:10.670670 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-40d9c00e-eb5e-4764-ad34-def92bd7fbdc tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.170516 Aug 28 01:29:10.671326 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1520/3266] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:29:10 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:29:11.694300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c3b839ae-6f38-4d05-9e4b-7af045fbc658 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:29:11.694949 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c3b839ae-6f38-4d05-9e4b-7af045fbc658 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:29:11.701911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3b839ae-6f38-4d05-9e4b-7af045fbc658 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:11.702395 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3b839ae-6f38-4d05-9e4b-7af045fbc658 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:11.702825 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3b839ae-6f38-4d05-9e4b-7af045fbc658 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:11.703528 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3b839ae-6f38-4d05-9e4b-7af045fbc658 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:11.703977 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3b839ae-6f38-4d05-9e4b-7af045fbc658 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:11.704395 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c3b839ae-6f38-4d05-9e4b-7af045fbc658 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:11.849373 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c3b839ae-6f38-4d05-9e4b-7af045fbc658 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:29:11.858161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c3b839ae-6f38-4d05-9e4b-7af045fbc658 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.168569 Aug 28 01:29:11.858776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1747/3267] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:29:11 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:29:12.880915 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-57ee83f5-3168-4983-be86-0bd8079529b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:29:12.882136 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-57ee83f5-3168-4983-be86-0bd8079529b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:29:12.893763 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-57ee83f5-3168-4983-be86-0bd8079529b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:12.894251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-57ee83f5-3168-4983-be86-0bd8079529b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:12.894683 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-57ee83f5-3168-4983-be86-0bd8079529b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:12.895384 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-57ee83f5-3168-4983-be86-0bd8079529b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:12.895834 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-57ee83f5-3168-4983-be86-0bd8079529b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:12.896249 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-57ee83f5-3168-4983-be86-0bd8079529b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:13.043572 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-57ee83f5-3168-4983-be86-0bd8079529b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:29:13.056928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-57ee83f5-3168-4983-be86-0bd8079529b7 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.180579 Aug 28 01:29:13.057574 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1521/3268] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:29:12 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:29:14.080856 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-4506f80e-d9e1-4f70-a7b1-00989909111d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:29:14.081493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-4506f80e-d9e1-4f70-a7b1-00989909111d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:29:14.088414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4506f80e-d9e1-4f70-a7b1-00989909111d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:14.088896 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4506f80e-d9e1-4f70-a7b1-00989909111d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:14.089356 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4506f80e-d9e1-4f70-a7b1-00989909111d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:14.090231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4506f80e-d9e1-4f70-a7b1-00989909111d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:14.090683 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4506f80e-d9e1-4f70-a7b1-00989909111d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:14.091098 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-4506f80e-d9e1-4f70-a7b1-00989909111d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:14.094351 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=80 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:29:14.235032 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-4506f80e-d9e1-4f70-a7b1-00989909111d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:29:14.244270 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-4506f80e-d9e1-4f70-a7b1-00989909111d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.168761 Aug 28 01:29:14.244869 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1748/3269] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:29:14 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:29:15.269676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8e5fdf9a-86fb-41c9-b294-c374d9f3758a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:29:15.270704 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8e5fdf9a-86fb-41c9-b294-c374d9f3758a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:29:15.283168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e5fdf9a-86fb-41c9-b294-c374d9f3758a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:15.283645 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e5fdf9a-86fb-41c9-b294-c374d9f3758a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:15.284086 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e5fdf9a-86fb-41c9-b294-c374d9f3758a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:15.284789 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e5fdf9a-86fb-41c9-b294-c374d9f3758a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:15.285260 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e5fdf9a-86fb-41c9-b294-c374d9f3758a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:15.285675 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8e5fdf9a-86fb-41c9-b294-c374d9f3758a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:15.426773 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8e5fdf9a-86fb-41c9-b294-c374d9f3758a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:29:15.437094 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8e5fdf9a-86fb-41c9-b294-c374d9f3758a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.173695 Aug 28 01:29:15.437965 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1522/3270] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:29:15 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:29:16.462366 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e10166e6-b7a4-4fd2-9d6d-32886c2bcee5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:29:16.463010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e10166e6-b7a4-4fd2-9d6d-32886c2bcee5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:29:16.470293 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e10166e6-b7a4-4fd2-9d6d-32886c2bcee5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:16.470781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e10166e6-b7a4-4fd2-9d6d-32886c2bcee5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:16.471218 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e10166e6-b7a4-4fd2-9d6d-32886c2bcee5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:16.471906 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e10166e6-b7a4-4fd2-9d6d-32886c2bcee5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:16.472432 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e10166e6-b7a4-4fd2-9d6d-32886c2bcee5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:16.472987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e10166e6-b7a4-4fd2-9d6d-32886c2bcee5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:16.619550 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e10166e6-b7a4-4fd2-9d6d-32886c2bcee5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:29:16.628111 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e10166e6-b7a4-4fd2-9d6d-32886c2bcee5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.169596 Aug 28 01:29:16.628680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1749/3271] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:29:16 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:29:17.651606 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0214aef7-eb13-4773-849f-1a8625e2abdb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:29:17.652398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0214aef7-eb13-4773-849f-1a8625e2abdb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 03969ac5-934e-4201-8087-0927eae2bbd2] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:29:17.662577 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0214aef7-eb13-4773-849f-1a8625e2abdb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:17.663340 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0214aef7-eb13-4773-849f-1a8625e2abdb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:17.664053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0214aef7-eb13-4773-849f-1a8625e2abdb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:17.665126 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0214aef7-eb13-4773-849f-1a8625e2abdb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:17.665754 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0214aef7-eb13-4773-849f-1a8625e2abdb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:17.666354 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0214aef7-eb13-4773-849f-1a8625e2abdb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:17.813180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0214aef7-eb13-4773-849f-1a8625e2abdb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:29:17.821828 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0214aef7-eb13-4773-849f-1a8625e2abdb tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2" status: 200 len: 1735 microversion: 2.1 time: 0.175817 Aug 28 01:29:17.822439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1523/3272] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:29:17 2022] GET /compute/v2.1/servers/03969ac5-934e-4201-8087-0927eae2bbd2 => generated 1735 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:29:17.842191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5cc88989-72de-4402-ba0e-9c36693e6be2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '.version_select at 0x7f0e72066430>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:29:17.865448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5cc88989-72de-4402-ba0e-9c36693e6be2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-225780864" status: 202 len: 0 microversion: 2.1 time: 0.026945 Aug 28 01:29:17.876218 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1750/3273] 10.222.0.42 () {58 vars in 1218 bytes} [Sun Aug 28 01:29:17 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-225780864 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 01:29:19.252161 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-5d923bb1-c3e1-45c4-87fe-bdb8e85b9f27 req-339ac9c5-6581-433e-831f-73f238f8f133 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "03969ac5-934e-4201-8087-0927eae2bbd2", "tag": "b9bc50e7-87a3-480a-8c62-87fe75411e15"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:29:19.262048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-5d923bb1-c3e1-45c4-87fe-bdb8e85b9f27 req-339ac9c5-6581-433e-831f-73f238f8f133 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:29:19.262516 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-5d923bb1-c3e1-45c4-87fe-bdb8e85b9f27 req-339ac9c5-6581-433e-831f-73f238f8f133 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:29:19.262938 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-5d923bb1-c3e1-45c4-87fe-bdb8e85b9f27 req-339ac9c5-6581-433e-831f-73f238f8f133 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:29:19.274207 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-5d923bb1-c3e1-45c4-87fe-bdb8e85b9f27 req-339ac9c5-6581-433e-831f-73f238f8f133 service nova] Creating event network-changed:b9bc50e7-87a3-480a-8c62-87fe75411e15 for instance 03969ac5-934e-4201-8087-0927eae2bbd2 on hv-cinder-maste Aug 28 01:29:19.279377 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-5d923bb1-c3e1-45c4-87fe-bdb8e85b9f27 req-339ac9c5-6581-433e-831f-73f238f8f133 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.145385 Aug 28 01:29:19.279970 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1524/3274] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:29:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:30:54.114609 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=11 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:30:54.118129 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=12,nova_api:DELETE=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:30:54.127160 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0a12b71a-e39e-428b-8c47-9e564618516d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:30:54.127703 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0a12b71a-e39e-428b-8c47-9e564618516d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:30:54.134394 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a12b71a-e39e-428b-8c47-9e564618516d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:30:54.134882 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a12b71a-e39e-428b-8c47-9e564618516d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:30:54.135320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a12b71a-e39e-428b-8c47-9e564618516d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:30:54.136023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a12b71a-e39e-428b-8c47-9e564618516d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:30:54.136468 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a12b71a-e39e-428b-8c47-9e564618516d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:30:54.136892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0a12b71a-e39e-428b-8c47-9e564618516d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:30:54.163366 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0a12b71a-e39e-428b-8c47-9e564618516d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Going to try to terminate instance {{(pid=120525) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 28 01:30:54.214897 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.compute.api [None req-0a12b71a-e39e-428b-8c47-9e564618516d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Instance is already in deleting state, ignoring this request Aug 28 01:30:54.215737 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0a12b71a-e39e-428b-8c47-9e564618516d tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "DELETE /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 204 len: 0 microversion: 2.20 time: 0.104867 Aug 28 01:30:54.216221 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1751/3275] 10.222.0.42 () {60 vars in 1280 bytes} [Sun Aug 28 01:30:54 2022] DELETE /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 0 bytes in 106 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Aug 28 01:30:54.231618 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR oslo.messaging._drivers.impl_rabbit [-] [a75834a5-0c81-44d8-993d-e8453085eab8] AMQP server on 10.222.0.42:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Aug 28 01:30:54.234599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 28 01:30:54.234791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Sun Aug 28 01:30:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df (ip 10.222.0.42) !!! Aug 28 01:30:54.237808 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=25 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:30:54.239043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=69 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:30:54.248126 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-94be0515-0c8d-424a-a2c4-ea6f64b32e1e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:30:54.248829 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-94be0515-0c8d-424a-a2c4-ea6f64b32e1e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:30:54.260659 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94be0515-0c8d-424a-a2c4-ea6f64b32e1e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:30:54.261168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94be0515-0c8d-424a-a2c4-ea6f64b32e1e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:30:54.261596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94be0515-0c8d-424a-a2c4-ea6f64b32e1e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:30:54.262284 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94be0515-0c8d-424a-a2c4-ea6f64b32e1e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:30:54.262727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94be0515-0c8d-424a-a2c4-ea6f64b32e1e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:30:54.263141 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94be0515-0c8d-424a-a2c4-ea6f64b32e1e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:30:54.398466 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-94be0515-0c8d-424a-a2c4-ea6f64b32e1e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:30:54.412527 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-94be0515-0c8d-424a-a2c4-ea6f64b32e1e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:30:54.414105 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-94be0515-0c8d-424a-a2c4-ea6f64b32e1e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:30:54.415241 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-94be0515-0c8d-424a-a2c4-ea6f64b32e1e tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1603 microversion: 2.20 time: 0.187266 Aug 28 01:30:54.415794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1525/3276] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:30:54 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1603 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:30:55.438659 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-57bd9fef-c6f1-46ea-9e4d-b24da403e27f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:30:55.439306 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-57bd9fef-c6f1-46ea-9e4d-b24da403e27f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:30:55.446371 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-57bd9fef-c6f1-46ea-9e4d-b24da403e27f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:30:55.446867 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-57bd9fef-c6f1-46ea-9e4d-b24da403e27f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:30:55.447305 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-57bd9fef-c6f1-46ea-9e4d-b24da403e27f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:30:55.448038 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-57bd9fef-c6f1-46ea-9e4d-b24da403e27f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:30:55.449595 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-57bd9fef-c6f1-46ea-9e4d-b24da403e27f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:30:55.452622 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-57bd9fef-c6f1-46ea-9e4d-b24da403e27f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:30:55.601286 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-57bd9fef-c6f1-46ea-9e4d-b24da403e27f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:30:55.610120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-57bd9fef-c6f1-46ea-9e4d-b24da403e27f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:30:55.611991 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-57bd9fef-c6f1-46ea-9e4d-b24da403e27f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:30:55.613188 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-57bd9fef-c6f1-46ea-9e4d-b24da403e27f tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1603 microversion: 2.20 time: 0.178953 Aug 28 01:30:55.614747 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1752/3277] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:30:55 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1603 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:30:56.352294 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO oslo.messaging._drivers.impl_rabbit [-] [a75834a5-0c81-44d8-993d-e8453085eab8] Reconnected to AMQP server on 10.222.0.42:5672 via [amqp] client with port 47974. Aug 28 01:30:56.409180 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-79ec8c48-50c5-44c2-9cce-8323337d1e7e req-f75226d2-211f-4d7a-983a-f71a0333482d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dd61123b-7a96-4c9e-936c-5794a4fdd7df", "name": "network-vif-deleted", "tag": "522d4a1b-32bc-4106-afb0-5995aecc53c0"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:30:56.419177 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-79ec8c48-50c5-44c2-9cce-8323337d1e7e req-f75226d2-211f-4d7a-983a-f71a0333482d service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:30:56.419658 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-79ec8c48-50c5-44c2-9cce-8323337d1e7e req-f75226d2-211f-4d7a-983a-f71a0333482d service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:30:56.420083 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-79ec8c48-50c5-44c2-9cce-8323337d1e7e req-f75226d2-211f-4d7a-983a-f71a0333482d service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:30:56.431710 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-79ec8c48-50c5-44c2-9cce-8323337d1e7e req-f75226d2-211f-4d7a-983a-f71a0333482d service nova] Creating event network-vif-deleted:522d4a1b-32bc-4106-afb0-5995aecc53c0 for instance dd61123b-7a96-4c9e-936c-5794a4fdd7df on hv-cinder-maste Aug 28 01:30:56.437573 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-79ec8c48-50c5-44c2-9cce-8323337d1e7e req-f75226d2-211f-4d7a-983a-f71a0333482d service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.120812 Aug 28 01:30:56.438173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1526/3278] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:30:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:30:56.637389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8bb68253-97a8-4703-b87b-ada8371b30dd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:30:56.639294 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8bb68253-97a8-4703-b87b-ada8371b30dd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:30:56.646473 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8bb68253-97a8-4703-b87b-ada8371b30dd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:30:56.646967 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8bb68253-97a8-4703-b87b-ada8371b30dd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:30:56.647410 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8bb68253-97a8-4703-b87b-ada8371b30dd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:30:56.648844 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8bb68253-97a8-4703-b87b-ada8371b30dd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:30:56.649586 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8bb68253-97a8-4703-b87b-ada8371b30dd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:30:56.649873 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8bb68253-97a8-4703-b87b-ada8371b30dd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:30:56.725502 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8bb68253-97a8-4703-b87b-ada8371b30dd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:30:56.738506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8bb68253-97a8-4703-b87b-ada8371b30dd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:30:56.739981 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8bb68253-97a8-4703-b87b-ada8371b30dd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:30:56.741871 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8bb68253-97a8-4703-b87b-ada8371b30dd tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1366 microversion: 2.20 time: 0.108502 Aug 28 01:30:56.742781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1753/3279] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:30:56 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1366 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:30:57.765883 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-da2ef37a-688a-4c44-b237-1c301a201121 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:30:57.766523 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-da2ef37a-688a-4c44-b237-1c301a201121 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:30:57.773505 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da2ef37a-688a-4c44-b237-1c301a201121 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:30:57.773996 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da2ef37a-688a-4c44-b237-1c301a201121 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:30:57.774426 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da2ef37a-688a-4c44-b237-1c301a201121 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:30:57.775127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da2ef37a-688a-4c44-b237-1c301a201121 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:30:57.775572 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da2ef37a-688a-4c44-b237-1c301a201121 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:30:57.775986 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da2ef37a-688a-4c44-b237-1c301a201121 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:30:57.817212 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da2ef37a-688a-4c44-b237-1c301a201121 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:30:57.817690 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da2ef37a-688a-4c44-b237-1c301a201121 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:30:57.818136 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-da2ef37a-688a-4c44-b237-1c301a201121 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:30:57.818589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.objects.instance [None req-da2ef37a-688a-4c44-b237-1c301a201121 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lazy-loading 'fault' on Instance uuid dd61123b-7a96-4c9e-936c-5794a4fdd7df {{(pid=120524) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:30:57.879549 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-da2ef37a-688a-4c44-b237-1c301a201121 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:30:57.891601 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-da2ef37a-688a-4c44-b237-1c301a201121 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:30:57.894482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-da2ef37a-688a-4c44-b237-1c301a201121 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '4894ef8b630e4bc084799c26096f7c66', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '19f069b9ad7a4a5984fa947ad755db0a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:30:57.895761 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-da2ef37a-688a-4c44-b237-1c301a201121 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 200 len: 1371 microversion: 2.20 time: 0.134871 Aug 28 01:30:57.896528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1527/3280] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:30:57 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 1371 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Aug 28 01:30:58.460666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-6692b3e7-a827-4d30-8bb6-0bd1c8f16bc2 req-772bbcc1-0034-4a63-aecb-1691a90f7908 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "03969ac5-934e-4201-8087-0927eae2bbd2", "name": "network-vif-deleted", "tag": "b9bc50e7-87a3-480a-8c62-87fe75411e15"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:30:58.472128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-6692b3e7-a827-4d30-8bb6-0bd1c8f16bc2 req-772bbcc1-0034-4a63-aecb-1691a90f7908 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:30:58.472672 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-6692b3e7-a827-4d30-8bb6-0bd1c8f16bc2 req-772bbcc1-0034-4a63-aecb-1691a90f7908 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:30:58.473320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-6692b3e7-a827-4d30-8bb6-0bd1c8f16bc2 req-772bbcc1-0034-4a63-aecb-1691a90f7908 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:30:58.495259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-6692b3e7-a827-4d30-8bb6-0bd1c8f16bc2 req-772bbcc1-0034-4a63-aecb-1691a90f7908 service nova] Creating event network-vif-deleted:b9bc50e7-87a3-480a-8c62-87fe75411e15 for instance 03969ac5-934e-4201-8087-0927eae2bbd2 on hv-cinder-maste Aug 28 01:30:58.506430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-6692b3e7-a827-4d30-8bb6-0bd1c8f16bc2 req-772bbcc1-0034-4a63-aecb-1691a90f7908 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055523 Aug 28 01:30:58.507397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1754/3281] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:30:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:30:58.923271 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4bc176e7-7c19-4890-b84e-2a259c1fd299 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:30:58.924176 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4bc176e7-7c19-4890-b84e-2a259c1fd299 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] [instance: dd61123b-7a96-4c9e-936c-5794a4fdd7df] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:30:58.932441 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4bc176e7-7c19-4890-b84e-2a259c1fd299 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:30:58.933563 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4bc176e7-7c19-4890-b84e-2a259c1fd299 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:30:58.934057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4bc176e7-7c19-4890-b84e-2a259c1fd299 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:30:58.935122 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4bc176e7-7c19-4890-b84e-2a259c1fd299 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:30:58.935594 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4bc176e7-7c19-4890-b84e-2a259c1fd299 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:30:58.936101 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4bc176e7-7c19-4890-b84e-2a259c1fd299 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:30:58.955045 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-4bc176e7-7c19-4890-b84e-2a259c1fd299 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] HTTP exception thrown: Instance dd61123b-7a96-4c9e-936c-5794a4fdd7df could not be found. Aug 28 01:30:58.956859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4bc176e7-7c19-4890-b84e-2a259c1fd299 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Request method failure captured: Aug 28 01:30:58.956859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df HTTP/1.1 Aug 28 01:30:58.956859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:30:58.956859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:30:58.956859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:30:58.956859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:30:58.956859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:30:58.956859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:30:58.956859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:30:58.956859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:30:58.956859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCrzjmVndE3J0T1bKuUY8Ipz6UdCdHTla_ZBX1kD9euBtohEB_ihuwpRHdKiiS67obfjsh2qj6v6fsgqePClTJKl_tEuVTP07R5_yii_rH6-mTIVEymXqmkZjC4lF12x3ZpjUNycUL0XFzkb5MxCQ1wXhvXTMZ4cpU8MxkUTfk5aOilw Aug 28 01:30:58.956859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:30:58.956859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:30:58.956859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:30:58.956859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:30:58.956859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Openstack-Nova-Api-Version: 2.20 Aug 28 01:30:58.956859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:30:58.956859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:30:58.956859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 19f069b9ad7a4a5984fa947ad755db0a Aug 28 01:30:58.956859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-349481608 Aug 28 01:30:58.956859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:30:58.956859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:30:58.958929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_19f069b9ad7a4a5984fa947ad755db0a", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/19f069b9ad7a4a5984fa947ad755db0a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/19f069b9ad7a4a5984fa947ad755db0a"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/19f069b9ad7a4a5984fa947ad755db0a"}]}] Aug 28 01:30:58.958929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-AttachVolumeShelveTestJSON-349481608 Aug 28 01:30:58.958929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 19f069b9ad7a4a5984fa947ad755db0a Aug 28 01:30:58.958929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-349481608 Aug 28 01:30:58.958929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-AttachVolumeShelveTestJSON-349481608-project Aug 28 01:30:58.958929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:30:58.958929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:30:58.958929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: 4894ef8b630e4bc084799c26096f7c66 Aug 28 01:30:58.958929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-AttachVolumeShelveTestJSON-349481608-project Aug 28 01:30:58.958929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:30:58.958929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Instance dd61123b-7a96-4c9e-936c-5794a4fdd7df could not be found. Aug 28 01:30:58.958929 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'id': 'dd61123b-7a96-4c9e-936c-5794a4fdd7df'} Aug 28 01:30:58.959919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:30:58.959919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:30:58.959919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:30:58.959919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:30:58.959919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:30:58.959919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:30:58.959919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:30:58.959919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:30:58.959919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:30:58.959919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:30:58.959919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dd61123b-7a96-4c9e-936c-5794a4fdd7df could not be found. Aug 28 01:30:58.959919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:30:58.959919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:30:58.959919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:30:58.959919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:30:58.959919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:30:58.959919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:30:58.959919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:30:58.959919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:30:58.959919 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:30:58.961524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:30:58.961524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 01:30:58.961524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 01:30:58.961524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:30:58.961524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:30:58.961524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:30:58.961524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:30:58.961524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dd61123b-7a96-4c9e-936c-5794a4fdd7df could not be found. Aug 28 01:30:58.961524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:30:58.961524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:30:58.961524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:30:58.961524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:30:58.961524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:30:58.961524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:30:58.961524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:30:58.961524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:30:58.961524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dd61123b-7a96-4c9e-936c-5794a4fdd7df could not be found. Aug 28 01:30:58.961524 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:30:58.963102 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4bc176e7-7c19-4890-b84e-2a259c1fd299 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] Returning 404 to user: Instance dd61123b-7a96-4c9e-936c-5794a4fdd7df could not be found. {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:30:58.963102 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4bc176e7-7c19-4890-b84e-2a259c1fd299 tempest-AttachVolumeShelveTestJSON-349481608 tempest-AttachVolumeShelveTestJSON-349481608-project] 10.222.0.42 "GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df" status: 404 len: 111 microversion: 2.20 time: 0.040988 Aug 28 01:30:58.963102 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1528/3282] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Aug 28 01:30:58 2022] GET /compute/v2.1/servers/dd61123b-7a96-4c9e-936c-5794a4fdd7df => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Aug 28 01:31:00.551233 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-57c688c6-062b-4f9c-a27d-4f77cde870d9 req-026310f9-7907-4aaa-9591-a5d7bcd404c8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2cbc0806-fe4b-4018-93bf-b4cf3c292bdc", "name": "network-vif-deleted", "tag": "a816f8eb-89c2-45b9-b35d-cf9bca6f4a7c"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:31:00.564684 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-57c688c6-062b-4f9c-a27d-4f77cde870d9 req-026310f9-7907-4aaa-9591-a5d7bcd404c8 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:31:00.565483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-57c688c6-062b-4f9c-a27d-4f77cde870d9 req-026310f9-7907-4aaa-9591-a5d7bcd404c8 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:31:00.566247 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-57c688c6-062b-4f9c-a27d-4f77cde870d9 req-026310f9-7907-4aaa-9591-a5d7bcd404c8 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:31:00.582807 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-57c688c6-062b-4f9c-a27d-4f77cde870d9 req-026310f9-7907-4aaa-9591-a5d7bcd404c8 service nova] Creating event network-vif-deleted:a816f8eb-89c2-45b9-b35d-cf9bca6f4a7c for instance 2cbc0806-fe4b-4018-93bf-b4cf3c292bdc on hv-cinder-maste Aug 28 01:31:00.588350 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-57c688c6-062b-4f9c-a27d-4f77cde870d9 req-026310f9-7907-4aaa-9591-a5d7bcd404c8 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042649 Aug 28 01:31:00.589172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1755/3283] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:31:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:31:04.270584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3b3f2eb0-4390-4269-a953-a789da6a699c tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] Action: 'create', calling method: .version_select at 0x7f0e6ea00ca0>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-323260166", "metadata": {"Type": "work"}}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:31:04.279232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3b3f2eb0-4390-4269-a953-a789da6a699c tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] Request method failure captured: Aug 28 01:31:04.279232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Aug 28 01:31:04.279232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:31:04.279232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:31:04.279232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:31:04.279232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 121 Aug 28 01:31:04.279232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:31:04.279232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:31:04.279232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:31:04.279232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:31:04.279232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCsVYKUfle16gH4n31Jsg8tJuvtIRKSj_kqW-XD5bIhbWZ9tVC8PkUl3WXoNh-a5kAji8U8xsYkAu4HXGbnrgeis5Q3rs66ZxGkttc2rVSsmLcMm1YhozNjmiOvHA8n7wvnav_KesgetGSlU3I5azZz_PJaqell1uwxDhJDFqwXVj9W8 Aug 28 01:31:04.279232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:31:04.279232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:31:04.279232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:31:04.279232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:31:04.279232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:31:04.279232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:31:04.279232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 765f789deef04206a3f89cfda8d1ee08 Aug 28 01:31:04.279232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-VolumesNegativeTest-1419196676 Aug 28 01:31:04.279232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:31:04.279232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:31:04.281179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_765f789deef04206a3f89cfda8d1ee08", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/765f789deef04206a3f89cfda8d1ee08"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/765f789deef04206a3f89cfda8d1ee08"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/765f789deef04206a3f89cfda8d1ee08"}]}] Aug 28 01:31:04.281179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-VolumesNegativeTest-1419196676 Aug 28 01:31:04.281179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 765f789deef04206a3f89cfda8d1ee08 Aug 28 01:31:04.281179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-VolumesNegativeTest-1419196676 Aug 28 01:31:04.281179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-VolumesNegativeTest-1419196676-project Aug 28 01:31:04.281179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:31:04.281179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:31:04.281179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: eebc2779346f4fdbbf3bb89027638c24 Aug 28 01:31:04.281179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-VolumesNegativeTest-1419196676-project Aug 28 01:31:04.281179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Aug 28 01:31:04.281179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-323260166", "metadata": {"Type": "work"}}} Aug 28 01:31:04.281179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: Aug 28 01:31:04.281179 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Aug 28 01:31:04.282311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-323260166', 'metadata': {'Type': 'work'}}}} Aug 28 01:31:04.282311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:31:04.282311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:31:04.282311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Aug 28 01:31:04.282311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Aug 28 01:31:04.282311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 28 01:31:04.282311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise error Aug 28 01:31:04.282311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Aug 28 01:31:04.282311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.282311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Aug 28 01:31:04.282311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Aug 28 01:31:04.282311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.282311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi On instance['volume']: Aug 28 01:31:04.282311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi '#$%' Aug 28 01:31:04.282311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.282311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:31:04.282311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.282311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:31:04.282311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:31:04.282311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:31:04.282311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:31:04.282311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:31:04.284144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 28 01:31:04.284144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 28 01:31:04.284144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:31:04.284144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:31:04.284144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Aug 28 01:31:04.284144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Aug 28 01:31:04.284144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Aug 28 01:31:04.284144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Aug 28 01:31:04.284144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Aug 28 01:31:04.284144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Aug 28 01:31:04.284144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Aug 28 01:31:04.284144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.284144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:31:04.284144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.284144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:31:04.284144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:31:04.284144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:31:04.284144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Aug 28 01:31:04.284144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Aug 28 01:31:04.285790 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Aug 28 01:31:04.285790 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.285790 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3b3f2eb0-4390-4269-a953-a789da6a699c tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:31:04.285790 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3b3f2eb0-4390-4269-a953-a789da6a699c tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] 10.222.0.42 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.132142 Aug 28 01:31:04.285790 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1529/3284] 10.222.0.42 () {60 vars in 1156 bytes} [Sun Aug 28 01:31:04 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 134 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 28 01:31:04.293739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c24a49cb-8e16-4434-9080-d727990f0c2c tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] Action: 'create', calling method: .version_select at 0x7f0e72096550>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-2127869256", "metadata": {"Type": "work"}}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:31:04.299814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c24a49cb-8e16-4434-9080-d727990f0c2c tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] Request method failure captured: Aug 28 01:31:04.299814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Aug 28 01:31:04.299814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:31:04.299814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:31:04.299814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:31:04.299814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 120 Aug 28 01:31:04.299814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:31:04.299814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:31:04.299814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:31:04.299814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:31:04.299814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCsVYKUfle16gH4n31Jsg8tJuvtIRKSj_kqW-XD5bIhbWZ9tVC8PkUl3WXoNh-a5kAji8U8xsYkAu4HXGbnrgeis5Q3rs66ZxGkttc2rVSsmLcMm1YhozNjmiOvHA8n7wvnav_KesgetGSlU3I5azZz_PJaqell1uwxDhJDFqwXVj9W8 Aug 28 01:31:04.299814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:31:04.299814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:31:04.299814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:31:04.299814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:31:04.299814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:31:04.299814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:31:04.299814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 765f789deef04206a3f89cfda8d1ee08 Aug 28 01:31:04.299814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-VolumesNegativeTest-1419196676 Aug 28 01:31:04.299814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:31:04.299814 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:31:04.301765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_765f789deef04206a3f89cfda8d1ee08", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/765f789deef04206a3f89cfda8d1ee08"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/765f789deef04206a3f89cfda8d1ee08"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/765f789deef04206a3f89cfda8d1ee08"}]}] Aug 28 01:31:04.301765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-VolumesNegativeTest-1419196676 Aug 28 01:31:04.301765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 765f789deef04206a3f89cfda8d1ee08 Aug 28 01:31:04.301765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-VolumesNegativeTest-1419196676 Aug 28 01:31:04.301765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-VolumesNegativeTest-1419196676-project Aug 28 01:31:04.301765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:31:04.301765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:31:04.301765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: eebc2779346f4fdbbf3bb89027638c24 Aug 28 01:31:04.301765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-VolumesNegativeTest-1419196676-project Aug 28 01:31:04.301765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Aug 28 01:31:04.301765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-2127869256", "metadata": {"Type": "work"}}} Aug 28 01:31:04.301765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: Aug 28 01:31:04.301765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Aug 28 01:31:04.302759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-2127869256', 'metadata': {'Type': 'work'}}}} Aug 28 01:31:04.302759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:31:04.302759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:31:04.302759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Aug 28 01:31:04.302759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Aug 28 01:31:04.302759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 28 01:31:04.302759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise error Aug 28 01:31:04.302759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Aug 28 01:31:04.302759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.302759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Aug 28 01:31:04.302759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Aug 28 01:31:04.302759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.302759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi On instance['volume']: Aug 28 01:31:04.302759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi '0' Aug 28 01:31:04.302759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.302759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:31:04.302759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.302759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:31:04.302759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:31:04.302759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:31:04.302759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:31:04.302759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:31:04.304364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 28 01:31:04.304364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 28 01:31:04.304364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:31:04.304364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:31:04.304364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Aug 28 01:31:04.304364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Aug 28 01:31:04.304364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Aug 28 01:31:04.304364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Aug 28 01:31:04.304364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Aug 28 01:31:04.304364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Aug 28 01:31:04.304364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Aug 28 01:31:04.304364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.304364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:31:04.304364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.304364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:31:04.304364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:31:04.304364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:31:04.304364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Aug 28 01:31:04.304364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Aug 28 01:31:04.305773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Aug 28 01:31:04.305773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.305773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c24a49cb-8e16-4434-9080-d727990f0c2c tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:31:04.305773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c24a49cb-8e16-4434-9080-d727990f0c2c tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] 10.222.0.42 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.013542 Aug 28 01:31:04.305773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1756/3285] 10.222.0.42 () {60 vars in 1156 bytes} [Sun Aug 28 01:31:04 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 15 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Aug 28 01:31:04.312305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4066bf6a-79f8-48af-bf7d-6a679e19599e tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] Action: 'create', calling method: .version_select at 0x7f0e6eaf1b80>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-840488307", "metadata": {"Type": "work"}}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:31:04.315894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4066bf6a-79f8-48af-bf7d-6a679e19599e tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] Request method failure captured: Aug 28 01:31:04.315894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Aug 28 01:31:04.315894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:31:04.315894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:31:04.315894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:31:04.315894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 118 Aug 28 01:31:04.315894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:31:04.315894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:31:04.315894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:31:04.315894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:31:04.315894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCsVYKUfle16gH4n31Jsg8tJuvtIRKSj_kqW-XD5bIhbWZ9tVC8PkUl3WXoNh-a5kAji8U8xsYkAu4HXGbnrgeis5Q3rs66ZxGkttc2rVSsmLcMm1YhozNjmiOvHA8n7wvnav_KesgetGSlU3I5azZz_PJaqell1uwxDhJDFqwXVj9W8 Aug 28 01:31:04.315894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:31:04.315894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:31:04.315894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:31:04.315894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:31:04.315894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:31:04.315894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:31:04.315894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 765f789deef04206a3f89cfda8d1ee08 Aug 28 01:31:04.315894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-VolumesNegativeTest-1419196676 Aug 28 01:31:04.315894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:31:04.315894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:31:04.317852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_765f789deef04206a3f89cfda8d1ee08", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/765f789deef04206a3f89cfda8d1ee08"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/765f789deef04206a3f89cfda8d1ee08"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/765f789deef04206a3f89cfda8d1ee08"}]}] Aug 28 01:31:04.317852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-VolumesNegativeTest-1419196676 Aug 28 01:31:04.317852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 765f789deef04206a3f89cfda8d1ee08 Aug 28 01:31:04.317852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-VolumesNegativeTest-1419196676 Aug 28 01:31:04.317852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-VolumesNegativeTest-1419196676-project Aug 28 01:31:04.317852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:31:04.317852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:31:04.317852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: eebc2779346f4fdbbf3bb89027638c24 Aug 28 01:31:04.317852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-VolumesNegativeTest-1419196676-project Aug 28 01:31:04.317852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Aug 28 01:31:04.317852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-840488307", "metadata": {"Type": "work"}}} Aug 28 01:31:04.317852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: Aug 28 01:31:04.317852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Aug 28 01:31:04.318911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-840488307', 'metadata': {'Type': 'work'}}}} Aug 28 01:31:04.318911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:31:04.318911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:31:04.318911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Aug 28 01:31:04.318911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Aug 28 01:31:04.318911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 268, in validate Aug 28 01:31:04.318911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise error Aug 28 01:31:04.318911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Aug 28 01:31:04.318911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.318911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Aug 28 01:31:04.318911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Aug 28 01:31:04.318911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.318911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi On instance['volume']: Aug 28 01:31:04.318911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi '' Aug 28 01:31:04.318911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.318911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:31:04.318911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.318911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:31:04.318911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:31:04.318911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:31:04.318911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:31:04.318911 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:31:04.320635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 28 01:31:04.320635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 28 01:31:04.320635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:31:04.320635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:31:04.320635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Aug 28 01:31:04.320635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Aug 28 01:31:04.320635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Aug 28 01:31:04.320635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Aug 28 01:31:04.320635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Aug 28 01:31:04.320635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Aug 28 01:31:04.320635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Aug 28 01:31:04.320635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.320635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:31:04.320635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.320635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:31:04.320635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:31:04.320635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:31:04.320635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Aug 28 01:31:04.320635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Aug 28 01:31:04.322201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Aug 28 01:31:04.322201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.322201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4066bf6a-79f8-48af-bf7d-6a679e19599e tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:31:04.322201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4066bf6a-79f8-48af-bf7d-6a679e19599e tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] 10.222.0.42 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.010502 Aug 28 01:31:04.322201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1530/3286] 10.222.0.42 () {60 vars in 1156 bytes} [Sun Aug 28 01:31:04 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 12 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Aug 28 01:31:04.328746 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2903d4d9-8bae-4bd1-a61f-315259e2930e tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] Calling method '.version_select at 0x7f0e71ffe5e0>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:31:04.359663 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-2903d4d9-8bae-4bd1-a61f-315259e2930e tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] HTTP exception thrown: Volume tempest-invalid-1589187037 could not be found. Aug 28 01:31:04.362698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2903d4d9-8bae-4bd1-a61f-315259e2930e tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] Request method failure captured: Aug 28 01:31:04.362698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-1589187037 HTTP/1.1 Aug 28 01:31:04.362698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:31:04.362698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:31:04.362698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:31:04.362698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:31:04.362698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:31:04.362698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:31:04.362698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:31:04.362698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:31:04.362698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCsVYKUfle16gH4n31Jsg8tJuvtIRKSj_kqW-XD5bIhbWZ9tVC8PkUl3WXoNh-a5kAji8U8xsYkAu4HXGbnrgeis5Q3rs66ZxGkttc2rVSsmLcMm1YhozNjmiOvHA8n7wvnav_KesgetGSlU3I5azZz_PJaqell1uwxDhJDFqwXVj9W8 Aug 28 01:31:04.362698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:31:04.362698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:31:04.362698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:31:04.362698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:31:04.362698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:31:04.362698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:31:04.362698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 765f789deef04206a3f89cfda8d1ee08 Aug 28 01:31:04.362698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-VolumesNegativeTest-1419196676 Aug 28 01:31:04.362698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:31:04.362698 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:31:04.364627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_765f789deef04206a3f89cfda8d1ee08", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/765f789deef04206a3f89cfda8d1ee08"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/765f789deef04206a3f89cfda8d1ee08"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/765f789deef04206a3f89cfda8d1ee08"}]}] Aug 28 01:31:04.364627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-VolumesNegativeTest-1419196676 Aug 28 01:31:04.364627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 765f789deef04206a3f89cfda8d1ee08 Aug 28 01:31:04.364627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-VolumesNegativeTest-1419196676 Aug 28 01:31:04.364627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-VolumesNegativeTest-1419196676-project Aug 28 01:31:04.364627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:31:04.364627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:31:04.364627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: eebc2779346f4fdbbf3bb89027638c24 Aug 28 01:31:04.364627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-VolumesNegativeTest-1419196676-project Aug 28 01:31:04.364627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: Aug 28 01:31:04.364627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Volume tempest-invalid-1589187037 could not be found. Aug 28 01:31:04.364627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'id': 'tempest-invalid-1589187037'} Aug 28 01:31:04.364627 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:31:04.365868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:31:04.365868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:31:04.365868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:31:04.365868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 28 01:31:04.365868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 28 01:31:04.365868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 28 01:31:04.365868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 28 01:31:04.365868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 28 01:31:04.365868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 28 01:31:04.365868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 28 01:31:04.365868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 28 01:31:04.365868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:31:04.365868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:31:04.365868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:31:04.365868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:31:04.365868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-1589187037 could not be found. (HTTP 404) (Request-ID: req-d922693a-cc7c-4393-932e-e7cbae8143ec) Aug 28 01:31:04.365868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.365868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:31:04.365868 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.367853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:31:04.367853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Aug 28 01:31:04.367853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Aug 28 01:31:04.367853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:31:04.367853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:31:04.367853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 28 01:31:04.367853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 28 01:31:04.367853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 28 01:31:04.367853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 28 01:31:04.367853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:31:04.367853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:31:04.367853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 28 01:31:04.367853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 28 01:31:04.367853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 28 01:31:04.367853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 28 01:31:04.367853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 28 01:31:04.367853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 28 01:31:04.367853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 28 01:31:04.367853 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 28 01:31:04.369918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:31:04.369918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:31:04.369918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:31:04.369918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:31:04.369918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-1589187037 could not be found. Aug 28 01:31:04.369918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.369918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:31:04.369918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.369918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:31:04.369918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:31:04.369918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:31:04.369918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:31:04.369918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:31:04.369918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 28 01:31:04.369918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 28 01:31:04.369918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:31:04.369918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:31:04.369918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Aug 28 01:31:04.369918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:31:04.369918 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-1589187037 could not be found. Aug 28 01:31:04.371598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.371598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:31:04.371598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.371598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:31:04.371598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:31:04.371598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:31:04.371598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:31:04.371598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:31:04.371598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-1589187037 could not be found. Aug 28 01:31:04.371598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.371598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-2903d4d9-8bae-4bd1-a61f-315259e2930e tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] Returning 404 to user: Volume tempest-invalid-1589187037 could not be found. {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:31:04.371598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-2903d4d9-8bae-4bd1-a61f-315259e2930e tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/tempest-invalid-1589187037" status: 404 len: 99 microversion: 2.1 time: 0.040603 Aug 28 01:31:04.371598 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1757/3287] 10.222.0.42 () {58 vars in 1218 bytes} [Sun Aug 28 01:31:04 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-1589187037 => generated 99 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Aug 28 01:31:04.374880 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-20f5f510-3d45-45d7-8e06-0b0e48f06ef6 tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004075 Aug 28 01:31:04.375375 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1531/3288] 10.222.0.42 () {58 vars in 1140 bytes} [Sun Aug 28 01:31:04 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Aug 28 01:31:04.384858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c9fb57ef-7be5-4659-ab0a-b02825033878 tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003883 Aug 28 01:31:04.385395 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1758/3289] 10.222.0.42 () {58 vars in 1137 bytes} [Sun Aug 28 01:31:04 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Aug 28 01:31:04.394846 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d507280f-340e-4629-b6ce-64447f1ee144 tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] Calling method '.version_select at 0x7f0e7006a280>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:31:04.425629 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-d507280f-340e-4629-b6ce-64447f1ee144 tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] HTTP exception thrown: Volume 66190a01-9f0a-4f83-abf4-8a61be4a2127 could not be found. Aug 28 01:31:04.427615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d507280f-340e-4629-b6ce-64447f1ee144 tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] Request method failure captured: Aug 28 01:31:04.427615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: DELETE /compute/v2.1/os-volumes/66190a01-9f0a-4f83-abf4-8a61be4a2127 HTTP/1.1 Aug 28 01:31:04.427615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:31:04.427615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:31:04.427615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:31:04.427615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:31:04.427615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:31:04.427615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:31:04.427615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:31:04.427615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:31:04.427615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCsVYKUfle16gH4n31Jsg8tJuvtIRKSj_kqW-XD5bIhbWZ9tVC8PkUl3WXoNh-a5kAji8U8xsYkAu4HXGbnrgeis5Q3rs66ZxGkttc2rVSsmLcMm1YhozNjmiOvHA8n7wvnav_KesgetGSlU3I5azZz_PJaqell1uwxDhJDFqwXVj9W8 Aug 28 01:31:04.427615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:31:04.427615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:31:04.427615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:31:04.427615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:31:04.427615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:31:04.427615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:31:04.427615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 765f789deef04206a3f89cfda8d1ee08 Aug 28 01:31:04.427615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-VolumesNegativeTest-1419196676 Aug 28 01:31:04.427615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:31:04.427615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:31:04.429442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_765f789deef04206a3f89cfda8d1ee08", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/765f789deef04206a3f89cfda8d1ee08"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/765f789deef04206a3f89cfda8d1ee08"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/765f789deef04206a3f89cfda8d1ee08"}]}] Aug 28 01:31:04.429442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-VolumesNegativeTest-1419196676 Aug 28 01:31:04.429442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 765f789deef04206a3f89cfda8d1ee08 Aug 28 01:31:04.429442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-VolumesNegativeTest-1419196676 Aug 28 01:31:04.429442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-VolumesNegativeTest-1419196676-project Aug 28 01:31:04.429442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:31:04.429442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:31:04.429442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: eebc2779346f4fdbbf3bb89027638c24 Aug 28 01:31:04.429442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-VolumesNegativeTest-1419196676-project Aug 28 01:31:04.429442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: Aug 28 01:31:04.429442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Volume 66190a01-9f0a-4f83-abf4-8a61be4a2127 could not be found. Aug 28 01:31:04.429442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'id': '66190a01-9f0a-4f83-abf4-8a61be4a2127'} Aug 28 01:31:04.429442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:31:04.430616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:31:04.430616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:31:04.430616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:31:04.430616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 28 01:31:04.430616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 28 01:31:04.430616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 28 01:31:04.430616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 28 01:31:04.430616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 28 01:31:04.430616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 28 01:31:04.430616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 28 01:31:04.430616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 28 01:31:04.430616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:31:04.430616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:31:04.430616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:31:04.430616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:31:04.430616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 66190a01-9f0a-4f83-abf4-8a61be4a2127 could not be found. (HTTP 404) (Request-ID: req-adc2902c-1896-4bc9-ae4d-40b229ea9734) Aug 28 01:31:04.430616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.430616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:31:04.430616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.431974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:31:04.431974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Aug 28 01:31:04.431974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Aug 28 01:31:04.431974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:31:04.431974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:31:04.431974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 28 01:31:04.431974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 28 01:31:04.431974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 28 01:31:04.431974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 28 01:31:04.431974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:31:04.431974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:31:04.431974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Aug 28 01:31:04.431974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Aug 28 01:31:04.431974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Aug 28 01:31:04.431974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._delete(loc) Aug 28 01:31:04.431974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Aug 28 01:31:04.431974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Aug 28 01:31:04.431974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Aug 28 01:31:04.431974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Aug 28 01:31:04.433451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:31:04.433451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:31:04.433451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:31:04.433451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:31:04.433451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 66190a01-9f0a-4f83-abf4-8a61be4a2127 could not be found. Aug 28 01:31:04.433451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.433451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:31:04.433451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.433451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:31:04.433451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:31:04.433451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:31:04.433451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:31:04.433451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:31:04.433451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 28 01:31:04.433451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 28 01:31:04.433451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:31:04.433451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:31:04.433451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Aug 28 01:31:04.433451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:31:04.435143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 66190a01-9f0a-4f83-abf4-8a61be4a2127 could not be found. Aug 28 01:31:04.435143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.435143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:31:04.435143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.435143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:31:04.435143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:31:04.435143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:31:04.435143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:31:04.435143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:31:04.435143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 66190a01-9f0a-4f83-abf4-8a61be4a2127 could not be found. Aug 28 01:31:04.435143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.435143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d507280f-340e-4629-b6ce-64447f1ee144 tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] Returning 404 to user: Volume 66190a01-9f0a-4f83-abf4-8a61be4a2127 could not be found. {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:31:04.435143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d507280f-340e-4629-b6ce-64447f1ee144 tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/66190a01-9f0a-4f83-abf4-8a61be4a2127" status: 404 len: 109 microversion: 2.1 time: 0.038554 Aug 28 01:31:04.435143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1532/3290] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Aug 28 01:31:04 2022] DELETE /compute/v2.1/os-volumes/66190a01-9f0a-4f83-abf4-8a61be4a2127 => generated 109 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Aug 28 01:31:04.440663 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-370a91fd-ed3f-4445-b52e-7e979db6cd86 tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] Calling method '.version_select at 0x7f0e6ea60ca0>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:31:04.476214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-370a91fd-ed3f-4445-b52e-7e979db6cd86 tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] HTTP exception thrown: Volume 6689d8f2-78ff-4424-a876-1ba8c8a00ad3 could not be found. Aug 28 01:31:04.478803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-370a91fd-ed3f-4445-b52e-7e979db6cd86 tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] Request method failure captured: Aug 28 01:31:04.478803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: GET /compute/v2.1/os-volumes/6689d8f2-78ff-4424-a876-1ba8c8a00ad3 HTTP/1.1 Aug 28 01:31:04.478803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:31:04.478803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:31:04.478803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:31:04.478803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:31:04.478803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:31:04.478803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:31:04.478803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:31:04.478803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:31:04.478803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCsVYKUfle16gH4n31Jsg8tJuvtIRKSj_kqW-XD5bIhbWZ9tVC8PkUl3WXoNh-a5kAji8U8xsYkAu4HXGbnrgeis5Q3rs66ZxGkttc2rVSsmLcMm1YhozNjmiOvHA8n7wvnav_KesgetGSlU3I5azZz_PJaqell1uwxDhJDFqwXVj9W8 Aug 28 01:31:04.478803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:31:04.478803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:31:04.478803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:31:04.478803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:31:04.478803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:31:04.478803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:31:04.478803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 765f789deef04206a3f89cfda8d1ee08 Aug 28 01:31:04.478803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-VolumesNegativeTest-1419196676 Aug 28 01:31:04.478803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:31:04.478803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:31:04.480751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_765f789deef04206a3f89cfda8d1ee08", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/765f789deef04206a3f89cfda8d1ee08"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/765f789deef04206a3f89cfda8d1ee08"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/765f789deef04206a3f89cfda8d1ee08"}]}] Aug 28 01:31:04.480751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-VolumesNegativeTest-1419196676 Aug 28 01:31:04.480751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 765f789deef04206a3f89cfda8d1ee08 Aug 28 01:31:04.480751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-VolumesNegativeTest-1419196676 Aug 28 01:31:04.480751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-VolumesNegativeTest-1419196676-project Aug 28 01:31:04.480751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:31:04.480751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:31:04.480751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: eebc2779346f4fdbbf3bb89027638c24 Aug 28 01:31:04.480751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-VolumesNegativeTest-1419196676-project Aug 28 01:31:04.480751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: Aug 28 01:31:04.480751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Volume 6689d8f2-78ff-4424-a876-1ba8c8a00ad3 could not be found. Aug 28 01:31:04.480751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'id': '6689d8f2-78ff-4424-a876-1ba8c8a00ad3'} Aug 28 01:31:04.480751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:31:04.481994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:31:04.481994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:31:04.481994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:31:04.481994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 28 01:31:04.481994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 28 01:31:04.481994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 28 01:31:04.481994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 28 01:31:04.481994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 28 01:31:04.481994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 28 01:31:04.481994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 28 01:31:04.481994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 28 01:31:04.481994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:31:04.481994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:31:04.481994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:31:04.481994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:31:04.481994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 6689d8f2-78ff-4424-a876-1ba8c8a00ad3 could not be found. (HTTP 404) (Request-ID: req-6c4c60f9-6c56-4128-b5c1-90a89d239372) Aug 28 01:31:04.481994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.481994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:31:04.481994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.483675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:31:04.483675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 28 01:31:04.483675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 28 01:31:04.483675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:31:04.483675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:31:04.483675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 28 01:31:04.483675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 28 01:31:04.483675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 28 01:31:04.483675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 28 01:31:04.483675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:31:04.483675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:31:04.483675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 28 01:31:04.483675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 28 01:31:04.483675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 28 01:31:04.483675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 28 01:31:04.483675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 28 01:31:04.483675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 28 01:31:04.483675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 28 01:31:04.483675 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 28 01:31:04.485478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:31:04.485478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:31:04.485478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:31:04.485478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:31:04.485478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 6689d8f2-78ff-4424-a876-1ba8c8a00ad3 could not be found. Aug 28 01:31:04.485478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.485478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:31:04.485478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.485478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:31:04.485478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:31:04.485478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:31:04.485478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:31:04.485478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:31:04.485478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 28 01:31:04.485478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 28 01:31:04.485478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:31:04.485478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:31:04.485478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 28 01:31:04.485478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:31:04.487105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 6689d8f2-78ff-4424-a876-1ba8c8a00ad3 could not be found. Aug 28 01:31:04.487105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.487105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:31:04.487105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.487105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:31:04.487105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:31:04.487105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:31:04.487105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:31:04.487105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:31:04.487105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 6689d8f2-78ff-4424-a876-1ba8c8a00ad3 could not be found. Aug 28 01:31:04.487105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:31:04.487105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-370a91fd-ed3f-4445-b52e-7e979db6cd86 tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] Returning 404 to user: Volume 6689d8f2-78ff-4424-a876-1ba8c8a00ad3 could not be found. {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:31:04.487105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-370a91fd-ed3f-4445-b52e-7e979db6cd86 tempest-VolumesNegativeTest-1419196676 tempest-VolumesNegativeTest-1419196676-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/6689d8f2-78ff-4424-a876-1ba8c8a00ad3" status: 404 len: 109 microversion: 2.1 time: 0.043718 Aug 28 01:31:04.487105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1759/3291] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Aug 28 01:31:04 2022] GET /compute/v2.1/os-volumes/6689d8f2-78ff-4424-a876-1ba8c8a00ad3 => generated 109 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:35:18.980816 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR oslo.messaging._drivers.impl_rabbit [-] [a75834a5-0c81-44d8-993d-e8453085eab8] AMQP server on 10.222.0.42:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Aug 28 01:35:18.981390 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=10 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:35:18.981937 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=18 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:35:18.983074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 28 01:35:18.983223 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Sun Aug 28 01:35:18 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c (ip 10.222.0.42) !!! Aug 28 01:35:18.984412 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 28 01:35:18.984545 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Sun Aug 28 01:35:18 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c (ip 10.222.0.42) !!! Aug 28 01:35:18.985540 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 28 01:35:18.985642 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Sun Aug 28 01:35:18 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c (ip 10.222.0.42) !!! Aug 28 01:35:18.991478 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e0526fa5-5a53-4f09-8d78-4599a813e0ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:35:18.991975 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e0526fa5-5a53-4f09-8d78-4599a813e0ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:35:19.002157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e0526fa5-5a53-4f09-8d78-4599a813e0ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:35:19.002632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e0526fa5-5a53-4f09-8d78-4599a813e0ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:35:19.003065 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e0526fa5-5a53-4f09-8d78-4599a813e0ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:35:19.003752 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e0526fa5-5a53-4f09-8d78-4599a813e0ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:35:19.004201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e0526fa5-5a53-4f09-8d78-4599a813e0ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:35:19.004616 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e0526fa5-5a53-4f09-8d78-4599a813e0ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:35:19.018822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-e0526fa5-5a53-4f09-8d78-4599a813e0ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] HTTP exception thrown: Instance dff36aeb-f0c7-41f0-8ecc-9924adf8289c could not be found. Aug 28 01:35:19.020372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e0526fa5-5a53-4f09-8d78-4599a813e0ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Request method failure captured: Aug 28 01:35:19.020372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: DELETE /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c HTTP/1.1 Aug 28 01:35:19.020372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:35:19.020372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:35:19.020372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:35:19.020372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:35:19.020372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:35:19.020372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:35:19.020372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:35:19.020372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:35:19.020372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCrzduxHoOEhvI8P-Mde9cL1oxkmc4fMm9FmGsHxLSNCBjncK-ctZ1RF74JsMq7QMGP1Qh6DECNOIg5DjWuRWqO-CpXkeFE6YBKwQT8SoQT23SwG4DtFzbguCYZajUyKFut9Dn943LOjL5V3-apkT_NK3VlDjeI9Rm1KYQSsNW0BOiQg Aug 28 01:35:19.020372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:35:19.020372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:35:19.020372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:35:19.020372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:35:19.020372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:35:19.020372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:35:19.020372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:35:19.020372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:35:19.020372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:35:19.020372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:35:19.021578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_95df81e14a8549fca0851234f4a00b89", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/95df81e14a8549fca0851234f4a00b89"}]}] Aug 28 01:35:19.021578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:35:19.021578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:35:19.021578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:35:19.021578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:35:19.021578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:35:19.021578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:35:19.021578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: d209174d0db04b3584556824c6d0a3d4 Aug 28 01:35:19.021578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:35:19.021578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:35:19.021578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Instance dff36aeb-f0c7-41f0-8ecc-9924adf8289c could not be found. Aug 28 01:35:19.021578 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'id': 'dff36aeb-f0c7-41f0-8ecc-9924adf8289c'} Aug 28 01:35:19.022243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:35:19.022243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:35:19.022243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:35:19.022243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:35:19.022243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:35:19.022243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:35:19.022243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:35:19.022243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:35:19.022243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:35:19.022243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:35:19.022243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dff36aeb-f0c7-41f0-8ecc-9924adf8289c could not be found. Aug 28 01:35:19.022243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:35:19.022243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:35:19.022243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:35:19.022243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:35:19.022243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:35:19.022243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:35:19.022243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:35:19.022243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:35:19.022243 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:35:19.023320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:35:19.023320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 28 01:35:19.023320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 28 01:35:19.023320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 28 01:35:19.023320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 28 01:35:19.023320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:35:19.023320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:35:19.023320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:35:19.023320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:35:19.023320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dff36aeb-f0c7-41f0-8ecc-9924adf8289c could not be found. Aug 28 01:35:19.023320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:35:19.023320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:35:19.023320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:35:19.023320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:35:19.023320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:35:19.023320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:35:19.023320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:35:19.023320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:35:19.023320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dff36aeb-f0c7-41f0-8ecc-9924adf8289c could not be found. Aug 28 01:35:19.023320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:35:19.024449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e0526fa5-5a53-4f09-8d78-4599a813e0ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Returning 404 to user: Instance dff36aeb-f0c7-41f0-8ecc-9924adf8289c could not be found. {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:35:19.024449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e0526fa5-5a53-4f09-8d78-4599a813e0ea tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "DELETE /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 404 len: 111 microversion: 2.1 time: 0.045104 Aug 28 01:35:19.024449 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1533/3292] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 01:35:18 2022] DELETE /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:35:19.028504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR oslo.messaging._drivers.impl_rabbit [-] [c920d2e8-ed1b-49b0-a050-78ee4a07b0a8] AMQP server on 10.222.0.42:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Aug 28 01:35:19.028998 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=10 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:35:19.029699 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=29,nova_cell1:UPDATE=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:35:19.031030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 28 01:35:19.031134 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Sun Aug 28 01:35:19 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c (ip 10.222.0.42) !!! Aug 28 01:35:19.033139 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 28 01:35:19.033248 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Sun Aug 28 01:35:19 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c (ip 10.222.0.42) !!! Aug 28 01:35:19.034704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 28 01:35:19.034820 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Sun Aug 28 01:35:19 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c (ip 10.222.0.42) !!! Aug 28 01:35:19.049429 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-67669ce1-9bf4-4873-be0d-6fcaee62a2a4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:35:19.050159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-67669ce1-9bf4-4873-be0d-6fcaee62a2a4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] [instance: dff36aeb-f0c7-41f0-8ecc-9924adf8289c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:35:19.060809 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67669ce1-9bf4-4873-be0d-6fcaee62a2a4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:35:19.061304 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67669ce1-9bf4-4873-be0d-6fcaee62a2a4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:35:19.061738 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67669ce1-9bf4-4873-be0d-6fcaee62a2a4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:35:19.062433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67669ce1-9bf4-4873-be0d-6fcaee62a2a4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:35:19.062880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67669ce1-9bf4-4873-be0d-6fcaee62a2a4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:35:19.063352 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-67669ce1-9bf4-4873-be0d-6fcaee62a2a4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:35:19.077912 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-67669ce1-9bf4-4873-be0d-6fcaee62a2a4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] HTTP exception thrown: Instance dff36aeb-f0c7-41f0-8ecc-9924adf8289c could not be found. Aug 28 01:35:19.079462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-67669ce1-9bf4-4873-be0d-6fcaee62a2a4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Request method failure captured: Aug 28 01:35:19.079462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c HTTP/1.1 Aug 28 01:35:19.079462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:35:19.079462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:35:19.079462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:35:19.079462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:35:19.079462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:35:19.079462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:35:19.079462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:35:19.079462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:35:19.079462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCrzduxHoOEhvI8P-Mde9cL1oxkmc4fMm9FmGsHxLSNCBjncK-ctZ1RF74JsMq7QMGP1Qh6DECNOIg5DjWuRWqO-CpXkeFE6YBKwQT8SoQT23SwG4DtFzbguCYZajUyKFut9Dn943LOjL5V3-apkT_NK3VlDjeI9Rm1KYQSsNW0BOiQg Aug 28 01:35:19.079462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:35:19.079462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:35:19.079462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:35:19.079462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:35:19.079462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:35:19.079462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:35:19.079462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:35:19.079462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:35:19.079462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:35:19.079462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:35:19.081219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_95df81e14a8549fca0851234f4a00b89", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/95df81e14a8549fca0851234f4a00b89"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/95df81e14a8549fca0851234f4a00b89"}]}] Aug 28 01:35:19.081219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:35:19.081219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 95df81e14a8549fca0851234f4a00b89 Aug 28 01:35:19.081219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1105821037 Aug 28 01:35:19.081219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:35:19.081219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:35:19.081219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:35:19.081219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: d209174d0db04b3584556824c6d0a3d4 Aug 28 01:35:19.081219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-AttachVolumeNegativeTest-1105821037-project Aug 28 01:35:19.081219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 01:35:19.081219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Instance dff36aeb-f0c7-41f0-8ecc-9924adf8289c could not be found. Aug 28 01:35:19.081219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'id': 'dff36aeb-f0c7-41f0-8ecc-9924adf8289c'} Aug 28 01:35:19.081950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:35:19.081950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:35:19.081950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:35:19.081950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:35:19.081950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:35:19.081950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:35:19.081950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:35:19.081950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:35:19.081950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:35:19.081950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:35:19.081950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dff36aeb-f0c7-41f0-8ecc-9924adf8289c could not be found. Aug 28 01:35:19.081950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:35:19.081950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:35:19.081950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:35:19.081950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:35:19.081950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:35:19.081950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:35:19.081950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:35:19.081950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:35:19.081950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:35:19.083692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:35:19.083692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 01:35:19.083692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 01:35:19.083692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:35:19.083692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:35:19.083692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:35:19.083692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:35:19.083692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dff36aeb-f0c7-41f0-8ecc-9924adf8289c could not be found. Aug 28 01:35:19.083692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:35:19.083692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:35:19.083692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:35:19.083692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:35:19.083692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:35:19.083692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:35:19.083692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:35:19.083692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:35:19.083692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dff36aeb-f0c7-41f0-8ecc-9924adf8289c could not be found. Aug 28 01:35:19.083692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:35:19.084646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-67669ce1-9bf4-4873-be0d-6fcaee62a2a4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] Returning 404 to user: Instance dff36aeb-f0c7-41f0-8ecc-9924adf8289c could not be found. {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:35:19.084646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-67669ce1-9bf4-4873-be0d-6fcaee62a2a4 tempest-AttachVolumeNegativeTest-1105821037 tempest-AttachVolumeNegativeTest-1105821037-project] 10.222.0.42 "GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c" status: 404 len: 111 microversion: 2.1 time: 0.054230 Aug 28 01:35:19.084646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1760/3293] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:35:19 2022] GET /compute/v2.1/servers/dff36aeb-f0c7-41f0-8ecc-9924adf8289c => generated 111 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:35:56.635256 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8c184607-87ec-47ac-80d7-f50c75c4c7e0 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-723509968", "imageRef": "2c2585b2-bde5-4652-aed7-9f53b1bd5d94", "flavorRef": "452"}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:35:56.644332 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=2 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:35:56.645674 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=2 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:35:56.664501 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO oslo.messaging._drivers.impl_rabbit [-] [a75834a5-0c81-44d8-993d-e8453085eab8] Reconnected to AMQP server on 10.222.0.42:5672 via [amqp] client with port 50922. Aug 28 01:35:56.915718 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.network.neutron [None req-8c184607-87ec-47ac-80d7-f50c75c4c7e0 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] validate_networks() for None {{(pid=120524) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 28 01:35:57.224894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-8c184607-87ec-47ac-80d7-f50c75c4c7e0 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Flavor limits 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 28 01:35:57.225350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-8c184607-87ec-47ac-80d7-f50c75c4c7e0 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Image limits 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 28 01:35:57.226057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-8c184607-87ec-47ac-80d7-f50c75c4c7e0 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Flavor pref 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 28 01:35:57.226462 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-8c184607-87ec-47ac-80d7-f50c75c4c7e0 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Image pref 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 28 01:35:57.226892 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-8c184607-87ec-47ac-80d7-f50c75c4c7e0 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 28 01:35:57.236446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8c184607-87ec-47ac-80d7-f50c75c4c7e0 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:35:57.236926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8c184607-87ec-47ac-80d7-f50c75c4c7e0 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:35:57.237394 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8c184607-87ec-47ac-80d7-f50c75c4c7e0 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:35:57.238146 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8c184607-87ec-47ac-80d7-f50c75c4c7e0 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:35:57.238590 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8c184607-87ec-47ac-80d7-f50c75c4c7e0 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:35:57.239003 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8c184607-87ec-47ac-80d7-f50c75c4c7e0 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:35:57.257195 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-8c184607-87ec-47ac-80d7-f50c75c4c7e0 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Getting quotas for project 802f856adff04f85b6731d6a97331da6. Resources: {'ram', 'instances', 'cores'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 01:35:57.261813 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-8c184607-87ec-47ac-80d7-f50c75c4c7e0 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Getting quotas for user f757f2efd63f4526892784e78a402ccd and project 802f856adff04f85b6731d6a97331da6. Resources: {'ram', 'instances', 'cores'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 01:35:57.274632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8c184607-87ec-47ac-80d7-f50c75c4c7e0 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Going to run 1 instances... {{(pid=120524) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 28 01:35:57.276934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8c184607-87ec-47ac-80d7-f50c75c4c7e0 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] [instance: 804c9d12-0553-40b5-815d-980078f32f2f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2c2585b2-bde5-4652-aed7-9f53b1bd5d94',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120524) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 28 01:35:57.325173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8c184607-87ec-47ac-80d7-f50c75c4c7e0 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.693934 Aug 28 01:35:57.325812 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1534/3294] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Aug 28 01:35:56 2022] POST /compute/v2.1/servers => generated 380 bytes in 695 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 28 01:35:57.336907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-11489c2a-5ed8-4873-8640-6c80e88d8d5f tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:35:57.337857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-11489c2a-5ed8-4873-8640-6c80e88d8d5f tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] [instance: 804c9d12-0553-40b5-815d-980078f32f2f] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:35:57.341902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=2 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:35:57.343016 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=2 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:35:57.355443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO oslo.messaging._drivers.impl_rabbit [-] [c920d2e8-ed1b-49b0-a050-78ee4a07b0a8] Reconnected to AMQP server on 10.222.0.42:5672 via [amqp] client with port 50944. Aug 28 01:35:57.515036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-11489c2a-5ed8-4873-8640-6c80e88d8d5f tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f757f2efd63f4526892784e78a402ccd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '802f856adff04f85b6731d6a97331da6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:35:57.523740 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-11489c2a-5ed8-4873-8640-6c80e88d8d5f tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] 10.222.0.42 "GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f" status: 200 len: 1214 microversion: 2.1 time: 0.190705 Aug 28 01:35:57.524366 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1761/3295] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:35:57 2022] GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f => generated 1214 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:35:58.545049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b630dcf9-fd68-410f-908b-787fc9677a39 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:35:58.545624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b630dcf9-fd68-410f-908b-787fc9677a39 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] [instance: 804c9d12-0553-40b5-815d-980078f32f2f] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:35:58.552642 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b630dcf9-fd68-410f-908b-787fc9677a39 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:35:58.553239 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b630dcf9-fd68-410f-908b-787fc9677a39 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:35:58.553669 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b630dcf9-fd68-410f-908b-787fc9677a39 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:35:58.554711 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b630dcf9-fd68-410f-908b-787fc9677a39 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:35:58.554794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b630dcf9-fd68-410f-908b-787fc9677a39 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:35:58.555293 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b630dcf9-fd68-410f-908b-787fc9677a39 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:35:58.625305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b630dcf9-fd68-410f-908b-787fc9677a39 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f757f2efd63f4526892784e78a402ccd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '802f856adff04f85b6731d6a97331da6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:35:58.633804 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b630dcf9-fd68-410f-908b-787fc9677a39 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] 10.222.0.42 "GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f" status: 200 len: 1266 microversion: 2.1 time: 0.092804 Aug 28 01:35:58.634496 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1535/3296] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:35:58 2022] GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f => generated 1266 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:35:59.656609 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ff4b7e1b-4060-4edc-9b7d-9a0ff0e9420e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:35:59.658309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ff4b7e1b-4060-4edc-9b7d-9a0ff0e9420e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] [instance: 804c9d12-0553-40b5-815d-980078f32f2f] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:35:59.666733 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ff4b7e1b-4060-4edc-9b7d-9a0ff0e9420e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:35:59.667231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ff4b7e1b-4060-4edc-9b7d-9a0ff0e9420e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:35:59.667696 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ff4b7e1b-4060-4edc-9b7d-9a0ff0e9420e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:35:59.668407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ff4b7e1b-4060-4edc-9b7d-9a0ff0e9420e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:35:59.668857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ff4b7e1b-4060-4edc-9b7d-9a0ff0e9420e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:35:59.669322 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ff4b7e1b-4060-4edc-9b7d-9a0ff0e9420e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:35:59.906527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ff4b7e1b-4060-4edc-9b7d-9a0ff0e9420e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f757f2efd63f4526892784e78a402ccd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '802f856adff04f85b6731d6a97331da6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:35:59.914708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ff4b7e1b-4060-4edc-9b7d-9a0ff0e9420e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] 10.222.0.42 "GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f" status: 200 len: 1314 microversion: 2.1 time: 0.262971 Aug 28 01:35:59.915359 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1762/3297] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:35:59 2022] GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f => generated 1314 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:00.302864 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-140312bb-848e-48fd-bf0a-b663abcd674b req-04c84957-1ea2-47ea-a715-0345973d27e7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "804c9d12-0553-40b5-815d-980078f32f2f", "tag": "d5838c45-b309-4df6-ba0a-4f2936bc2905"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:36:00.313884 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-140312bb-848e-48fd-bf0a-b663abcd674b req-04c84957-1ea2-47ea-a715-0345973d27e7 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:00.314445 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-140312bb-848e-48fd-bf0a-b663abcd674b req-04c84957-1ea2-47ea-a715-0345973d27e7 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:00.314778 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-140312bb-848e-48fd-bf0a-b663abcd674b req-04c84957-1ea2-47ea-a715-0345973d27e7 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:00.330066 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-140312bb-848e-48fd-bf0a-b663abcd674b req-04c84957-1ea2-47ea-a715-0345973d27e7 service nova] Creating event network-changed:d5838c45-b309-4df6-ba0a-4f2936bc2905 for instance 804c9d12-0553-40b5-815d-980078f32f2f on hv-cinder-maste Aug 28 01:36:00.336537 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-140312bb-848e-48fd-bf0a-b663abcd674b req-04c84957-1ea2-47ea-a715-0345973d27e7 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.122804 Aug 28 01:36:00.337173 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1536/3298] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:36:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:36:00.937038 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-28203c38-6d1c-469e-9cc8-7b4fdaf61462 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:00.937633 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-28203c38-6d1c-469e-9cc8-7b4fdaf61462 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] [instance: 804c9d12-0553-40b5-815d-980078f32f2f] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:00.944341 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-28203c38-6d1c-469e-9cc8-7b4fdaf61462 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:00.944829 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-28203c38-6d1c-469e-9cc8-7b4fdaf61462 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:00.945298 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-28203c38-6d1c-469e-9cc8-7b4fdaf61462 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:00.945993 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-28203c38-6d1c-469e-9cc8-7b4fdaf61462 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:00.946439 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-28203c38-6d1c-469e-9cc8-7b4fdaf61462 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:00.946870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-28203c38-6d1c-469e-9cc8-7b4fdaf61462 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:01.090276 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-28203c38-6d1c-469e-9cc8-7b4fdaf61462 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f757f2efd63f4526892784e78a402ccd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '802f856adff04f85b6731d6a97331da6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:01.098246 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-28203c38-6d1c-469e-9cc8-7b4fdaf61462 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] 10.222.0.42 "GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f" status: 200 len: 1314 microversion: 2.1 time: 0.165143 Aug 28 01:36:01.098819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1763/3299] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:00 2022] GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f => generated 1314 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:02.120652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c9cbabd9-0175-4608-978b-ed593333db8f tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:02.122059 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-c9cbabd9-0175-4608-978b-ed593333db8f tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] [instance: 804c9d12-0553-40b5-815d-980078f32f2f] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:02.128529 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9cbabd9-0175-4608-978b-ed593333db8f tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:02.129036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9cbabd9-0175-4608-978b-ed593333db8f tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:02.129484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9cbabd9-0175-4608-978b-ed593333db8f tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:02.130199 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9cbabd9-0175-4608-978b-ed593333db8f tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:02.130650 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9cbabd9-0175-4608-978b-ed593333db8f tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:02.131066 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-c9cbabd9-0175-4608-978b-ed593333db8f tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:02.275539 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-c9cbabd9-0175-4608-978b-ed593333db8f tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f757f2efd63f4526892784e78a402ccd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '802f856adff04f85b6731d6a97331da6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:02.284017 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c9cbabd9-0175-4608-978b-ed593333db8f tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] 10.222.0.42 "GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f" status: 200 len: 1314 microversion: 2.1 time: 0.168473 Aug 28 01:36:02.284592 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1537/3300] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:02 2022] GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f => generated 1314 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:03.306268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6cb08b24-fa69-4652-9058-0aff89618702 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:03.307355 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6cb08b24-fa69-4652-9058-0aff89618702 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] [instance: 804c9d12-0553-40b5-815d-980078f32f2f] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:03.320170 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6cb08b24-fa69-4652-9058-0aff89618702 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:03.321395 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6cb08b24-fa69-4652-9058-0aff89618702 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:03.321840 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6cb08b24-fa69-4652-9058-0aff89618702 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:03.323281 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6cb08b24-fa69-4652-9058-0aff89618702 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:03.323749 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6cb08b24-fa69-4652-9058-0aff89618702 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:03.332337 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6cb08b24-fa69-4652-9058-0aff89618702 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:03.526376 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6cb08b24-fa69-4652-9058-0aff89618702 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f757f2efd63f4526892784e78a402ccd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '802f856adff04f85b6731d6a97331da6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:03.537335 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6cb08b24-fa69-4652-9058-0aff89618702 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] 10.222.0.42 "GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f" status: 200 len: 1314 microversion: 2.1 time: 0.235413 Aug 28 01:36:03.538013 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1764/3301] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:03 2022] GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f => generated 1314 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:04.559499 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7e38ac21-995d-472d-a07e-5b66a148628e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:04.560121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7e38ac21-995d-472d-a07e-5b66a148628e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] [instance: 804c9d12-0553-40b5-815d-980078f32f2f] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:04.566850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e38ac21-995d-472d-a07e-5b66a148628e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:04.567328 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e38ac21-995d-472d-a07e-5b66a148628e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:04.567763 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e38ac21-995d-472d-a07e-5b66a148628e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:04.568482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e38ac21-995d-472d-a07e-5b66a148628e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:04.568926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e38ac21-995d-472d-a07e-5b66a148628e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:04.569360 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e38ac21-995d-472d-a07e-5b66a148628e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:04.707331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-7e38ac21-995d-472d-a07e-5b66a148628e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f757f2efd63f4526892784e78a402ccd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '802f856adff04f85b6731d6a97331da6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:04.715461 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7e38ac21-995d-472d-a07e-5b66a148628e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] 10.222.0.42 "GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f" status: 200 len: 1314 microversion: 2.1 time: 0.160380 Aug 28 01:36:04.716062 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1538/3302] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:04 2022] GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f => generated 1314 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:05.737990 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-46505cdc-adfa-43e8-a252-79314f83ca0a tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:05.738772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-46505cdc-adfa-43e8-a252-79314f83ca0a tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] [instance: 804c9d12-0553-40b5-815d-980078f32f2f] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:05.745878 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46505cdc-adfa-43e8-a252-79314f83ca0a tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:05.746399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46505cdc-adfa-43e8-a252-79314f83ca0a tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:05.746810 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46505cdc-adfa-43e8-a252-79314f83ca0a tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:05.747524 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46505cdc-adfa-43e8-a252-79314f83ca0a tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:05.748111 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46505cdc-adfa-43e8-a252-79314f83ca0a tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:05.748398 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-46505cdc-adfa-43e8-a252-79314f83ca0a tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:05.884264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-46505cdc-adfa-43e8-a252-79314f83ca0a tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f757f2efd63f4526892784e78a402ccd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '802f856adff04f85b6731d6a97331da6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:05.892354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-46505cdc-adfa-43e8-a252-79314f83ca0a tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] 10.222.0.42 "GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f" status: 200 len: 1314 microversion: 2.1 time: 0.158364 Aug 28 01:36:05.892933 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1765/3303] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:05 2022] GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f => generated 1314 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:06.914165 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0a05450b-d2f2-4ae8-9108-68bb43f47fde tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:06.914745 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0a05450b-d2f2-4ae8-9108-68bb43f47fde tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] [instance: 804c9d12-0553-40b5-815d-980078f32f2f] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:06.922210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0a05450b-d2f2-4ae8-9108-68bb43f47fde tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:06.922706 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0a05450b-d2f2-4ae8-9108-68bb43f47fde tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:06.923135 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0a05450b-d2f2-4ae8-9108-68bb43f47fde tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:06.923856 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0a05450b-d2f2-4ae8-9108-68bb43f47fde tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:06.924281 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0a05450b-d2f2-4ae8-9108-68bb43f47fde tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:06.924701 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0a05450b-d2f2-4ae8-9108-68bb43f47fde tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:07.065993 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0a05450b-d2f2-4ae8-9108-68bb43f47fde tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f757f2efd63f4526892784e78a402ccd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '802f856adff04f85b6731d6a97331da6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:07.074599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0a05450b-d2f2-4ae8-9108-68bb43f47fde tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] 10.222.0.42 "GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f" status: 200 len: 1314 microversion: 2.1 time: 0.164211 Aug 28 01:36:07.075201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1539/3304] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:06 2022] GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f => generated 1314 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:07.809919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=2 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:36:07.881911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-ee08cf39-1f2b-4673-bfd8-d089102098d6 req-014a27cf-7e36-4fe3-8e30-287373acea9e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "804c9d12-0553-40b5-815d-980078f32f2f", "name": "network-vif-plugged", "status": "completed", "tag": "d5838c45-b309-4df6-ba0a-4f2936bc2905"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:36:07.892593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-ee08cf39-1f2b-4673-bfd8-d089102098d6 req-014a27cf-7e36-4fe3-8e30-287373acea9e service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:07.893171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-ee08cf39-1f2b-4673-bfd8-d089102098d6 req-014a27cf-7e36-4fe3-8e30-287373acea9e service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:07.893691 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-ee08cf39-1f2b-4673-bfd8-d089102098d6 req-014a27cf-7e36-4fe3-8e30-287373acea9e service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:07.906732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-ee08cf39-1f2b-4673-bfd8-d089102098d6 req-014a27cf-7e36-4fe3-8e30-287373acea9e service nova] Creating event network-vif-plugged:d5838c45-b309-4df6-ba0a-4f2936bc2905 for instance 804c9d12-0553-40b5-815d-980078f32f2f on hv-cinder-maste Aug 28 01:36:07.913282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-ee08cf39-1f2b-4673-bfd8-d089102098d6 req-014a27cf-7e36-4fe3-8e30-287373acea9e service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.109156 Aug 28 01:36:07.913874 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1766/3305] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:36:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:36:08.097107 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1c87f4ce-110c-4cac-9f6f-98ae2f38a95d tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:08.097728 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1c87f4ce-110c-4cac-9f6f-98ae2f38a95d tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] [instance: 804c9d12-0553-40b5-815d-980078f32f2f] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:08.099904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=2 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:36:08.100638 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=6 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:36:08.108106 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c87f4ce-110c-4cac-9f6f-98ae2f38a95d tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:08.108593 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c87f4ce-110c-4cac-9f6f-98ae2f38a95d tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:08.109057 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c87f4ce-110c-4cac-9f6f-98ae2f38a95d tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:08.109766 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c87f4ce-110c-4cac-9f6f-98ae2f38a95d tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:08.110206 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c87f4ce-110c-4cac-9f6f-98ae2f38a95d tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:08.110627 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c87f4ce-110c-4cac-9f6f-98ae2f38a95d tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:08.283269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1c87f4ce-110c-4cac-9f6f-98ae2f38a95d tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f757f2efd63f4526892784e78a402ccd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '802f856adff04f85b6731d6a97331da6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:08.291974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1c87f4ce-110c-4cac-9f6f-98ae2f38a95d tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] 10.222.0.42 "GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f" status: 200 len: 1314 microversion: 2.1 time: 0.199705 Aug 28 01:36:08.292553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1540/3306] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:08 2022] GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f => generated 1314 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:09.314551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ffd1e520-f606-4179-ad21-88f6fd8ca383 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:09.315269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ffd1e520-f606-4179-ad21-88f6fd8ca383 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] [instance: 804c9d12-0553-40b5-815d-980078f32f2f] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:09.322634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ffd1e520-f606-4179-ad21-88f6fd8ca383 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:09.323130 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ffd1e520-f606-4179-ad21-88f6fd8ca383 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:09.323565 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ffd1e520-f606-4179-ad21-88f6fd8ca383 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:09.324267 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ffd1e520-f606-4179-ad21-88f6fd8ca383 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:09.324753 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ffd1e520-f606-4179-ad21-88f6fd8ca383 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:09.325214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ffd1e520-f606-4179-ad21-88f6fd8ca383 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:09.471584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ffd1e520-f606-4179-ad21-88f6fd8ca383 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f757f2efd63f4526892784e78a402ccd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '802f856adff04f85b6731d6a97331da6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:09.479993 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ffd1e520-f606-4179-ad21-88f6fd8ca383 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] 10.222.0.42 "GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f" status: 200 len: 1314 microversion: 2.1 time: 0.170333 Aug 28 01:36:09.480576 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1767/3307] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:09 2022] GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f => generated 1314 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:10.501401 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0372a42a-b865-467d-8cf2-e87b907fc640 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:10.502012 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0372a42a-b865-467d-8cf2-e87b907fc640 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] [instance: 804c9d12-0553-40b5-815d-980078f32f2f] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:10.509488 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0372a42a-b865-467d-8cf2-e87b907fc640 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:10.509974 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0372a42a-b865-467d-8cf2-e87b907fc640 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:10.510408 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0372a42a-b865-467d-8cf2-e87b907fc640 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:10.511113 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0372a42a-b865-467d-8cf2-e87b907fc640 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:10.511555 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0372a42a-b865-467d-8cf2-e87b907fc640 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:10.512014 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0372a42a-b865-467d-8cf2-e87b907fc640 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:10.671903 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0372a42a-b865-467d-8cf2-e87b907fc640 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f757f2efd63f4526892784e78a402ccd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '802f856adff04f85b6731d6a97331da6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:10.681768 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0372a42a-b865-467d-8cf2-e87b907fc640 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] 10.222.0.42 "GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f" status: 200 len: 1464 microversion: 2.1 time: 0.183389 Aug 28 01:36:10.682020 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1541/3308] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:10 2022] GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f => generated 1464 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:10.744932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-12edd79e-9854-4981-87fa-83b11f97dcab tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:10.745519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-12edd79e-9854-4981-87fa-83b11f97dcab tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] [instance: 804c9d12-0553-40b5-815d-980078f32f2f] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:10.752420 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12edd79e-9854-4981-87fa-83b11f97dcab tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:10.752902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12edd79e-9854-4981-87fa-83b11f97dcab tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:10.753479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12edd79e-9854-4981-87fa-83b11f97dcab tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:10.754262 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12edd79e-9854-4981-87fa-83b11f97dcab tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:10.754762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12edd79e-9854-4981-87fa-83b11f97dcab tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:10.755320 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12edd79e-9854-4981-87fa-83b11f97dcab tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:10.783237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-12edd79e-9854-4981-87fa-83b11f97dcab tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] [instance: 804c9d12-0553-40b5-815d-980078f32f2f] Going to try to terminate instance {{(pid=120525) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 28 01:36:10.903442 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-12edd79e-9854-4981-87fa-83b11f97dcab tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] 10.222.0.42 "DELETE /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f" status: 204 len: 0 microversion: 2.1 time: 0.161987 Aug 28 01:36:10.904402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1768/3309] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 01:36:10 2022] DELETE /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 28 01:36:10.913615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-41ba4983-7634-4c49-9388-147aebf92b39 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:10.914395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-41ba4983-7634-4c49-9388-147aebf92b39 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] [instance: 804c9d12-0553-40b5-815d-980078f32f2f] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:10.921783 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41ba4983-7634-4c49-9388-147aebf92b39 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:10.922388 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41ba4983-7634-4c49-9388-147aebf92b39 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:10.922913 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41ba4983-7634-4c49-9388-147aebf92b39 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:10.923713 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41ba4983-7634-4c49-9388-147aebf92b39 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:10.924286 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41ba4983-7634-4c49-9388-147aebf92b39 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:10.924794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-41ba4983-7634-4c49-9388-147aebf92b39 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:11.066512 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-41ba4983-7634-4c49-9388-147aebf92b39 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f757f2efd63f4526892784e78a402ccd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '802f856adff04f85b6731d6a97331da6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:11.074851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-41ba4983-7634-4c49-9388-147aebf92b39 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] 10.222.0.42 "GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f" status: 200 len: 1470 microversion: 2.1 time: 0.165420 Aug 28 01:36:11.075416 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1542/3310] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:10 2022] GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f => generated 1470 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:12.096922 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-775c1fb5-1a58-427b-9181-39c3e7004e66 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:12.097551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-775c1fb5-1a58-427b-9181-39c3e7004e66 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] [instance: 804c9d12-0553-40b5-815d-980078f32f2f] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:12.105309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-775c1fb5-1a58-427b-9181-39c3e7004e66 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:12.105799 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-775c1fb5-1a58-427b-9181-39c3e7004e66 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:12.106209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-775c1fb5-1a58-427b-9181-39c3e7004e66 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:12.106911 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-775c1fb5-1a58-427b-9181-39c3e7004e66 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:12.107354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-775c1fb5-1a58-427b-9181-39c3e7004e66 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:12.107769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-775c1fb5-1a58-427b-9181-39c3e7004e66 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:12.251326 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-775c1fb5-1a58-427b-9181-39c3e7004e66 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f757f2efd63f4526892784e78a402ccd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '802f856adff04f85b6731d6a97331da6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:12.260088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-775c1fb5-1a58-427b-9181-39c3e7004e66 tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] 10.222.0.42 "GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f" status: 200 len: 1470 microversion: 2.1 time: 0.167304 Aug 28 01:36:12.260662 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1769/3311] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:12 2022] GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f => generated 1470 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:13.128150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-d45e5aaf-1466-4c3e-a2a4-338ad30ea24d req-6a543974-3c9f-47b9-a335-5a8a93579f86 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "804c9d12-0553-40b5-815d-980078f32f2f", "name": "network-vif-deleted", "tag": "d5838c45-b309-4df6-ba0a-4f2936bc2905"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:36:13.139127 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-d45e5aaf-1466-4c3e-a2a4-338ad30ea24d req-6a543974-3c9f-47b9-a335-5a8a93579f86 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:13.139560 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-d45e5aaf-1466-4c3e-a2a4-338ad30ea24d req-6a543974-3c9f-47b9-a335-5a8a93579f86 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:13.140411 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-d45e5aaf-1466-4c3e-a2a4-338ad30ea24d req-6a543974-3c9f-47b9-a335-5a8a93579f86 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:13.153221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-d45e5aaf-1466-4c3e-a2a4-338ad30ea24d req-6a543974-3c9f-47b9-a335-5a8a93579f86 service nova] Creating event network-vif-deleted:d5838c45-b309-4df6-ba0a-4f2936bc2905 for instance 804c9d12-0553-40b5-815d-980078f32f2f on hv-cinder-maste Aug 28 01:36:13.159648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-d45e5aaf-1466-4c3e-a2a4-338ad30ea24d req-6a543974-3c9f-47b9-a335-5a8a93579f86 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034796 Aug 28 01:36:13.160710 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1543/3312] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:36:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:36:13.280791 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6d4773ac-1788-4156-894e-4655f7905b7e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:13.281701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6d4773ac-1788-4156-894e-4655f7905b7e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] [instance: 804c9d12-0553-40b5-815d-980078f32f2f] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:13.289061 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d4773ac-1788-4156-894e-4655f7905b7e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:13.289793 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d4773ac-1788-4156-894e-4655f7905b7e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:13.290328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d4773ac-1788-4156-894e-4655f7905b7e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:13.291432 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d4773ac-1788-4156-894e-4655f7905b7e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:13.291882 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d4773ac-1788-4156-894e-4655f7905b7e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:13.292302 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d4773ac-1788-4156-894e-4655f7905b7e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:13.325857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d4773ac-1788-4156-894e-4655f7905b7e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:13.326570 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d4773ac-1788-4156-894e-4655f7905b7e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:13.327135 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6d4773ac-1788-4156-894e-4655f7905b7e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:13.327748 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.objects.instance [None req-6d4773ac-1788-4156-894e-4655f7905b7e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lazy-loading 'fault' on Instance uuid 804c9d12-0553-40b5-815d-980078f32f2f {{(pid=120525) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:36:13.379090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6d4773ac-1788-4156-894e-4655f7905b7e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f757f2efd63f4526892784e78a402ccd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '802f856adff04f85b6731d6a97331da6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:13.386985 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6d4773ac-1788-4156-894e-4655f7905b7e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] 10.222.0.42 "GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f" status: 200 len: 1304 microversion: 2.1 time: 0.110254 Aug 28 01:36:13.387566 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1770/3313] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:13 2022] GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f => generated 1304 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:14.414446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-94a8b368-811f-4ecd-8a1e-0ad05ff6930e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:14.415313 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-94a8b368-811f-4ecd-8a1e-0ad05ff6930e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] [instance: 804c9d12-0553-40b5-815d-980078f32f2f] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:14.425363 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94a8b368-811f-4ecd-8a1e-0ad05ff6930e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:14.426084 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94a8b368-811f-4ecd-8a1e-0ad05ff6930e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:14.427154 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94a8b368-811f-4ecd-8a1e-0ad05ff6930e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:14.428102 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94a8b368-811f-4ecd-8a1e-0ad05ff6930e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:14.428793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94a8b368-811f-4ecd-8a1e-0ad05ff6930e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:14.429515 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-94a8b368-811f-4ecd-8a1e-0ad05ff6930e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:14.446979 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-94a8b368-811f-4ecd-8a1e-0ad05ff6930e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] HTTP exception thrown: Instance 804c9d12-0553-40b5-815d-980078f32f2f could not be found. Aug 28 01:36:14.448632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-94a8b368-811f-4ecd-8a1e-0ad05ff6930e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Request method failure captured: Aug 28 01:36:14.448632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f HTTP/1.1 Aug 28 01:36:14.448632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:36:14.448632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:36:14.448632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:36:14.448632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:36:14.448632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:36:14.448632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:36:14.448632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:36:14.448632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:36:14.448632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCsZ7hgpNSATTVFMaacSR1qp5hbWozPVT9oI8799IlUeL-PTQ3Y7y_olgZHTid4z1VsIABbaT0ptSjz-43fATf1COgG3V5U29Qg7idLjWFMX_adcswCEhIO2BEUqiV_4wpFOTPv3LVP7na5o5o9MxXoAJCzUveerWqhIStEnKNDrSdSA Aug 28 01:36:14.448632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:36:14.448632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:36:14.448632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:36:14.448632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:36:14.448632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:36:14.448632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:36:14.448632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 802f856adff04f85b6731d6a97331da6 Aug 28 01:36:14.448632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-VolumesNegativeTest-419326894 Aug 28 01:36:14.448632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:36:14.448632 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:36:14.450037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_802f856adff04f85b6731d6a97331da6", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/802f856adff04f85b6731d6a97331da6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/802f856adff04f85b6731d6a97331da6"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/802f856adff04f85b6731d6a97331da6"}]}] Aug 28 01:36:14.450037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-VolumesNegativeTest-419326894 Aug 28 01:36:14.450037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 802f856adff04f85b6731d6a97331da6 Aug 28 01:36:14.450037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-VolumesNegativeTest-419326894 Aug 28 01:36:14.450037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-VolumesNegativeTest-419326894-project Aug 28 01:36:14.450037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:36:14.450037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:36:14.450037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: f757f2efd63f4526892784e78a402ccd Aug 28 01:36:14.450037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-VolumesNegativeTest-419326894-project Aug 28 01:36:14.450037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:36:14.450037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Instance 804c9d12-0553-40b5-815d-980078f32f2f could not be found. Aug 28 01:36:14.450037 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'id': '804c9d12-0553-40b5-815d-980078f32f2f'} Aug 28 01:36:14.450762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:36:14.450762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:36:14.450762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:36:14.450762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:36:14.450762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:36:14.450762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:36:14.450762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:36:14.450762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:36:14.450762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:36:14.450762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:36:14.450762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 804c9d12-0553-40b5-815d-980078f32f2f could not be found. Aug 28 01:36:14.450762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:36:14.450762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:36:14.450762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:36:14.450762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:36:14.450762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:36:14.450762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:36:14.450762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:36:14.450762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:36:14.450762 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:36:14.452040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:36:14.452040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 01:36:14.452040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 01:36:14.452040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:36:14.452040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:36:14.452040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:36:14.452040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:36:14.452040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 804c9d12-0553-40b5-815d-980078f32f2f could not be found. Aug 28 01:36:14.452040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:36:14.452040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:36:14.452040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:36:14.452040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:36:14.452040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:36:14.452040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:36:14.452040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:36:14.452040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:36:14.452040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 804c9d12-0553-40b5-815d-980078f32f2f could not be found. Aug 28 01:36:14.452040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:36:14.453210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-94a8b368-811f-4ecd-8a1e-0ad05ff6930e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] Returning 404 to user: Instance 804c9d12-0553-40b5-815d-980078f32f2f could not be found. {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:36:14.453210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-94a8b368-811f-4ecd-8a1e-0ad05ff6930e tempest-VolumesNegativeTest-419326894 tempest-VolumesNegativeTest-419326894-project] 10.222.0.42 "GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f" status: 404 len: 111 microversion: 2.1 time: 0.042266 Aug 28 01:36:14.453210 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1544/3314] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:14 2022] GET /compute/v2.1/servers/804c9d12-0553-40b5-815d-980078f32f2f => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:36:48.567161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-37a72800-8fae-47e1-a648-e676be0070d4 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Action: 'create', calling method: .version_select at 0x7f0e71ff9940>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-1223840802", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAsDZYKB9YtIdMKgfVR7SOTsOKK1BkemNq4n7yFd1rpDtcv+5Tl3f6q9tBBo0hesI+MiwBasIeT1DyZi9nhyyuqKe2E70gEwgR5HuY0SyZ90Y8RGgx7z0ulSaNBtAboOqQ=="}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:36:48.573380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=29,nova_api:UPDATE=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:36:48.574036 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=60,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:36:48.588649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-37a72800-8fae-47e1-a648-e676be0070d4 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Getting quotas for project 8bd4256c51f64ef9aa0f21ea2fcb55e7. Resources: {'key_pairs'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 01:36:48.593253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-37a72800-8fae-47e1-a648-e676be0070d4 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Getting quotas for user c5bc73cffe1e4fa095051e109b5f0d0c and project 8bd4256c51f64ef9aa0f21ea2fcb55e7. Resources: {'key_pairs'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 01:36:48.627222 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-37a72800-8fae-47e1-a648-e676be0070d4 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.063791 Aug 28 01:36:48.628097 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1771/3315] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Aug 28 01:36:48 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:36:48.684164 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-affd59b3-072e-42b7-91ef-b5d1e5f6edfd tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-2113056106", "imageRef": "857ecfaf-3cea-4817-95a7-485e295cbe32", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-1223840802", "networks": [{"uuid": "02d0c8b8-ca5f-46fa-b8e4-febdb21cf1b8"}]}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:36:48.693451 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=3 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:36:48.694080 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=50 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:36:48.746842 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.network.neutron [None req-affd59b3-072e-42b7-91ef-b5d1e5f6edfd tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] validate_networks() for [('02d0c8b8-ca5f-46fa-b8e4-febdb21cf1b8', None, None, None, None, None)] {{(pid=120524) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 28 01:36:49.272886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-affd59b3-072e-42b7-91ef-b5d1e5f6edfd tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Flavor limits 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 28 01:36:49.273482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-affd59b3-072e-42b7-91ef-b5d1e5f6edfd tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Image limits 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 28 01:36:49.273948 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-affd59b3-072e-42b7-91ef-b5d1e5f6edfd tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Flavor pref 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 28 01:36:49.274350 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-affd59b3-072e-42b7-91ef-b5d1e5f6edfd tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Image pref 0:0:0 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 28 01:36:49.274744 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.virt.hardware [None req-affd59b3-072e-42b7-91ef-b5d1e5f6edfd tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120524) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 28 01:36:49.284534 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-affd59b3-072e-42b7-91ef-b5d1e5f6edfd tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:49.285049 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-affd59b3-072e-42b7-91ef-b5d1e5f6edfd tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:49.285486 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-affd59b3-072e-42b7-91ef-b5d1e5f6edfd tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:49.286185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-affd59b3-072e-42b7-91ef-b5d1e5f6edfd tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:49.286824 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-affd59b3-072e-42b7-91ef-b5d1e5f6edfd tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:49.287228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-affd59b3-072e-42b7-91ef-b5d1e5f6edfd tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:49.304176 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-affd59b3-072e-42b7-91ef-b5d1e5f6edfd tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Getting quotas for project 8bd4256c51f64ef9aa0f21ea2fcb55e7. Resources: {'ram', 'instances', 'cores'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 01:36:49.308480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-affd59b3-072e-42b7-91ef-b5d1e5f6edfd tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Getting quotas for user c5bc73cffe1e4fa095051e109b5f0d0c and project 8bd4256c51f64ef9aa0f21ea2fcb55e7. Resources: {'ram', 'instances', 'cores'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 01:36:49.319221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-affd59b3-072e-42b7-91ef-b5d1e5f6edfd tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Going to run 1 instances... {{(pid=120524) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 28 01:36:49.321512 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-affd59b3-072e-42b7-91ef-b5d1e5f6edfd tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='857ecfaf-3cea-4817-95a7-485e295cbe32',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120524) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 28 01:36:49.367806 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-affd59b3-072e-42b7-91ef-b5d1e5f6edfd tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.687764 Aug 28 01:36:49.368460 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1545/3316] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Aug 28 01:36:48 2022] POST /compute/v2.1/servers => generated 380 bytes in 689 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 28 01:36:49.379303 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a3b11c36-9c5c-4911-82f5-57118b8d4c1f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:49.379919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a3b11c36-9c5c-4911-82f5-57118b8d4c1f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:49.434631 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a3b11c36-9c5c-4911-82f5-57118b8d4c1f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:49.442081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a3b11c36-9c5c-4911-82f5-57118b8d4c1f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1259 microversion: 2.1 time: 0.066614 Aug 28 01:36:49.442840 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1772/3317] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:49 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1259 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:50.462926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4b21c8b6-9123-4b88-ac81-dc71b7ee532f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:50.463574 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4b21c8b6-9123-4b88-ac81-dc71b7ee532f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:50.470134 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b21c8b6-9123-4b88-ac81-dc71b7ee532f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:50.470614 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b21c8b6-9123-4b88-ac81-dc71b7ee532f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:50.471039 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b21c8b6-9123-4b88-ac81-dc71b7ee532f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:50.471730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b21c8b6-9123-4b88-ac81-dc71b7ee532f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:50.472140 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b21c8b6-9123-4b88-ac81-dc71b7ee532f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:50.472760 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b21c8b6-9123-4b88-ac81-dc71b7ee532f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:50.538244 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4b21c8b6-9123-4b88-ac81-dc71b7ee532f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:50.546937 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4b21c8b6-9123-4b88-ac81-dc71b7ee532f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1311 microversion: 2.1 time: 0.088138 Aug 28 01:36:50.547552 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1546/3318] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:50 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1311 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:51.569755 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-267b176b-29ab-4a02-9fbb-e47d3bbdc19a tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:51.570960 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-267b176b-29ab-4a02-9fbb-e47d3bbdc19a tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:51.579105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-267b176b-29ab-4a02-9fbb-e47d3bbdc19a tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:51.579826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-267b176b-29ab-4a02-9fbb-e47d3bbdc19a tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:51.580477 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-267b176b-29ab-4a02-9fbb-e47d3bbdc19a tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:51.581422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-267b176b-29ab-4a02-9fbb-e47d3bbdc19a tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:51.582011 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-267b176b-29ab-4a02-9fbb-e47d3bbdc19a tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:51.582723 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-267b176b-29ab-4a02-9fbb-e47d3bbdc19a tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:51.771859 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-267b176b-29ab-4a02-9fbb-e47d3bbdc19a tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:51.781573 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-267b176b-29ab-4a02-9fbb-e47d3bbdc19a tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1359 microversion: 2.1 time: 0.215575 Aug 28 01:36:51.782719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1773/3319] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:51 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1359 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:52.503658 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b687821e-68da-473c-8b0c-06295b547763 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:52.504778 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b687821e-68da-473c-8b0c-06295b547763 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:52.512019 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b687821e-68da-473c-8b0c-06295b547763 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:52.512767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b687821e-68da-473c-8b0c-06295b547763 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:52.513241 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b687821e-68da-473c-8b0c-06295b547763 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:52.513943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b687821e-68da-473c-8b0c-06295b547763 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:52.514392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b687821e-68da-473c-8b0c-06295b547763 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:52.514834 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b687821e-68da-473c-8b0c-06295b547763 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:52.529913 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-b687821e-68da-473c-8b0c-06295b547763 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] HTTP exception thrown: Instance e434e4f9-9d0e-46f2-bea3-395bae33fc92 could not be found. Aug 28 01:36:52.531580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b687821e-68da-473c-8b0c-06295b547763 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Request method failure captured: Aug 28 01:36:52.531580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: DELETE /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 HTTP/1.1 Aug 28 01:36:52.531580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:36:52.531580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:36:52.531580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:36:52.531580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:36:52.531580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:36:52.531580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:36:52.531580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:36:52.531580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:36:52.531580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCrzmE0DQ9Mzy06OSRTD7Zc_5zbHJmgF9lrNWigauhfXdKgrk3t1iKQIvc72yKH8l7izSVSee0mW2vlsUYm08FWc-PX6YtIoZHTUEF5-2Kw4nOfJw6h6qrTA-JveSh59xEFSntY-alln8RdPrN4NOCHaeIkj1safHIIs1D9ZkiRfDOzw Aug 28 01:36:52.531580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:36:52.531580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:36:52.531580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:36:52.531580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:36:52.531580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:36:52.531580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:36:52.531580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 2348c1ddbec1476bacb194214794015d Aug 28 01:36:52.531580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-AttachVolumeTestJSON-192914243 Aug 28 01:36:52.531580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:36:52.531580 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:36:52.532851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_2348c1ddbec1476bacb194214794015d", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2348c1ddbec1476bacb194214794015d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2348c1ddbec1476bacb194214794015d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/2348c1ddbec1476bacb194214794015d"}]}] Aug 28 01:36:52.532851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-AttachVolumeTestJSON-192914243 Aug 28 01:36:52.532851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 2348c1ddbec1476bacb194214794015d Aug 28 01:36:52.532851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-AttachVolumeTestJSON-192914243 Aug 28 01:36:52.532851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-AttachVolumeTestJSON-192914243-project Aug 28 01:36:52.532851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:36:52.532851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:36:52.532851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: 3de5d68f67784e33bdcd19a9cdfed744 Aug 28 01:36:52.532851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-AttachVolumeTestJSON-192914243-project Aug 28 01:36:52.532851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:36:52.532851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Instance e434e4f9-9d0e-46f2-bea3-395bae33fc92 could not be found. Aug 28 01:36:52.532851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'id': 'e434e4f9-9d0e-46f2-bea3-395bae33fc92'} Aug 28 01:36:52.533589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:36:52.533589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:36:52.533589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:36:52.533589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:36:52.533589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:36:52.533589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:36:52.533589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:36:52.533589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:36:52.533589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:36:52.533589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:36:52.533589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e434e4f9-9d0e-46f2-bea3-395bae33fc92 could not be found. Aug 28 01:36:52.533589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:36:52.533589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:36:52.533589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:36:52.533589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:36:52.533589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:36:52.533589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:36:52.533589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:36:52.533589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:36:52.533589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:36:52.534703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:36:52.534703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Aug 28 01:36:52.534703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Aug 28 01:36:52.534703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Aug 28 01:36:52.534703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Aug 28 01:36:52.534703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:36:52.534703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:36:52.534703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:36:52.534703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:36:52.534703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e434e4f9-9d0e-46f2-bea3-395bae33fc92 could not be found. Aug 28 01:36:52.534703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:36:52.534703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:36:52.534703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:36:52.534703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:36:52.534703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:36:52.534703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:36:52.534703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:36:52.534703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:36:52.534703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e434e4f9-9d0e-46f2-bea3-395bae33fc92 could not be found. Aug 28 01:36:52.534703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:36:52.535853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b687821e-68da-473c-8b0c-06295b547763 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Returning 404 to user: Instance e434e4f9-9d0e-46f2-bea3-395bae33fc92 could not be found. {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:36:52.535853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b687821e-68da-473c-8b0c-06295b547763 tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "DELETE /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 404 len: 111 microversion: 2.1 time: 0.033376 Aug 28 01:36:52.535853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1547/3320] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 01:36:52 2022] DELETE /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:36:52.541736 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9c81d408-7833-4eff-9efd-850ea364a68a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:52.542347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9c81d408-7833-4eff-9efd-850ea364a68a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] [instance: e434e4f9-9d0e-46f2-bea3-395bae33fc92] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:52.548997 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c81d408-7833-4eff-9efd-850ea364a68a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:52.549345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c81d408-7833-4eff-9efd-850ea364a68a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:52.549895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c81d408-7833-4eff-9efd-850ea364a68a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:52.550478 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c81d408-7833-4eff-9efd-850ea364a68a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:52.550945 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c81d408-7833-4eff-9efd-850ea364a68a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:52.551343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9c81d408-7833-4eff-9efd-850ea364a68a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:52.566611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-9c81d408-7833-4eff-9efd-850ea364a68a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] HTTP exception thrown: Instance e434e4f9-9d0e-46f2-bea3-395bae33fc92 could not be found. Aug 28 01:36:52.568324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9c81d408-7833-4eff-9efd-850ea364a68a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Request method failure captured: Aug 28 01:36:52.568324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 HTTP/1.1 Aug 28 01:36:52.568324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:36:52.568324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:36:52.568324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:36:52.568324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:36:52.568324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:36:52.568324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:36:52.568324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:36:52.568324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:36:52.568324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCrzmE0DQ9Mzy06OSRTD7Zc_5zbHJmgF9lrNWigauhfXdKgrk3t1iKQIvc72yKH8l7izSVSee0mW2vlsUYm08FWc-PX6YtIoZHTUEF5-2Kw4nOfJw6h6qrTA-JveSh59xEFSntY-alln8RdPrN4NOCHaeIkj1safHIIs1D9ZkiRfDOzw Aug 28 01:36:52.568324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:36:52.568324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:36:52.568324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:36:52.568324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:36:52.568324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:36:52.568324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:36:52.568324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 2348c1ddbec1476bacb194214794015d Aug 28 01:36:52.568324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-AttachVolumeTestJSON-192914243 Aug 28 01:36:52.568324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:36:52.568324 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:36:52.569718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_2348c1ddbec1476bacb194214794015d", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2348c1ddbec1476bacb194214794015d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2348c1ddbec1476bacb194214794015d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/2348c1ddbec1476bacb194214794015d"}]}] Aug 28 01:36:52.569718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-AttachVolumeTestJSON-192914243 Aug 28 01:36:52.569718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 2348c1ddbec1476bacb194214794015d Aug 28 01:36:52.569718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-AttachVolumeTestJSON-192914243 Aug 28 01:36:52.569718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-AttachVolumeTestJSON-192914243-project Aug 28 01:36:52.569718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:36:52.569718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:36:52.569718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: 3de5d68f67784e33bdcd19a9cdfed744 Aug 28 01:36:52.569718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-AttachVolumeTestJSON-192914243-project Aug 28 01:36:52.569718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 01:36:52.569718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Instance e434e4f9-9d0e-46f2-bea3-395bae33fc92 could not be found. Aug 28 01:36:52.569718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'id': 'e434e4f9-9d0e-46f2-bea3-395bae33fc92'} Aug 28 01:36:52.570861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:36:52.570861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:36:52.570861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:36:52.570861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:36:52.570861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:36:52.570861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:36:52.570861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:36:52.570861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:36:52.570861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:36:52.570861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:36:52.570861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e434e4f9-9d0e-46f2-bea3-395bae33fc92 could not be found. Aug 28 01:36:52.570861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:36:52.570861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:36:52.570861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:36:52.570861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:36:52.570861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:36:52.570861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:36:52.570861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:36:52.570861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:36:52.570861 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:36:52.571987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:36:52.571987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 01:36:52.571987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 01:36:52.571987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:36:52.571987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:36:52.571987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:36:52.571987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:36:52.571987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e434e4f9-9d0e-46f2-bea3-395bae33fc92 could not be found. Aug 28 01:36:52.571987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:36:52.571987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:36:52.571987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:36:52.571987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:36:52.571987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:36:52.571987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:36:52.571987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:36:52.571987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:36:52.571987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e434e4f9-9d0e-46f2-bea3-395bae33fc92 could not be found. Aug 28 01:36:52.571987 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:36:52.573265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9c81d408-7833-4eff-9efd-850ea364a68a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] Returning 404 to user: Instance e434e4f9-9d0e-46f2-bea3-395bae33fc92 could not be found. {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:36:52.573265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9c81d408-7833-4eff-9efd-850ea364a68a tempest-AttachVolumeTestJSON-192914243 tempest-AttachVolumeTestJSON-192914243-project] 10.222.0.42 "GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92" status: 404 len: 111 microversion: 2.1 time: 0.031941 Aug 28 01:36:52.573265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1774/3321] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:52 2022] GET /compute/v2.1/servers/e434e4f9-9d0e-46f2-bea3-395bae33fc92 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:36:52.815874 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a3e8bc8f-68fb-4147-b4dd-fdefa44aa97e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:52.817512 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a3e8bc8f-68fb-4147-b4dd-fdefa44aa97e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:52.832166 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3e8bc8f-68fb-4147-b4dd-fdefa44aa97e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:52.835341 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3e8bc8f-68fb-4147-b4dd-fdefa44aa97e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:52.836031 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3e8bc8f-68fb-4147-b4dd-fdefa44aa97e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:52.837558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3e8bc8f-68fb-4147-b4dd-fdefa44aa97e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:52.839378 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3e8bc8f-68fb-4147-b4dd-fdefa44aa97e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:52.840654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a3e8bc8f-68fb-4147-b4dd-fdefa44aa97e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:53.102103 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a3e8bc8f-68fb-4147-b4dd-fdefa44aa97e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:53.114224 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a3e8bc8f-68fb-4147-b4dd-fdefa44aa97e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1359 microversion: 2.1 time: 0.315542 Aug 28 01:36:53.114838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1548/3322] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:52 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1359 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:53.360038 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-09639971-684b-487a-9d2b-46fab1e1ec10 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:53.360952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-09639971-684b-487a-9d2b-46fab1e1ec10 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:53.376111 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09639971-684b-487a-9d2b-46fab1e1ec10 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:53.376604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09639971-684b-487a-9d2b-46fab1e1ec10 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:53.377253 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09639971-684b-487a-9d2b-46fab1e1ec10 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:53.378234 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09639971-684b-487a-9d2b-46fab1e1ec10 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:53.378776 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09639971-684b-487a-9d2b-46fab1e1ec10 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:53.379241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-09639971-684b-487a-9d2b-46fab1e1ec10 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:53.422294 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-09639971-684b-487a-9d2b-46fab1e1ec10 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Going to try to terminate instance {{(pid=120525) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 28 01:36:53.535440 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-09639971-684b-487a-9d2b-46fab1e1ec10 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "DELETE /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 204 len: 0 microversion: 2.1 time: 0.181011 Aug 28 01:36:53.536781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1775/3323] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 01:36:53 2022] DELETE /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 28 01:36:53.545105 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9bdc6cb2-e78c-4e70-848d-b5b7493549fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:53.546297 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9bdc6cb2-e78c-4e70-848d-b5b7493549fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:53.558301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bdc6cb2-e78c-4e70-848d-b5b7493549fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:53.559105 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bdc6cb2-e78c-4e70-848d-b5b7493549fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:53.560238 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bdc6cb2-e78c-4e70-848d-b5b7493549fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:53.561827 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bdc6cb2-e78c-4e70-848d-b5b7493549fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:53.563610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bdc6cb2-e78c-4e70-848d-b5b7493549fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:53.564242 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9bdc6cb2-e78c-4e70-848d-b5b7493549fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:53.819321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-9bdc6cb2-e78c-4e70-848d-b5b7493549fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:53.827344 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9bdc6cb2-e78c-4e70-848d-b5b7493549fd tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1509 microversion: 2.1 time: 0.286329 Aug 28 01:36:53.827943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1549/3324] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:53 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1509 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:54.135273 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6c02c4ac-6e4d-4007-93f4-c4b94b1d9e6a tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:54.136192 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6c02c4ac-6e4d-4007-93f4-c4b94b1d9e6a tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:54.143593 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c02c4ac-6e4d-4007-93f4-c4b94b1d9e6a tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:54.144155 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c02c4ac-6e4d-4007-93f4-c4b94b1d9e6a tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:54.144498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c02c4ac-6e4d-4007-93f4-c4b94b1d9e6a tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:54.145365 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c02c4ac-6e4d-4007-93f4-c4b94b1d9e6a tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:54.145938 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c02c4ac-6e4d-4007-93f4-c4b94b1d9e6a tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:54.146213 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6c02c4ac-6e4d-4007-93f4-c4b94b1d9e6a tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:54.706332 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6c02c4ac-6e4d-4007-93f4-c4b94b1d9e6a tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:54.714142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6c02c4ac-6e4d-4007-93f4-c4b94b1d9e6a tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1359 microversion: 2.1 time: 0.583176 Aug 28 01:36:54.714686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1776/3325] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:54 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1359 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:54.848242 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-05fb6ada-807f-49e6-8dc4-bec8aadbe737 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:54.848768 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-05fb6ada-807f-49e6-8dc4-bec8aadbe737 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:54.855721 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-05fb6ada-807f-49e6-8dc4-bec8aadbe737 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:54.856208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-05fb6ada-807f-49e6-8dc4-bec8aadbe737 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:54.856652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-05fb6ada-807f-49e6-8dc4-bec8aadbe737 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:54.857376 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-05fb6ada-807f-49e6-8dc4-bec8aadbe737 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:54.857820 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-05fb6ada-807f-49e6-8dc4-bec8aadbe737 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:54.858220 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-05fb6ada-807f-49e6-8dc4-bec8aadbe737 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:55.089693 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-05fb6ada-807f-49e6-8dc4-bec8aadbe737 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:55.098652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-05fb6ada-807f-49e6-8dc4-bec8aadbe737 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1509 microversion: 2.1 time: 0.254478 Aug 28 01:36:55.099152 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1550/3326] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:54 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1509 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:55.736334 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f6bbd5a9-2285-419b-9647-85e5cd8b29ab tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:55.736889 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f6bbd5a9-2285-419b-9647-85e5cd8b29ab tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:55.743342 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6bbd5a9-2285-419b-9647-85e5cd8b29ab tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:55.743820 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6bbd5a9-2285-419b-9647-85e5cd8b29ab tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:55.744242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6bbd5a9-2285-419b-9647-85e5cd8b29ab tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:55.744925 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6bbd5a9-2285-419b-9647-85e5cd8b29ab tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:55.750390 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6bbd5a9-2285-419b-9647-85e5cd8b29ab tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:55.750828 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f6bbd5a9-2285-419b-9647-85e5cd8b29ab tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:55.908807 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f6bbd5a9-2285-419b-9647-85e5cd8b29ab tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:55.918671 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f6bbd5a9-2285-419b-9647-85e5cd8b29ab tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1359 microversion: 2.1 time: 0.187066 Aug 28 01:36:55.933065 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1777/3327] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:55 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1359 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:56.132192 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e651961f-65ce-43a0-a9c8-720181d2faf2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:56.134007 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e651961f-65ce-43a0-a9c8-720181d2faf2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:56.142692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e651961f-65ce-43a0-a9c8-720181d2faf2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:56.144401 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e651961f-65ce-43a0-a9c8-720181d2faf2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:56.145074 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e651961f-65ce-43a0-a9c8-720181d2faf2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:56.146382 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e651961f-65ce-43a0-a9c8-720181d2faf2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:56.147069 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e651961f-65ce-43a0-a9c8-720181d2faf2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:56.147997 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e651961f-65ce-43a0-a9c8-720181d2faf2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:56.318696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e651961f-65ce-43a0-a9c8-720181d2faf2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:56.327454 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e651961f-65ce-43a0-a9c8-720181d2faf2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1509 microversion: 2.1 time: 0.205085 Aug 28 01:36:56.341048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1551/3328] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:56 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1509 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:56.940919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3bd45ee9-9198-4c28-8621-d9929125cb09 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:56.941633 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3bd45ee9-9198-4c28-8621-d9929125cb09 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:56.947879 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bd45ee9-9198-4c28-8621-d9929125cb09 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:56.948604 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bd45ee9-9198-4c28-8621-d9929125cb09 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:56.949060 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bd45ee9-9198-4c28-8621-d9929125cb09 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:56.950168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bd45ee9-9198-4c28-8621-d9929125cb09 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:56.950736 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bd45ee9-9198-4c28-8621-d9929125cb09 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:56.951185 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3bd45ee9-9198-4c28-8621-d9929125cb09 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:57.100603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3bd45ee9-9198-4c28-8621-d9929125cb09 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:57.127219 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3bd45ee9-9198-4c28-8621-d9929125cb09 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1359 microversion: 2.1 time: 0.190204 Aug 28 01:36:57.141047 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1778/3329] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:56 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1359 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:57.347734 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a33defdd-d1e7-4a75-874e-55bbc2d4291a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:57.348337 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a33defdd-d1e7-4a75-874e-55bbc2d4291a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:57.354878 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a33defdd-d1e7-4a75-874e-55bbc2d4291a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:57.355366 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a33defdd-d1e7-4a75-874e-55bbc2d4291a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:57.355812 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a33defdd-d1e7-4a75-874e-55bbc2d4291a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:57.356534 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a33defdd-d1e7-4a75-874e-55bbc2d4291a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:57.356953 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a33defdd-d1e7-4a75-874e-55bbc2d4291a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:57.357487 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a33defdd-d1e7-4a75-874e-55bbc2d4291a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:57.537242 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a33defdd-d1e7-4a75-874e-55bbc2d4291a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:57.547355 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a33defdd-d1e7-4a75-874e-55bbc2d4291a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1509 microversion: 2.1 time: 0.203040 Aug 28 01:36:57.548188 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1552/3330] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:57 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1509 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:58.147584 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-16d5be7f-f3b1-4220-86c1-dd9e3b1bac12 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:58.148240 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-16d5be7f-f3b1-4220-86c1-dd9e3b1bac12 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:58.154438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16d5be7f-f3b1-4220-86c1-dd9e3b1bac12 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:58.154910 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16d5be7f-f3b1-4220-86c1-dd9e3b1bac12 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:58.155345 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16d5be7f-f3b1-4220-86c1-dd9e3b1bac12 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:58.156086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16d5be7f-f3b1-4220-86c1-dd9e3b1bac12 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:58.156537 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16d5be7f-f3b1-4220-86c1-dd9e3b1bac12 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:58.156976 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16d5be7f-f3b1-4220-86c1-dd9e3b1bac12 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:58.310407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-16d5be7f-f3b1-4220-86c1-dd9e3b1bac12 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:58.318812 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-16d5be7f-f3b1-4220-86c1-dd9e3b1bac12 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1359 microversion: 2.1 time: 0.174957 Aug 28 01:36:58.319438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1779/3331] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:58 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1359 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:36:58.440879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a262be8a-28c1-4a7f-8ab7-38bc68894a8d tempest-VolumesGetTestJSON-2113097466 tempest-VolumesGetTestJSON-2113097466-project] Action: 'create', calling method: .version_select at 0x7f0e6e724940>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-602472547", "metadata": {"Type": "work"}}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:36:58.570017 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-173305c4-186d-4c09-9b43-e64a1c11118e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:58.570740 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-173305c4-186d-4c09-9b43-e64a1c11118e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:58.578115 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-173305c4-186d-4c09-9b43-e64a1c11118e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:58.578860 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-173305c4-186d-4c09-9b43-e64a1c11118e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:58.579462 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-173305c4-186d-4c09-9b43-e64a1c11118e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:58.580327 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-173305c4-186d-4c09-9b43-e64a1c11118e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:58.581073 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-173305c4-186d-4c09-9b43-e64a1c11118e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:58.581717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-173305c4-186d-4c09-9b43-e64a1c11118e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:58.754055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-173305c4-186d-4c09-9b43-e64a1c11118e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:58.762734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-173305c4-186d-4c09-9b43-e64a1c11118e tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1509 microversion: 2.1 time: 0.196616 Aug 28 01:36:58.773850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1780/3332] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:58 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1509 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:36:58.867596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a262be8a-28c1-4a7f-8ab7-38bc68894a8d tempest-VolumesGetTestJSON-2113097466 tempest-VolumesGetTestJSON-2113097466-project] 10.222.0.42 "POST /compute/v2.1/os-volumes" status: 200 len: 352 microversion: 2.1 time: 0.571110 Aug 28 01:36:58.877437 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1553/3333] 10.222.0.42 () {60 vars in 1156 bytes} [Sun Aug 28 01:36:58 2022] POST /compute/v2.1/os-volumes => generated 352 bytes in 573 msecs (HTTP/1.1 200) 10 headers in 448 bytes (2 switches on core 0) Aug 28 01:36:58.882393 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-03ed5060-3aee-4277-bb80-2552ee7a61bd tempest-VolumesGetTestJSON-2113097466 tempest-VolumesGetTestJSON-2113097466-project] Calling method '.version_select at 0x7f0e720fa820>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:58.933600 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-03ed5060-3aee-4277-bb80-2552ee7a61bd tempest-VolumesGetTestJSON-2113097466 tempest-VolumesGetTestJSON-2113097466-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/38f957c5-86af-49f7-893e-1827e2f1d112" status: 200 len: 352 microversion: 2.1 time: 0.055189 Aug 28 01:36:58.934196 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1781/3334] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Aug 28 01:36:58 2022] GET /compute/v2.1/os-volumes/38f957c5-86af-49f7-893e-1827e2f1d112 => generated 352 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Aug 28 01:36:59.337822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=2 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:36:59.338691 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=6 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:36:59.344231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-938c1aaa-0e75-4bfa-a394-cdfd2e5316e4 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:59.345289 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-938c1aaa-0e75-4bfa-a394-cdfd2e5316e4 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:59.353253 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-938c1aaa-0e75-4bfa-a394-cdfd2e5316e4 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:59.353735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-938c1aaa-0e75-4bfa-a394-cdfd2e5316e4 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:59.354170 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-938c1aaa-0e75-4bfa-a394-cdfd2e5316e4 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:59.354880 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-938c1aaa-0e75-4bfa-a394-cdfd2e5316e4 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:59.355322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-938c1aaa-0e75-4bfa-a394-cdfd2e5316e4 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:59.355739 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-938c1aaa-0e75-4bfa-a394-cdfd2e5316e4 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:59.499854 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-938c1aaa-0e75-4bfa-a394-cdfd2e5316e4 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:36:59.511333 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-938c1aaa-0e75-4bfa-a394-cdfd2e5316e4 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1359 microversion: 2.1 time: 0.174516 Aug 28 01:36:59.511908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1554/3335] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:59 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1359 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:36:59.785581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1af80003-be26-41f6-940f-4ca541485921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:36:59.786148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1af80003-be26-41f6-940f-4ca541485921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:36:59.790081 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=6 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:36:59.800941 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1af80003-be26-41f6-940f-4ca541485921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:59.801546 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1af80003-be26-41f6-940f-4ca541485921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:59.802049 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1af80003-be26-41f6-940f-4ca541485921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:59.802747 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1af80003-be26-41f6-940f-4ca541485921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:36:59.803697 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1af80003-be26-41f6-940f-4ca541485921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:36:59.803697 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1af80003-be26-41f6-940f-4ca541485921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:36:59.949195 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d3491b55-3408-489a-acf1-934746f2d3ae tempest-VolumesGetTestJSON-2113097466 tempest-VolumesGetTestJSON-2113097466-project] Calling method '.version_select at 0x7f0e6e6ab0d0>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:00.027336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d3491b55-3408-489a-acf1-934746f2d3ae tempest-VolumesGetTestJSON-2113097466 tempest-VolumesGetTestJSON-2113097466-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/38f957c5-86af-49f7-893e-1827e2f1d112" status: 200 len: 353 microversion: 2.1 time: 0.082160 Aug 28 01:37:00.027753 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1555/3336] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Aug 28 01:36:59 2022] GET /compute/v2.1/os-volumes/38f957c5-86af-49f7-893e-1827e2f1d112 => generated 353 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:37:00.035503 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1af80003-be26-41f6-940f-4ca541485921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:00.040369 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7a845022-31de-4641-92a9-9d194198231d tempest-VolumesGetTestJSON-2113097466 tempest-VolumesGetTestJSON-2113097466-project] Calling method '.version_select at 0x7f0e6e7249d0>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:00.043548 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1af80003-be26-41f6-940f-4ca541485921 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1509 microversion: 2.1 time: 0.262923 Aug 28 01:37:00.044183 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1782/3337] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:36:59 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1509 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:00.094586 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7a845022-31de-4641-92a9-9d194198231d tempest-VolumesGetTestJSON-2113097466 tempest-VolumesGetTestJSON-2113097466-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/38f957c5-86af-49f7-893e-1827e2f1d112" status: 200 len: 353 microversion: 2.1 time: 0.057951 Aug 28 01:37:00.095498 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1556/3338] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Aug 28 01:37:00 2022] GET /compute/v2.1/os-volumes/38f957c5-86af-49f7-893e-1827e2f1d112 => generated 353 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:37:00.119211 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-511a28c9-e94c-47d7-864b-18e3db3f2611 tempest-VolumesGetTestJSON-2113097466 tempest-VolumesGetTestJSON-2113097466-project] Calling method '.version_select at 0x7f0e71ff9af0>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:00.189763 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-511a28c9-e94c-47d7-864b-18e3db3f2611 tempest-VolumesGetTestJSON-2113097466 tempest-VolumesGetTestJSON-2113097466-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/38f957c5-86af-49f7-893e-1827e2f1d112" status: 202 len: 0 microversion: 2.1 time: 0.075146 Aug 28 01:37:00.190353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1783/3339] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Aug 28 01:37:00 2022] DELETE /compute/v2.1/os-volumes/38f957c5-86af-49f7-893e-1827e2f1d112 => generated 0 bytes in 77 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 01:37:00.199100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-c0cd169e-d651-4b70-bd77-17feeae979ee tempest-VolumesGetTestJSON-2113097466 tempest-VolumesGetTestJSON-2113097466-project] Calling method '.version_select at 0x7f0e6e98d940>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:00.275746 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-c0cd169e-d651-4b70-bd77-17feeae979ee tempest-VolumesGetTestJSON-2113097466 tempest-VolumesGetTestJSON-2113097466-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/38f957c5-86af-49f7-893e-1827e2f1d112" status: 200 len: 352 microversion: 2.1 time: 0.080897 Aug 28 01:37:00.276568 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1557/3340] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Aug 28 01:37:00 2022] GET /compute/v2.1/os-volumes/38f957c5-86af-49f7-893e-1827e2f1d112 => generated 352 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:37:00.535393 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e01a2c89-a41f-46b8-9022-6dd7b5a361e5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:00.536086 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e01a2c89-a41f-46b8-9022-6dd7b5a361e5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:00.543844 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e01a2c89-a41f-46b8-9022-6dd7b5a361e5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:00.544329 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e01a2c89-a41f-46b8-9022-6dd7b5a361e5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:00.544757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e01a2c89-a41f-46b8-9022-6dd7b5a361e5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:00.545490 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e01a2c89-a41f-46b8-9022-6dd7b5a361e5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:00.545931 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e01a2c89-a41f-46b8-9022-6dd7b5a361e5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:00.546583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e01a2c89-a41f-46b8-9022-6dd7b5a361e5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:00.706218 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e01a2c89-a41f-46b8-9022-6dd7b5a361e5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:00.715428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e01a2c89-a41f-46b8-9022-6dd7b5a361e5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1359 microversion: 2.1 time: 0.186177 Aug 28 01:37:00.716605 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1784/3341] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:00 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1359 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:01.065461 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-76dc51c9-8838-44ba-9ece-d767083ca5a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:01.066870 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-76dc51c9-8838-44ba-9ece-d767083ca5a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:01.074226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-76dc51c9-8838-44ba-9ece-d767083ca5a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:01.074805 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-76dc51c9-8838-44ba-9ece-d767083ca5a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:01.075305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-76dc51c9-8838-44ba-9ece-d767083ca5a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:01.076083 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-76dc51c9-8838-44ba-9ece-d767083ca5a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:01.076599 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-76dc51c9-8838-44ba-9ece-d767083ca5a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:01.077119 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-76dc51c9-8838-44ba-9ece-d767083ca5a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:01.219154 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-76dc51c9-8838-44ba-9ece-d767083ca5a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:01.228326 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-76dc51c9-8838-44ba-9ece-d767083ca5a1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1509 microversion: 2.1 time: 0.166909 Aug 28 01:37:01.229022 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1558/3342] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:01 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1509 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:01.294949 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ce430ef6-e7f6-4c06-be7b-40ab598c60ca tempest-VolumesGetTestJSON-2113097466 tempest-VolumesGetTestJSON-2113097466-project] Calling method '.version_select at 0x7f0e720e83a0>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:01.331919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-ce430ef6-e7f6-4c06-be7b-40ab598c60ca tempest-VolumesGetTestJSON-2113097466 tempest-VolumesGetTestJSON-2113097466-project] HTTP exception thrown: Volume 38f957c5-86af-49f7-893e-1827e2f1d112 could not be found. Aug 28 01:37:01.334474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ce430ef6-e7f6-4c06-be7b-40ab598c60ca tempest-VolumesGetTestJSON-2113097466 tempest-VolumesGetTestJSON-2113097466-project] Request method failure captured: Aug 28 01:37:01.334474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: GET /compute/v2.1/os-volumes/38f957c5-86af-49f7-893e-1827e2f1d112 HTTP/1.1 Aug 28 01:37:01.334474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:37:01.334474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:37:01.334474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:37:01.334474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:37:01.334474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:37:01.334474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:37:01.334474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:37:01.334474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:37:01.334474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCsa6g5PMzrH3X3hdfaINgTmrn6g8t1CfgJRnl4W0jSt5XkbTO3Git_0Aad7ZowT1OVNzntFfD62Rc3KdFIJfQX1CikAnj9gsAqySAWUhJ9kc3KzwGUR1-jQUNpqr6bIVXCFWn-itsxFN8ZRnZH4gR19K3iuZCSDozqDLm8LLv7Lqu1s Aug 28 01:37:01.334474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:37:01.334474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:37:01.334474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:37:01.334474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:37:01.334474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:37:01.334474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:37:01.334474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 9f22397792284cdc8601883d9b7f1f43 Aug 28 01:37:01.334474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-VolumesGetTestJSON-2113097466 Aug 28 01:37:01.334474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:37:01.334474 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:37:01.336120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_9f22397792284cdc8601883d9b7f1f43", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/9f22397792284cdc8601883d9b7f1f43"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/9f22397792284cdc8601883d9b7f1f43"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/9f22397792284cdc8601883d9b7f1f43"}]}] Aug 28 01:37:01.336120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-VolumesGetTestJSON-2113097466 Aug 28 01:37:01.336120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 9f22397792284cdc8601883d9b7f1f43 Aug 28 01:37:01.336120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-VolumesGetTestJSON-2113097466 Aug 28 01:37:01.336120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-VolumesGetTestJSON-2113097466-project Aug 28 01:37:01.336120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:37:01.336120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:37:01.336120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: 382c06b725b54f09b976f8782eb0ee90 Aug 28 01:37:01.336120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-VolumesGetTestJSON-2113097466-project Aug 28 01:37:01.336120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: Aug 28 01:37:01.336120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Volume 38f957c5-86af-49f7-893e-1827e2f1d112 could not be found. Aug 28 01:37:01.336120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'id': '38f957c5-86af-49f7-893e-1827e2f1d112'} Aug 28 01:37:01.336120 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:37:01.337953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:37:01.337953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:37:01.337953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:37:01.337953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 28 01:37:01.337953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 28 01:37:01.337953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 28 01:37:01.337953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 28 01:37:01.337953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 28 01:37:01.337953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 28 01:37:01.337953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 28 01:37:01.337953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 28 01:37:01.337953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:37:01.337953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:37:01.337953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:37:01.337953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:37:01.337953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 38f957c5-86af-49f7-893e-1827e2f1d112 could not be found. (HTTP 404) (Request-ID: req-9a77a2d1-459b-4864-a3fe-62cef32983a0) Aug 28 01:37:01.337953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:37:01.337953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:37:01.337953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:37:01.339328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:37:01.339328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Aug 28 01:37:01.339328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Aug 28 01:37:01.339328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Aug 28 01:37:01.339328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Aug 28 01:37:01.339328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Aug 28 01:37:01.339328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Aug 28 01:37:01.339328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Aug 28 01:37:01.339328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Aug 28 01:37:01.339328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Aug 28 01:37:01.339328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Aug 28 01:37:01.339328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Aug 28 01:37:01.339328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi item = cinderclient( Aug 28 01:37:01.339328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Aug 28 01:37:01.339328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Aug 28 01:37:01.339328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Aug 28 01:37:01.339328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Aug 28 01:37:01.339328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Aug 28 01:37:01.339328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Aug 28 01:37:01.341148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Aug 28 01:37:01.341148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Aug 28 01:37:01.341148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Aug 28 01:37:01.341148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Aug 28 01:37:01.341148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 38f957c5-86af-49f7-893e-1827e2f1d112 could not be found. Aug 28 01:37:01.341148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:37:01.341148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:37:01.341148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:37:01.341148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:37:01.341148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:37:01.341148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:37:01.341148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:37:01.341148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:37:01.341148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Aug 28 01:37:01.341148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Aug 28 01:37:01.341148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:37:01.341148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:37:01.341148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Aug 28 01:37:01.341148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:37:01.342378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 38f957c5-86af-49f7-893e-1827e2f1d112 could not be found. Aug 28 01:37:01.342378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:37:01.342378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:37:01.342378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:37:01.342378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:37:01.342378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:37:01.342378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:37:01.342378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:37:01.342378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:37:01.342378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 38f957c5-86af-49f7-893e-1827e2f1d112 could not be found. Aug 28 01:37:01.342378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:37:01.345854 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ce430ef6-e7f6-4c06-be7b-40ab598c60ca tempest-VolumesGetTestJSON-2113097466 tempest-VolumesGetTestJSON-2113097466-project] Returning 404 to user: Volume 38f957c5-86af-49f7-893e-1827e2f1d112 could not be found. {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:37:01.347000 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ce430ef6-e7f6-4c06-be7b-40ab598c60ca tempest-VolumesGetTestJSON-2113097466 tempest-VolumesGetTestJSON-2113097466-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/38f957c5-86af-49f7-893e-1827e2f1d112" status: 404 len: 109 microversion: 2.1 time: 0.055974 Aug 28 01:37:01.347845 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1785/3343] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Aug 28 01:37:01 2022] GET /compute/v2.1/os-volumes/38f957c5-86af-49f7-893e-1827e2f1d112 => generated 109 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:37:01.735811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5fe8c9bf-5c78-44c8-b530-e842323fb333 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:01.736804 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5fe8c9bf-5c78-44c8-b530-e842323fb333 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:01.743208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5fe8c9bf-5c78-44c8-b530-e842323fb333 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:01.743684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5fe8c9bf-5c78-44c8-b530-e842323fb333 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:01.744238 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5fe8c9bf-5c78-44c8-b530-e842323fb333 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:01.744932 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5fe8c9bf-5c78-44c8-b530-e842323fb333 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:01.745399 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5fe8c9bf-5c78-44c8-b530-e842323fb333 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:01.746230 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5fe8c9bf-5c78-44c8-b530-e842323fb333 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:01.888649 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5fe8c9bf-5c78-44c8-b530-e842323fb333 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:01.896815 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5fe8c9bf-5c78-44c8-b530-e842323fb333 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1359 microversion: 2.1 time: 0.164947 Aug 28 01:37:01.897786 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1559/3344] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:01 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1359 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:02.249597 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-05346d70-c5a1-4ab2-a91f-06e658362db1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:02.250204 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-05346d70-c5a1-4ab2-a91f-06e658362db1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:02.257444 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05346d70-c5a1-4ab2-a91f-06e658362db1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:02.257928 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05346d70-c5a1-4ab2-a91f-06e658362db1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:02.258358 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05346d70-c5a1-4ab2-a91f-06e658362db1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:02.259055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05346d70-c5a1-4ab2-a91f-06e658362db1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:02.259611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05346d70-c5a1-4ab2-a91f-06e658362db1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:02.259961 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-05346d70-c5a1-4ab2-a91f-06e658362db1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:02.390992 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-05346d70-c5a1-4ab2-a91f-06e658362db1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:02.399125 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-05346d70-c5a1-4ab2-a91f-06e658362db1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1509 microversion: 2.1 time: 0.153410 Aug 28 01:37:02.399713 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1786/3345] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:02 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1509 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:02.917302 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ec95642c-d48c-4b4c-a48b-87aa4bca3488 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:02.917922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ec95642c-d48c-4b4c-a48b-87aa4bca3488 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:02.924439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ec95642c-d48c-4b4c-a48b-87aa4bca3488 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:02.924928 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ec95642c-d48c-4b4c-a48b-87aa4bca3488 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:02.925391 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ec95642c-d48c-4b4c-a48b-87aa4bca3488 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:02.926086 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ec95642c-d48c-4b4c-a48b-87aa4bca3488 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:02.926538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ec95642c-d48c-4b4c-a48b-87aa4bca3488 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:02.926953 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ec95642c-d48c-4b4c-a48b-87aa4bca3488 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:03.058623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ec95642c-d48c-4b4c-a48b-87aa4bca3488 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:03.066593 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ec95642c-d48c-4b4c-a48b-87aa4bca3488 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1359 microversion: 2.1 time: 0.153069 Aug 28 01:37:03.067175 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1560/3346] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:02 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1359 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:03.420029 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6cbffc84-5bf7-468f-b320-4e81823f8db4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:03.420640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6cbffc84-5bf7-468f-b320-4e81823f8db4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:03.427272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6cbffc84-5bf7-468f-b320-4e81823f8db4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:03.427762 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6cbffc84-5bf7-468f-b320-4e81823f8db4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:03.428194 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6cbffc84-5bf7-468f-b320-4e81823f8db4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:03.428891 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6cbffc84-5bf7-468f-b320-4e81823f8db4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:03.429391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6cbffc84-5bf7-468f-b320-4e81823f8db4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:03.430150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6cbffc84-5bf7-468f-b320-4e81823f8db4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:03.575035 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6cbffc84-5bf7-468f-b320-4e81823f8db4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:03.583411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6cbffc84-5bf7-468f-b320-4e81823f8db4 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1509 microversion: 2.1 time: 0.166884 Aug 28 01:37:03.583993 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1787/3347] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:03 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1509 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:04.087114 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3c871d0e-25f7-4667-9cd2-a4c2070624df tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:04.087735 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3c871d0e-25f7-4667-9cd2-a4c2070624df tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:04.094696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c871d0e-25f7-4667-9cd2-a4c2070624df tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:04.095214 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c871d0e-25f7-4667-9cd2-a4c2070624df tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:04.095656 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c871d0e-25f7-4667-9cd2-a4c2070624df tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:04.096385 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c871d0e-25f7-4667-9cd2-a4c2070624df tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:04.096814 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c871d0e-25f7-4667-9cd2-a4c2070624df tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:04.097281 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3c871d0e-25f7-4667-9cd2-a4c2070624df tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:04.297351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3c871d0e-25f7-4667-9cd2-a4c2070624df tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:04.306310 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3c871d0e-25f7-4667-9cd2-a4c2070624df tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1359 microversion: 2.1 time: 0.222840 Aug 28 01:37:04.306905 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1561/3348] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:04 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1359 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:37:04.606733 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0cd95ad1-86f1-47a3-b264-9b2de79db235 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:04.607325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0cd95ad1-86f1-47a3-b264-9b2de79db235 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:04.613880 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0cd95ad1-86f1-47a3-b264-9b2de79db235 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:04.614353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0cd95ad1-86f1-47a3-b264-9b2de79db235 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:04.614780 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0cd95ad1-86f1-47a3-b264-9b2de79db235 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:04.615441 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0cd95ad1-86f1-47a3-b264-9b2de79db235 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:04.615881 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0cd95ad1-86f1-47a3-b264-9b2de79db235 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:04.616294 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0cd95ad1-86f1-47a3-b264-9b2de79db235 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:04.752309 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0cd95ad1-86f1-47a3-b264-9b2de79db235 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:04.760611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0cd95ad1-86f1-47a3-b264-9b2de79db235 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1509 microversion: 2.1 time: 0.157725 Aug 28 01:37:04.761297 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1788/3349] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:04 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1509 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:05.327415 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ce9b8d80-30e4-4649-8020-38a403b9014d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:05.328027 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ce9b8d80-30e4-4649-8020-38a403b9014d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:05.334856 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce9b8d80-30e4-4649-8020-38a403b9014d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:05.335540 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce9b8d80-30e4-4649-8020-38a403b9014d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:05.335979 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce9b8d80-30e4-4649-8020-38a403b9014d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:05.336674 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce9b8d80-30e4-4649-8020-38a403b9014d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:05.337154 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce9b8d80-30e4-4649-8020-38a403b9014d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:05.337573 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ce9b8d80-30e4-4649-8020-38a403b9014d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:05.477539 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ce9b8d80-30e4-4649-8020-38a403b9014d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:05.487944 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ce9b8d80-30e4-4649-8020-38a403b9014d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1359 microversion: 2.1 time: 0.162318 Aug 28 01:37:05.487944 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1562/3350] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:05 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1359 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:05.782758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5ae68d24-2292-4f1b-94c1-be3ebdfdc71a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:05.783479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5ae68d24-2292-4f1b-94c1-be3ebdfdc71a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:05.791168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ae68d24-2292-4f1b-94c1-be3ebdfdc71a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:05.791760 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ae68d24-2292-4f1b-94c1-be3ebdfdc71a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:05.792183 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ae68d24-2292-4f1b-94c1-be3ebdfdc71a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:05.793053 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ae68d24-2292-4f1b-94c1-be3ebdfdc71a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:05.793527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ae68d24-2292-4f1b-94c1-be3ebdfdc71a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:05.793945 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ae68d24-2292-4f1b-94c1-be3ebdfdc71a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:05.934436 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-5ae68d24-2292-4f1b-94c1-be3ebdfdc71a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:05.942599 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5ae68d24-2292-4f1b-94c1-be3ebdfdc71a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1509 microversion: 2.1 time: 0.164650 Aug 28 01:37:05.943313 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1789/3351] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:05 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1509 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:06.506623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-4b2cf9c7-f69c-4add-8f99-f5ef3ecc4b73 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:06.507231 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-4b2cf9c7-f69c-4add-8f99-f5ef3ecc4b73 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:06.513823 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b2cf9c7-f69c-4add-8f99-f5ef3ecc4b73 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:06.514311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b2cf9c7-f69c-4add-8f99-f5ef3ecc4b73 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:06.514743 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b2cf9c7-f69c-4add-8f99-f5ef3ecc4b73 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:06.515447 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b2cf9c7-f69c-4add-8f99-f5ef3ecc4b73 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:06.515872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b2cf9c7-f69c-4add-8f99-f5ef3ecc4b73 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:06.516303 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-4b2cf9c7-f69c-4add-8f99-f5ef3ecc4b73 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:06.659232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-4b2cf9c7-f69c-4add-8f99-f5ef3ecc4b73 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:06.667381 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-4b2cf9c7-f69c-4add-8f99-f5ef3ecc4b73 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1359 microversion: 2.1 time: 0.164630 Aug 28 01:37:06.667959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1563/3352] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:06 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1359 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:06.964704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d5d38c15-6c58-49e6-94cf-a4a64780545a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:06.965552 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d5d38c15-6c58-49e6-94cf-a4a64780545a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:06.972239 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5d38c15-6c58-49e6-94cf-a4a64780545a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:06.972802 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5d38c15-6c58-49e6-94cf-a4a64780545a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:06.973265 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5d38c15-6c58-49e6-94cf-a4a64780545a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:06.973957 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5d38c15-6c58-49e6-94cf-a4a64780545a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:06.974403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5d38c15-6c58-49e6-94cf-a4a64780545a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:06.974824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d5d38c15-6c58-49e6-94cf-a4a64780545a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:07.113440 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d5d38c15-6c58-49e6-94cf-a4a64780545a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:07.129965 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d5d38c15-6c58-49e6-94cf-a4a64780545a tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1509 microversion: 2.1 time: 0.168373 Aug 28 01:37:07.130772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1790/3353] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:06 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1509 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:07.689509 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e3f9ba02-a921-4100-9a02-425784f23542 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:07.690296 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e3f9ba02-a921-4100-9a02-425784f23542 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:07.699305 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3f9ba02-a921-4100-9a02-425784f23542 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:07.700009 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3f9ba02-a921-4100-9a02-425784f23542 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:07.700790 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3f9ba02-a921-4100-9a02-425784f23542 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:07.701872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3f9ba02-a921-4100-9a02-425784f23542 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:07.702472 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3f9ba02-a921-4100-9a02-425784f23542 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:07.703060 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e3f9ba02-a921-4100-9a02-425784f23542 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:07.958931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-e3f9ba02-a921-4100-9a02-425784f23542 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:07.970543 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e3f9ba02-a921-4100-9a02-425784f23542 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1359 microversion: 2.1 time: 0.284676 Aug 28 01:37:07.971388 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1564/3354] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:07 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1359 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:08.069337 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-3e4d223d-86d1-4c97-a3c0-d3eff08af99d req-cf5f061c-9c53-4d9e-88dc-5d6b9115b1cb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5aa21554-b4b9-49b1-9fd4-d802097e1a1b", "tag": "e12ff0e5-7520-4d3f-b645-8686e263d5b8"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:37:08.082603 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-3e4d223d-86d1-4c97-a3c0-d3eff08af99d req-cf5f061c-9c53-4d9e-88dc-5d6b9115b1cb service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:08.083238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-3e4d223d-86d1-4c97-a3c0-d3eff08af99d req-cf5f061c-9c53-4d9e-88dc-5d6b9115b1cb service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:08.083736 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-3e4d223d-86d1-4c97-a3c0-d3eff08af99d req-cf5f061c-9c53-4d9e-88dc-5d6b9115b1cb service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:08.099811 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-3e4d223d-86d1-4c97-a3c0-d3eff08af99d req-cf5f061c-9c53-4d9e-88dc-5d6b9115b1cb service nova] Creating event network-changed:e12ff0e5-7520-4d3f-b645-8686e263d5b8 for instance 5aa21554-b4b9-49b1-9fd4-d802097e1a1b on hv-cinder-maste Aug 28 01:37:08.104893 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-3e4d223d-86d1-4c97-a3c0-d3eff08af99d req-cf5f061c-9c53-4d9e-88dc-5d6b9115b1cb service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045737 Aug 28 01:37:08.105924 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1791/3355] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:37:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:37:08.151793 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-151cb6a3-be86-4f64-b8f2-14df64501ef0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:08.152415 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-151cb6a3-be86-4f64-b8f2-14df64501ef0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:08.158517 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-151cb6a3-be86-4f64-b8f2-14df64501ef0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:08.158991 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-151cb6a3-be86-4f64-b8f2-14df64501ef0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:08.159521 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-151cb6a3-be86-4f64-b8f2-14df64501ef0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:08.160052 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-151cb6a3-be86-4f64-b8f2-14df64501ef0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:08.160495 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-151cb6a3-be86-4f64-b8f2-14df64501ef0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:08.161295 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-151cb6a3-be86-4f64-b8f2-14df64501ef0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:08.300013 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-151cb6a3-be86-4f64-b8f2-14df64501ef0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:08.307987 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-151cb6a3-be86-4f64-b8f2-14df64501ef0 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1509 microversion: 2.1 time: 0.160002 Aug 28 01:37:08.308564 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1565/3356] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:08 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1509 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:08.993638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-08c7505a-5a91-4c87-b455-7ecbf74b40e8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:08.995402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-08c7505a-5a91-4c87-b455-7ecbf74b40e8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:09.004140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08c7505a-5a91-4c87-b455-7ecbf74b40e8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:09.004781 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08c7505a-5a91-4c87-b455-7ecbf74b40e8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:09.005372 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08c7505a-5a91-4c87-b455-7ecbf74b40e8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:09.006188 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08c7505a-5a91-4c87-b455-7ecbf74b40e8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:09.006770 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08c7505a-5a91-4c87-b455-7ecbf74b40e8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:09.007295 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-08c7505a-5a91-4c87-b455-7ecbf74b40e8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:09.012274 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:37:09.330187 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1844c071-362b-4c68-8516-67556c59285d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:09.331575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1844c071-362b-4c68-8516-67556c59285d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:09.338885 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1844c071-362b-4c68-8516-67556c59285d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:09.339456 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1844c071-362b-4c68-8516-67556c59285d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:09.339959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1844c071-362b-4c68-8516-67556c59285d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:09.340721 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1844c071-362b-4c68-8516-67556c59285d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:09.341439 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1844c071-362b-4c68-8516-67556c59285d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:09.341927 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1844c071-362b-4c68-8516-67556c59285d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:09.501618 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-08c7505a-5a91-4c87-b455-7ecbf74b40e8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:09.514190 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-08c7505a-5a91-4c87-b455-7ecbf74b40e8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1359 microversion: 2.1 time: 0.524750 Aug 28 01:37:09.514779 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1792/3357] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:08 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1359 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:09.584100 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1844c071-362b-4c68-8516-67556c59285d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:09.587505 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=100 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:37:09.616178 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1844c071-362b-4c68-8516-67556c59285d tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1509 microversion: 2.1 time: 0.289919 Aug 28 01:37:09.618488 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1566/3358] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:09 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1509 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:37:10.119803 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-1ab9faa6-cf07-4fc6-bd06-ee8073da0ba2 req-36a626f3-aefe-48ed-9fcf-bf7c605ef195 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "460a7822-a544-4b66-9361-79ac7d118804", "name": "network-vif-deleted", "tag": "b01cc365-e4c3-4f15-b660-e7eb82f6fcf4"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:37:10.130148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-1ab9faa6-cf07-4fc6-bd06-ee8073da0ba2 req-36a626f3-aefe-48ed-9fcf-bf7c605ef195 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:10.130622 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-1ab9faa6-cf07-4fc6-bd06-ee8073da0ba2 req-36a626f3-aefe-48ed-9fcf-bf7c605ef195 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:10.131047 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-1ab9faa6-cf07-4fc6-bd06-ee8073da0ba2 req-36a626f3-aefe-48ed-9fcf-bf7c605ef195 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:10.142638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-1ab9faa6-cf07-4fc6-bd06-ee8073da0ba2 req-36a626f3-aefe-48ed-9fcf-bf7c605ef195 service nova] Creating event network-vif-deleted:b01cc365-e4c3-4f15-b660-e7eb82f6fcf4 for instance 460a7822-a544-4b66-9361-79ac7d118804 on hv-cinder-maste Aug 28 01:37:10.147827 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-1ab9faa6-cf07-4fc6-bd06-ee8073da0ba2 req-36a626f3-aefe-48ed-9fcf-bf7c605ef195 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031790 Aug 28 01:37:10.148411 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1793/3359] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:37:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:37:10.535468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-785f3e5b-4a7a-4c7c-83d0-74f6345bb2fc tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:10.536068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-785f3e5b-4a7a-4c7c-83d0-74f6345bb2fc tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:10.542613 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-785f3e5b-4a7a-4c7c-83d0-74f6345bb2fc tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:10.543110 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-785f3e5b-4a7a-4c7c-83d0-74f6345bb2fc tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:10.543515 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-785f3e5b-4a7a-4c7c-83d0-74f6345bb2fc tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:10.544189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-785f3e5b-4a7a-4c7c-83d0-74f6345bb2fc tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:10.544635 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-785f3e5b-4a7a-4c7c-83d0-74f6345bb2fc tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:10.545072 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-785f3e5b-4a7a-4c7c-83d0-74f6345bb2fc tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:10.640107 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-319c0425-9abf-49d4-9260-3eaddf5aeace tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:10.640788 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-319c0425-9abf-49d4-9260-3eaddf5aeace tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:10.648032 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-319c0425-9abf-49d4-9260-3eaddf5aeace tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:10.648516 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-319c0425-9abf-49d4-9260-3eaddf5aeace tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:10.648949 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-319c0425-9abf-49d4-9260-3eaddf5aeace tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:10.649670 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-319c0425-9abf-49d4-9260-3eaddf5aeace tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:10.650095 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-319c0425-9abf-49d4-9260-3eaddf5aeace tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:10.650533 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-319c0425-9abf-49d4-9260-3eaddf5aeace tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:10.677227 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-785f3e5b-4a7a-4c7c-83d0-74f6345bb2fc tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:10.685731 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-785f3e5b-4a7a-4c7c-83d0-74f6345bb2fc tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1359 microversion: 2.1 time: 0.154274 Aug 28 01:37:10.686283 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1567/3360] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:10 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1359 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:10.716247 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-319c0425-9abf-49d4-9260-3eaddf5aeace tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:10.725163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-319c0425-9abf-49d4-9260-3eaddf5aeace tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1467 microversion: 2.1 time: 0.089273 Aug 28 01:37:10.725771 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1794/3361] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:10 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1467 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:11.706380 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-577aecaf-70a9-4c52-8448-12b8f228ec7e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:11.707020 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-577aecaf-70a9-4c52-8448-12b8f228ec7e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:11.714053 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-577aecaf-70a9-4c52-8448-12b8f228ec7e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:11.714535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-577aecaf-70a9-4c52-8448-12b8f228ec7e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:11.714959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-577aecaf-70a9-4c52-8448-12b8f228ec7e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:11.715654 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-577aecaf-70a9-4c52-8448-12b8f228ec7e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:11.716094 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-577aecaf-70a9-4c52-8448-12b8f228ec7e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:11.716511 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-577aecaf-70a9-4c52-8448-12b8f228ec7e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:11.746279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-0de37ac9-7319-4682-ab8e-79335ca6b9e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:11.746916 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-0de37ac9-7319-4682-ab8e-79335ca6b9e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:11.754556 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0de37ac9-7319-4682-ab8e-79335ca6b9e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:11.755053 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0de37ac9-7319-4682-ab8e-79335ca6b9e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:11.755480 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0de37ac9-7319-4682-ab8e-79335ca6b9e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:11.756184 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0de37ac9-7319-4682-ab8e-79335ca6b9e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:11.756678 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0de37ac9-7319-4682-ab8e-79335ca6b9e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:11.757167 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-0de37ac9-7319-4682-ab8e-79335ca6b9e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:11.840489 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-0de37ac9-7319-4682-ab8e-79335ca6b9e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:11.850451 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-0de37ac9-7319-4682-ab8e-79335ca6b9e8 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 200 len: 1467 microversion: 2.1 time: 0.108033 Aug 28 01:37:11.851004 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1795/3362] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:11 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 1467 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:11.912248 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-577aecaf-70a9-4c52-8448-12b8f228ec7e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:11.922048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-577aecaf-70a9-4c52-8448-12b8f228ec7e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1359 microversion: 2.1 time: 0.219534 Aug 28 01:37:11.922684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1568/3363] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:11 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1359 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:12.872128 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5ea986e3-8f97-451e-9e03-3c9b3fea53b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:12.872735 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-5ea986e3-8f97-451e-9e03-3c9b3fea53b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: 460a7822-a544-4b66-9361-79ac7d118804] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:12.879774 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ea986e3-8f97-451e-9e03-3c9b3fea53b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:12.880241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ea986e3-8f97-451e-9e03-3c9b3fea53b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:12.880693 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ea986e3-8f97-451e-9e03-3c9b3fea53b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:12.881404 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ea986e3-8f97-451e-9e03-3c9b3fea53b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:12.881855 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ea986e3-8f97-451e-9e03-3c9b3fea53b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:12.882272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-5ea986e3-8f97-451e-9e03-3c9b3fea53b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:12.897264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-5ea986e3-8f97-451e-9e03-3c9b3fea53b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] HTTP exception thrown: Instance 460a7822-a544-4b66-9361-79ac7d118804 could not be found. Aug 28 01:37:12.898952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5ea986e3-8f97-451e-9e03-3c9b3fea53b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Request method failure captured: Aug 28 01:37:12.898952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 HTTP/1.1 Aug 28 01:37:12.898952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:37:12.898952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:37:12.898952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:37:12.898952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:37:12.898952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:37:12.898952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:37:12.898952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:37:12.898952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:37:12.898952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCrzWcuiPK_0ynCrragXxXd_fRw7lHCmbNVMAqr4NhyopxGAZIiQrt0m3y3NawnXLWqyOVM4YbBc-yLRrJ4cXICU0MsT01bsu6VzxaCxGd6JRCChv8LlCV7kFwwyy7Ffwd9JpYuFLguuT-xhEcVlfMIZxgSYB22XTBbtvAeHM_AtK1Xg Aug 28 01:37:12.898952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:37:12.898952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:37:12.898952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:37:12.898952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:37:12.898952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:37:12.898952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:37:12.898952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 01:37:12.898952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:37:12.898952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:37:12.898952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:37:12.900397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_6f3acee4af62483ab1f2cbf3a47b09e1", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/6f3acee4af62483ab1f2cbf3a47b09e1"}]}] Aug 28 01:37:12.900397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:37:12.900397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 01:37:12.900397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:37:12.900397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 01:37:12.900397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:37:12.900397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:37:12.900397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: 1ab2fcb3759045c7a076e5c8222a421d Aug 28 01:37:12.900397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 01:37:12.900397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 01:37:12.900397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Instance 460a7822-a544-4b66-9361-79ac7d118804 could not be found. Aug 28 01:37:12.900397 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'id': '460a7822-a544-4b66-9361-79ac7d118804'} Aug 28 01:37:12.901275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:37:12.901275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:37:12.901275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:37:12.901275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:37:12.901275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:37:12.901275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:37:12.901275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:37:12.901275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:37:12.901275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:37:12.901275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:37:12.901275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 460a7822-a544-4b66-9361-79ac7d118804 could not be found. Aug 28 01:37:12.901275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:37:12.901275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:37:12.901275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:37:12.901275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:37:12.901275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:37:12.901275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:37:12.901275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:37:12.901275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:37:12.901275 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:37:12.902692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:37:12.902692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 01:37:12.902692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 01:37:12.902692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:37:12.902692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:37:12.902692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:37:12.902692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:37:12.902692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 460a7822-a544-4b66-9361-79ac7d118804 could not be found. Aug 28 01:37:12.902692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:37:12.902692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:37:12.902692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:37:12.902692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:37:12.902692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:37:12.902692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:37:12.902692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:37:12.902692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:37:12.902692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 460a7822-a544-4b66-9361-79ac7d118804 could not be found. Aug 28 01:37:12.902692 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:37:12.904172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-5ea986e3-8f97-451e-9e03-3c9b3fea53b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Returning 404 to user: Instance 460a7822-a544-4b66-9361-79ac7d118804 could not be found. {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:37:12.904172 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-5ea986e3-8f97-451e-9e03-3c9b3fea53b5 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804" status: 404 len: 111 microversion: 2.1 time: 0.032220 Aug 28 01:37:12.904338 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1796/3364] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:12 2022] GET /compute/v2.1/servers/460a7822-a544-4b66-9361-79ac7d118804 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:37:12.910696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b6055e68-2a24-454a-b25a-b0a0961ae1f6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:12.911437 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b6055e68-2a24-454a-b25a-b0a0961ae1f6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:12.919687 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6055e68-2a24-454a-b25a-b0a0961ae1f6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:12.920442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6055e68-2a24-454a-b25a-b0a0961ae1f6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:12.920920 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6055e68-2a24-454a-b25a-b0a0961ae1f6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:12.921949 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6055e68-2a24-454a-b25a-b0a0961ae1f6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:12.922453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6055e68-2a24-454a-b25a-b0a0961ae1f6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:12.923055 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6055e68-2a24-454a-b25a-b0a0961ae1f6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:12.944402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ce41dac3-4358-4b5f-bf2e-08052fedb567 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:12.945163 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ce41dac3-4358-4b5f-bf2e-08052fedb567 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:12.951808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce41dac3-4358-4b5f-bf2e-08052fedb567 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:12.952398 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b6055e68-2a24-454a-b25a-b0a0961ae1f6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Going to try to terminate instance {{(pid=120524) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 28 01:37:12.952813 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce41dac3-4358-4b5f-bf2e-08052fedb567 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:12.953283 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce41dac3-4358-4b5f-bf2e-08052fedb567 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:12.954154 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce41dac3-4358-4b5f-bf2e-08052fedb567 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:12.954431 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce41dac3-4358-4b5f-bf2e-08052fedb567 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:12.954850 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ce41dac3-4358-4b5f-bf2e-08052fedb567 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:13.046623 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b6055e68-2a24-454a-b25a-b0a0961ae1f6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "DELETE /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 204 len: 0 microversion: 2.1 time: 0.139050 Aug 28 01:37:13.048030 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1569/3365] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 01:37:12 2022] DELETE /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 28 01:37:13.064521 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-dded7ca7-d8e1-4fb1-87d8-9464f87ad9c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:13.065275 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-dded7ca7-d8e1-4fb1-87d8-9464f87ad9c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:13.072195 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dded7ca7-d8e1-4fb1-87d8-9464f87ad9c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:13.072747 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dded7ca7-d8e1-4fb1-87d8-9464f87ad9c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:13.073198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dded7ca7-d8e1-4fb1-87d8-9464f87ad9c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:13.073899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dded7ca7-d8e1-4fb1-87d8-9464f87ad9c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:13.074363 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dded7ca7-d8e1-4fb1-87d8-9464f87ad9c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:13.074749 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dded7ca7-d8e1-4fb1-87d8-9464f87ad9c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:13.102488 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ce41dac3-4358-4b5f-bf2e-08052fedb567 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:13.111389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ce41dac3-4358-4b5f-bf2e-08052fedb567 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1359 microversion: 2.1 time: 0.171713 Aug 28 01:37:13.125311 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1797/3366] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:12 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1359 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:13.214307 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-dded7ca7-d8e1-4fb1-87d8-9464f87ad9c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:13.224157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-dded7ca7-d8e1-4fb1-87d8-9464f87ad9c2 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1521 microversion: 2.1 time: 0.171236 Aug 28 01:37:13.224730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1570/3367] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:13 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1521 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:14.132487 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c600f48f-d9e7-4b0e-a712-7e0ca679a357 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:14.133885 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c600f48f-d9e7-4b0e-a712-7e0ca679a357 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:14.141794 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c600f48f-d9e7-4b0e-a712-7e0ca679a357 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:14.142768 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c600f48f-d9e7-4b0e-a712-7e0ca679a357 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:14.143430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c600f48f-d9e7-4b0e-a712-7e0ca679a357 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:14.144906 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c600f48f-d9e7-4b0e-a712-7e0ca679a357 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:14.144906 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c600f48f-d9e7-4b0e-a712-7e0ca679a357 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:14.145351 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c600f48f-d9e7-4b0e-a712-7e0ca679a357 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:14.248156 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d8e11d3d-03c6-42e3-8657-c0c227f9b5d6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:14.249598 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d8e11d3d-03c6-42e3-8657-c0c227f9b5d6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:14.258481 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d8e11d3d-03c6-42e3-8657-c0c227f9b5d6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:14.259277 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d8e11d3d-03c6-42e3-8657-c0c227f9b5d6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:14.259924 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d8e11d3d-03c6-42e3-8657-c0c227f9b5d6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:14.261041 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d8e11d3d-03c6-42e3-8657-c0c227f9b5d6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:14.261703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d8e11d3d-03c6-42e3-8657-c0c227f9b5d6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:14.262351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d8e11d3d-03c6-42e3-8657-c0c227f9b5d6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:14.481268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c600f48f-d9e7-4b0e-a712-7e0ca679a357 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:14.493582 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c600f48f-d9e7-4b0e-a712-7e0ca679a357 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1359 microversion: 2.1 time: 0.364927 Aug 28 01:37:14.506955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1798/3368] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:14 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1359 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:14.550947 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d8e11d3d-03c6-42e3-8657-c0c227f9b5d6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1ab2fcb3759045c7a076e5c8222a421d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6f3acee4af62483ab1f2cbf3a47b09e1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:14.561414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d8e11d3d-03c6-42e3-8657-c0c227f9b5d6 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 200 len: 1521 microversion: 2.1 time: 0.318781 Aug 28 01:37:14.562011 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1571/3369] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:14 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 1521 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:37:14.626122 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-f033c6a1-203a-4450-a145-56d090e27487 req-ba637884-7695-45fe-99a9-49c3ab9707bf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f5f7fe76-9095-4d8d-967f-be0009648de9", "name": "network-vif-deleted", "tag": "ff741c2a-17ad-4c15-a6a0-687d9dfcad8b"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:37:14.638587 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-f033c6a1-203a-4450-a145-56d090e27487 req-ba637884-7695-45fe-99a9-49c3ab9707bf service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:14.639150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-f033c6a1-203a-4450-a145-56d090e27487 req-ba637884-7695-45fe-99a9-49c3ab9707bf service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:14.639650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-f033c6a1-203a-4450-a145-56d090e27487 req-ba637884-7695-45fe-99a9-49c3ab9707bf service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:14.652222 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-f033c6a1-203a-4450-a145-56d090e27487 req-ba637884-7695-45fe-99a9-49c3ab9707bf service nova] Creating event network-vif-deleted:ff741c2a-17ad-4c15-a6a0-687d9dfcad8b for instance f5f7fe76-9095-4d8d-967f-be0009648de9 on hv-cinder-maste Aug 28 01:37:14.658761 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-f033c6a1-203a-4450-a145-56d090e27487 req-ba637884-7695-45fe-99a9-49c3ab9707bf service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038442 Aug 28 01:37:14.659667 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1799/3370] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:37:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:37:15.514826 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d6aa8f87-d183-4bf6-9143-5b18aa203289 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:15.515376 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d6aa8f87-d183-4bf6-9143-5b18aa203289 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:15.522015 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6aa8f87-d183-4bf6-9143-5b18aa203289 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:15.522510 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6aa8f87-d183-4bf6-9143-5b18aa203289 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:15.522927 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6aa8f87-d183-4bf6-9143-5b18aa203289 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:15.523607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6aa8f87-d183-4bf6-9143-5b18aa203289 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:15.524048 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6aa8f87-d183-4bf6-9143-5b18aa203289 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:15.524453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d6aa8f87-d183-4bf6-9143-5b18aa203289 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:15.584250 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-85dc1f78-467d-4034-8622-ceba7930e0e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:15.585112 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-85dc1f78-467d-4034-8622-ceba7930e0e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] [instance: f5f7fe76-9095-4d8d-967f-be0009648de9] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:15.594269 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85dc1f78-467d-4034-8622-ceba7930e0e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:15.595779 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85dc1f78-467d-4034-8622-ceba7930e0e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:15.596395 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85dc1f78-467d-4034-8622-ceba7930e0e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:15.597294 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85dc1f78-467d-4034-8622-ceba7930e0e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:15.598533 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85dc1f78-467d-4034-8622-ceba7930e0e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:15.599124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-85dc1f78-467d-4034-8622-ceba7930e0e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:15.614376 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.wsgi [None req-85dc1f78-467d-4034-8622-ceba7930e0e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] HTTP exception thrown: Instance f5f7fe76-9095-4d8d-967f-be0009648de9 could not be found. Aug 28 01:37:15.616048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-85dc1f78-467d-4034-8622-ceba7930e0e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Request method failure captured: Aug 28 01:37:15.616048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: request: GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 HTTP/1.1 Aug 28 01:37:15.616048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept: application/json Aug 28 01:37:15.616048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Accept-Encoding: identity Aug 28 01:37:15.616048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Connection: close Aug 28 01:37:15.616048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Length: 0 Aug 28 01:37:15.616048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Content-Type: application/json Aug 28 01:37:15.616048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Host: 10.222.0.42 Aug 28 01:37:15.616048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Openstack-System-Scope: None Aug 28 01:37:15.616048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: User-Agent: python-urllib3/1.26.11 Aug 28 01:37:15.616048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Auth-Token: gAAAAABjCrzWcuiPK_0ynCrragXxXd_fRw7lHCmbNVMAqr4NhyopxGAZIiQrt0m3y3NawnXLWqyOVM4YbBc-yLRrJ4cXICU0MsT01bsu6VzxaCxGd6JRCChv8LlCV7kFwwyy7Ffwd9JpYuFLguuT-xhEcVlfMIZxgSYB22XTBbtvAeHM_AtK1Xg Aug 28 01:37:15.616048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Id: None Aug 28 01:37:15.616048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Domain-Name: None Aug 28 01:37:15.616048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Identity-Status: Confirmed Aug 28 01:37:15.616048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Is-Admin-Project: True Aug 28 01:37:15.616048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Id: default Aug 28 01:37:15.616048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Domain-Name: Default Aug 28 01:37:15.616048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 01:37:15.616048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:37:15.616048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Role: member,reader Aug 28 01:37:15.616048 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Roles: member,reader Aug 28 01:37:15.617300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_6f3acee4af62483ab1f2cbf3a47b09e1", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6f3acee4af62483ab1f2cbf3a47b09e1"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/6f3acee4af62483ab1f2cbf3a47b09e1"}]}] Aug 28 01:37:15.617300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:37:15.617300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Id: 6f3acee4af62483ab1f2cbf3a47b09e1 Aug 28 01:37:15.617300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-859719664 Aug 28 01:37:15.617300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 01:37:15.617300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Id: default Aug 28 01:37:15.617300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Domain-Name: Default Aug 28 01:37:15.617300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Id: 1ab2fcb3759045c7a076e5c8222a421d Aug 28 01:37:15.617300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: X-User-Name: tempest-ServerRescueNegativeTestJSON-859719664-project Aug 28 01:37:15.617300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: method: > Aug 28 01:37:15.617300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: exception: Instance f5f7fe76-9095-4d8d-967f-be0009648de9 could not be found. Aug 28 01:37:15.617300 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: action_args: {'id': 'f5f7fe76-9095-4d8d-967f-be0009648de9'} Aug 28 01:37:15.617954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:37:15.617954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:37:15.617954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:37:15.617954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:37:15.617954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:37:15.617954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:37:15.617954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:37:15.617954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:37:15.617954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:37:15.617954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:37:15.617954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f5f7fe76-9095-4d8d-967f-be0009648de9 could not be found. Aug 28 01:37:15.617954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:37:15.617954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:37:15.617954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:37:15.617954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:37:15.617954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:37:15.617954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:37:15.617954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:37:15.617954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:37:15.617954 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:37:15.619033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:37:15.619033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 01:37:15.619033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 01:37:15.619033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:37:15.619033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:37:15.619033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:37:15.619033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:37:15.619033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f5f7fe76-9095-4d8d-967f-be0009648de9 could not be found. Aug 28 01:37:15.619033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:37:15.619033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:37:15.619033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:37:15.619033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:37:15.619033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:37:15.619033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:37:15.619033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:37:15.619033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:37:15.619033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f5f7fe76-9095-4d8d-967f-be0009648de9 could not be found. Aug 28 01:37:15.619033 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR nova.api.openstack.wsgi Aug 28 01:37:15.619989 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-85dc1f78-467d-4034-8622-ceba7930e0e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] Returning 404 to user: Instance f5f7fe76-9095-4d8d-967f-be0009648de9 could not be found. {{(pid=120525) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:37:15.619989 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-85dc1f78-467d-4034-8622-ceba7930e0e1 tempest-ServerRescueNegativeTestJSON-859719664 tempest-ServerRescueNegativeTestJSON-859719664-project] 10.222.0.42 "GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9" status: 404 len: 111 microversion: 2.1 time: 0.039291 Aug 28 01:37:15.623223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1800/3371] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:15 2022] GET /compute/v2.1/servers/f5f7fe76-9095-4d8d-967f-be0009648de9 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:37:15.668640 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d6aa8f87-d183-4bf6-9143-5b18aa203289 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:15.680008 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d6aa8f87-d183-4bf6-9143-5b18aa203289 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1359 microversion: 2.1 time: 0.168364 Aug 28 01:37:15.689795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1572/3372] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:15 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1359 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:16.412947 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-1a662cea-bdfc-458a-85a0-0cc841088734 req-bb3f4a7a-78b5-42d7-88b6-b66ed867796f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5aa21554-b4b9-49b1-9fd4-d802097e1a1b", "name": "network-vif-plugged", "status": "completed", "tag": "e12ff0e5-7520-4d3f-b645-8686e263d5b8"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:37:16.423326 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-1a662cea-bdfc-458a-85a0-0cc841088734 req-bb3f4a7a-78b5-42d7-88b6-b66ed867796f service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:16.424370 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-1a662cea-bdfc-458a-85a0-0cc841088734 req-bb3f4a7a-78b5-42d7-88b6-b66ed867796f service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:16.426178 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-1a662cea-bdfc-458a-85a0-0cc841088734 req-bb3f4a7a-78b5-42d7-88b6-b66ed867796f service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:16.444055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-1a662cea-bdfc-458a-85a0-0cc841088734 req-bb3f4a7a-78b5-42d7-88b6-b66ed867796f service nova] Creating event network-vif-plugged:e12ff0e5-7520-4d3f-b645-8686e263d5b8 for instance 5aa21554-b4b9-49b1-9fd4-d802097e1a1b on hv-cinder-maste Aug 28 01:37:16.452058 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-1a662cea-bdfc-458a-85a0-0cc841088734 req-bb3f4a7a-78b5-42d7-88b6-b66ed867796f service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042374 Aug 28 01:37:16.453016 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1801/3373] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:37:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:37:16.702426 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-74478b77-c415-4a87-a583-af63fe270133 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:16.703754 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-74478b77-c415-4a87-a583-af63fe270133 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:16.716248 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-74478b77-c415-4a87-a583-af63fe270133 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:16.716749 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-74478b77-c415-4a87-a583-af63fe270133 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:16.717200 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-74478b77-c415-4a87-a583-af63fe270133 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:16.717910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-74478b77-c415-4a87-a583-af63fe270133 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:16.718353 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-74478b77-c415-4a87-a583-af63fe270133 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:16.719374 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-74478b77-c415-4a87-a583-af63fe270133 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:16.880774 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-74478b77-c415-4a87-a583-af63fe270133 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:16.888852 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-74478b77-c415-4a87-a583-af63fe270133 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1359 microversion: 2.1 time: 0.190618 Aug 28 01:37:16.889610 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1573/3374] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:16 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1359 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:17.910961 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c12151cc-15eb-4e7b-b15f-44b27c488518 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:17.911846 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c12151cc-15eb-4e7b-b15f-44b27c488518 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:17.918765 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c12151cc-15eb-4e7b-b15f-44b27c488518 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:17.919249 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c12151cc-15eb-4e7b-b15f-44b27c488518 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:17.919708 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c12151cc-15eb-4e7b-b15f-44b27c488518 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:17.920405 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c12151cc-15eb-4e7b-b15f-44b27c488518 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:17.920851 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c12151cc-15eb-4e7b-b15f-44b27c488518 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:17.921415 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c12151cc-15eb-4e7b-b15f-44b27c488518 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:18.058961 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c12151cc-15eb-4e7b-b15f-44b27c488518 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:18.067429 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c12151cc-15eb-4e7b-b15f-44b27c488518 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1359 microversion: 2.1 time: 0.161184 Aug 28 01:37:18.068059 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1802/3375] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:17 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1359 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:37:19.091067 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b6af95d6-d8a5-46dd-8d6c-7396ed8546c7 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:19.092770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b6af95d6-d8a5-46dd-8d6c-7396ed8546c7 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:19.100621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=69,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:37:19.105940 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6af95d6-d8a5-46dd-8d6c-7396ed8546c7 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:19.106604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6af95d6-d8a5-46dd-8d6c-7396ed8546c7 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:19.108485 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6af95d6-d8a5-46dd-8d6c-7396ed8546c7 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:19.110204 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6af95d6-d8a5-46dd-8d6c-7396ed8546c7 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:19.111114 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6af95d6-d8a5-46dd-8d6c-7396ed8546c7 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:19.112338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6af95d6-d8a5-46dd-8d6c-7396ed8546c7 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:19.282408 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b6af95d6-d8a5-46dd-8d6c-7396ed8546c7 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:19.290269 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b6af95d6-d8a5-46dd-8d6c-7396ed8546c7 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1550 microversion: 2.1 time: 0.204846 Aug 28 01:37:19.290837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1574/3376] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:19 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1550 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:19.311527 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-084f77a6-c966-465d-919b-61a86e083492 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:19.312156 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-084f77a6-c966-465d-919b-61a86e083492 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:19.315232 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=75,nova_api:INSERT=1,nova_api:UPDATE=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:37:19.320712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-084f77a6-c966-465d-919b-61a86e083492 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:19.321224 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-084f77a6-c966-465d-919b-61a86e083492 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:19.321657 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-084f77a6-c966-465d-919b-61a86e083492 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:19.322354 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-084f77a6-c966-465d-919b-61a86e083492 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:19.322805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-084f77a6-c966-465d-919b-61a86e083492 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:19.323223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-084f77a6-c966-465d-919b-61a86e083492 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:19.466426 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-084f77a6-c966-465d-919b-61a86e083492 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:19.475422 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-084f77a6-c966-465d-919b-61a86e083492 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1550 microversion: 2.1 time: 0.167538 Aug 28 01:37:19.476241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1803/3377] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:19 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1550 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:19.495195 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0102f517-b233-462b-840a-09abec68ef4f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:19.500080 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0102f517-b233-462b-840a-09abec68ef4f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:19.500925 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0102f517-b233-462b-840a-09abec68ef4f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Searching by: {'deleted': False, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7'} {{(pid=120524) get_all /opt/stack/nova/nova/compute/api.py:3017}} Aug 28 01:37:19.508483 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0102f517-b233-462b-840a-09abec68ef4f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:19.509198 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0102f517-b233-462b-840a-09abec68ef4f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:19.509629 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0102f517-b233-462b-840a-09abec68ef4f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:19.510373 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0102f517-b233-462b-840a-09abec68ef4f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:19.510818 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0102f517-b233-462b-840a-09abec68ef4f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:19.511232 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0102f517-b233-462b-840a-09abec68ef4f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:19.601699 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.multi_cell_list [None req-0102f517-b233-462b-840a-09abec68ef4f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=120524) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Aug 28 01:37:19.697801 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0102f517-b233-462b-840a-09abec68ef4f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.206297 Aug 28 01:37:19.698393 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1575/3378] 10.222.0.42 () {58 vars in 1125 bytes} [Sun Aug 28 01:37:19 2022] GET /compute/v2.1/servers => generated 343 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:37:19.709952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f2400f5b-8cb2-48d4-ad27-e7ad76f5a74f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:19.710955 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f2400f5b-8cb2-48d4-ad27-e7ad76f5a74f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:19.717416 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f2400f5b-8cb2-48d4-ad27-e7ad76f5a74f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:19.717942 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f2400f5b-8cb2-48d4-ad27-e7ad76f5a74f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:19.718690 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f2400f5b-8cb2-48d4-ad27-e7ad76f5a74f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:19.720015 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f2400f5b-8cb2-48d4-ad27-e7ad76f5a74f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:19.720851 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f2400f5b-8cb2-48d4-ad27-e7ad76f5a74f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:19.721673 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f2400f5b-8cb2-48d4-ad27-e7ad76f5a74f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:19.882537 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f2400f5b-8cb2-48d4-ad27-e7ad76f5a74f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:19.891301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f2400f5b-8cb2-48d4-ad27-e7ad76f5a74f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1550 microversion: 2.1 time: 0.185160 Aug 28 01:37:19.891878 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1804/3379] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:19 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1550 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:21.567781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3812e6ce-f902-4a40-ab6c-15032f56b80d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "1e6d6a7b-6972-4389-a732-0024a18db314"}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:37:21.570319 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3812e6ce-f902-4a40-ab6c-15032f56b80d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:21.582247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3812e6ce-f902-4a40-ab6c-15032f56b80d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:21.582731 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3812e6ce-f902-4a40-ab6c-15032f56b80d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:21.583162 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3812e6ce-f902-4a40-ab6c-15032f56b80d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:21.591006 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3812e6ce-f902-4a40-ab6c-15032f56b80d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:21.591507 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3812e6ce-f902-4a40-ab6c-15032f56b80d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:21.592044 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3812e6ce-f902-4a40-ab6c-15032f56b80d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:21.909237 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3812e6ce-f902-4a40-ab6c-15032f56b80d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "POST /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.345603 Aug 28 01:37:21.909859 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1576/3380] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Aug 28 01:37:21 2022] POST /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b/os-volume_attachments => generated 194 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:37:30.612290 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f0af6af6-e6e4-402b-bd64-806b89e65366 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:30.612886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f0af6af6-e6e4-402b-bd64-806b89e65366 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:30.615524 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=3 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:37:30.616152 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=56 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:37:30.622499 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0af6af6-e6e4-402b-bd64-806b89e65366 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:30.622969 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0af6af6-e6e4-402b-bd64-806b89e65366 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:30.623392 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0af6af6-e6e4-402b-bd64-806b89e65366 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:30.624064 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0af6af6-e6e4-402b-bd64-806b89e65366 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:30.624501 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0af6af6-e6e4-402b-bd64-806b89e65366 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:30.624902 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0af6af6-e6e4-402b-bd64-806b89e65366 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:30.764076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f0af6af6-e6e4-402b-bd64-806b89e65366 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:30.772482 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f0af6af6-e6e4-402b-bd64-806b89e65366 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1596 microversion: 2.1 time: 0.164472 Aug 28 01:37:30.773067 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1805/3381] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:30 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1596 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:37:32.859673 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=2 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:37:32.860777 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=5 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:37:32.861749 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=75,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:37:32.866598 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-b6991317-26b1-4852-a3e9-d5cc131abbcc req-b1ba0e0e-004c-4999-b524-f87ea2716755 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5aa21554-b4b9-49b1-9fd4-d802097e1a1b", "tag": "e12ff0e5-7520-4d3f-b645-8686e263d5b8"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:37:32.881135 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-b6991317-26b1-4852-a3e9-d5cc131abbcc req-b1ba0e0e-004c-4999-b524-f87ea2716755 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:32.881822 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-b6991317-26b1-4852-a3e9-d5cc131abbcc req-b1ba0e0e-004c-4999-b524-f87ea2716755 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:32.882528 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-b6991317-26b1-4852-a3e9-d5cc131abbcc req-b1ba0e0e-004c-4999-b524-f87ea2716755 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:32.898143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-b6991317-26b1-4852-a3e9-d5cc131abbcc req-b1ba0e0e-004c-4999-b524-f87ea2716755 service nova] Creating event network-changed:e12ff0e5-7520-4d3f-b645-8686e263d5b8 for instance 5aa21554-b4b9-49b1-9fd4-d802097e1a1b on hv-cinder-maste Aug 28 01:37:32.903428 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-b6991317-26b1-4852-a3e9-d5cc131abbcc req-b1ba0e0e-004c-4999-b524-f87ea2716755 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045542 Aug 28 01:37:32.904042 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1577/3382] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:37:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:37:33.839114 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e38e2260-49e4-4139-a852-bfbb787b9070 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-161210991"}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:37:33.839656 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e38e2260-49e4-4139-a852-bfbb787b9070 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:33.846371 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e38e2260-49e4-4139-a852-bfbb787b9070 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:33.846892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e38e2260-49e4-4139-a852-bfbb787b9070 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:33.847343 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e38e2260-49e4-4139-a852-bfbb787b9070 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:33.848097 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e38e2260-49e4-4139-a852-bfbb787b9070 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:33.848569 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e38e2260-49e4-4139-a852-bfbb787b9070 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:33.849231 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e38e2260-49e4-4139-a852-bfbb787b9070 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:33.976380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.network.security_group_api [None req-e38e2260-49e4-4139-a852-bfbb787b9070 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Adding security group e930cdad-cd1f-4811-a044-abe37b02f46d to port e12ff0e5-7520-4d3f-b645-8686e263d5b8 Aug 28 01:37:34.332103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e38e2260-49e4-4139-a852-bfbb787b9070 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "POST /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b/action" status: 202 len: 0 microversion: 2.1 time: 0.496891 Aug 28 01:37:34.332843 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1806/3383] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Aug 28 01:37:33 2022] POST /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b/action => generated 0 bytes in 498 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 01:37:34.342481 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-49a8c0b1-6bd4-4cfb-9bd5-9e1f4bb64f77 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:37:34.343208 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-49a8c0b1-6bd4-4cfb-9bd5-9e1f4bb64f77 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:37:34.349834 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49a8c0b1-6bd4-4cfb-9bd5-9e1f4bb64f77 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:34.350359 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49a8c0b1-6bd4-4cfb-9bd5-9e1f4bb64f77 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:34.350836 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49a8c0b1-6bd4-4cfb-9bd5-9e1f4bb64f77 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:34.351908 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49a8c0b1-6bd4-4cfb-9bd5-9e1f4bb64f77 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:34.352602 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49a8c0b1-6bd4-4cfb-9bd5-9e1f4bb64f77 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:34.353279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-49a8c0b1-6bd4-4cfb-9bd5-9e1f4bb64f77 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:34.511400 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-49a8c0b1-6bd4-4cfb-9bd5-9e1f4bb64f77 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:37:34.520926 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-49a8c0b1-6bd4-4cfb-9bd5-9e1f4bb64f77 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1642 microversion: 2.1 time: 0.182990 Aug 28 01:37:34.521644 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1578/3384] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:37:34 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1642 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:37:34.918526 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-5543637c-7c7f-45b6-9546-45b0d80ab842 req-1a8b56a6-32ad-490d-8f39-9e8310330809 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5aa21554-b4b9-49b1-9fd4-d802097e1a1b", "tag": "e12ff0e5-7520-4d3f-b645-8686e263d5b8"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:37:34.929041 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-5543637c-7c7f-45b6-9546-45b0d80ab842 req-1a8b56a6-32ad-490d-8f39-9e8310330809 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:37:34.930167 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-5543637c-7c7f-45b6-9546-45b0d80ab842 req-1a8b56a6-32ad-490d-8f39-9e8310330809 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:37:34.930599 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-5543637c-7c7f-45b6-9546-45b0d80ab842 req-1a8b56a6-32ad-490d-8f39-9e8310330809 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:37:34.942282 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-5543637c-7c7f-45b6-9546-45b0d80ab842 req-1a8b56a6-32ad-490d-8f39-9e8310330809 service nova] Creating event network-changed:e12ff0e5-7520-4d3f-b645-8686e263d5b8 for instance 5aa21554-b4b9-49b1-9fd4-d802097e1a1b on hv-cinder-maste Aug 28 01:37:34.947819 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-5543637c-7c7f-45b6-9546-45b0d80ab842 req-1a8b56a6-32ad-490d-8f39-9e8310330809 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033509 Aug 28 01:37:34.948413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1807/3385] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:37:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:39:42.753468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 28 01:39:42.754118 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Sun Aug 28 01:39:42 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b/action (ip 10.222.0.42) !!! Aug 28 01:39:42.755563 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 28 01:39:42.755699 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Sun Aug 28 01:39:42 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b/action (ip 10.222.0.42) !!! Aug 28 01:39:42.756576 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=4 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:39:42.757337 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=8 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:39:42.762390 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8ad6f482-b252-4c1c-980e-adf2c988a4cb tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:39:42.764707 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8ad6f482-b252-4c1c-980e-adf2c988a4cb tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:39:42.773896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ad6f482-b252-4c1c-980e-adf2c988a4cb tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:42.774364 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ad6f482-b252-4c1c-980e-adf2c988a4cb tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:42.774785 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ad6f482-b252-4c1c-980e-adf2c988a4cb tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:42.775467 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ad6f482-b252-4c1c-980e-adf2c988a4cb tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:42.775899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ad6f482-b252-4c1c-980e-adf2c988a4cb tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:42.776311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8ad6f482-b252-4c1c-980e-adf2c988a4cb tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:42.855133 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8ad6f482-b252-4c1c-980e-adf2c988a4cb tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "POST /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b/action" status: 202 len: 0 microversion: 2.1 time: 0.103652 Aug 28 01:39:42.855963 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1579/3386] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Aug 28 01:39:42 2022] POST /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b/action => generated 0 bytes in 106 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 01:39:42.862956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 28 01:39:42.863609 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Sun Aug 28 01:39:42 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b (ip 10.222.0.42) !!! Aug 28 01:39:42.864186 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 28 01:39:42.864400 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Sun Aug 28 01:39:42 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b (ip 10.222.0.42) !!! Aug 28 01:39:42.865292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=6 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:39:42.866054 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=12 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:39:42.870763 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-ac11c293-f43f-45c3-a55b-d57935d6df3e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:39:42.871506 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-ac11c293-f43f-45c3-a55b-d57935d6df3e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:39:42.880983 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac11c293-f43f-45c3-a55b-d57935d6df3e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:42.881483 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac11c293-f43f-45c3-a55b-d57935d6df3e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:42.881926 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac11c293-f43f-45c3-a55b-d57935d6df3e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:42.882602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac11c293-f43f-45c3-a55b-d57935d6df3e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:42.883049 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac11c293-f43f-45c3-a55b-d57935d6df3e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:42.883466 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-ac11c293-f43f-45c3-a55b-d57935d6df3e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:43.126543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-ac11c293-f43f-45c3-a55b-d57935d6df3e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:39:43.135522 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-ac11c293-f43f-45c3-a55b-d57935d6df3e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1753 microversion: 2.1 time: 0.274839 Aug 28 01:39:43.136089 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1808/3387] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:39:42 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1753 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:39:44.157619 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-07560093-c1bb-4c14-bbe3-e5a7df738ff7 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:39:44.158312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-07560093-c1bb-4c14-bbe3-e5a7df738ff7 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:39:44.165141 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07560093-c1bb-4c14-bbe3-e5a7df738ff7 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:44.165628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07560093-c1bb-4c14-bbe3-e5a7df738ff7 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:44.166098 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07560093-c1bb-4c14-bbe3-e5a7df738ff7 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:44.166764 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07560093-c1bb-4c14-bbe3-e5a7df738ff7 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:44.167240 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07560093-c1bb-4c14-bbe3-e5a7df738ff7 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:44.167659 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-07560093-c1bb-4c14-bbe3-e5a7df738ff7 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:44.319331 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-07560093-c1bb-4c14-bbe3-e5a7df738ff7 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:39:44.327850 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-07560093-c1bb-4c14-bbe3-e5a7df738ff7 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1758 microversion: 2.1 time: 0.175285 Aug 28 01:39:44.328422 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1580/3388] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:39:44 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1758 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:39:45.349468 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c314ee4f-528c-4e6a-96fb-dd3426969a7f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:39:45.350157 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c314ee4f-528c-4e6a-96fb-dd3426969a7f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:39:45.357027 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c314ee4f-528c-4e6a-96fb-dd3426969a7f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:45.357510 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c314ee4f-528c-4e6a-96fb-dd3426969a7f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:45.358030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c314ee4f-528c-4e6a-96fb-dd3426969a7f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:45.358717 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c314ee4f-528c-4e6a-96fb-dd3426969a7f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:45.359161 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c314ee4f-528c-4e6a-96fb-dd3426969a7f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:45.359566 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c314ee4f-528c-4e6a-96fb-dd3426969a7f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:45.511142 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c314ee4f-528c-4e6a-96fb-dd3426969a7f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:39:45.520012 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c314ee4f-528c-4e6a-96fb-dd3426969a7f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1758 microversion: 2.1 time: 0.174445 Aug 28 01:39:45.520583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1809/3389] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:39:45 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1758 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:39:46.541709 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-42193a4b-b236-41c6-a92b-ea4273b5e01b tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:39:46.542325 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-42193a4b-b236-41c6-a92b-ea4273b5e01b tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:39:46.549514 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42193a4b-b236-41c6-a92b-ea4273b5e01b tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:46.550004 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42193a4b-b236-41c6-a92b-ea4273b5e01b tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:46.550437 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42193a4b-b236-41c6-a92b-ea4273b5e01b tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:46.551135 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42193a4b-b236-41c6-a92b-ea4273b5e01b tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:46.551596 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42193a4b-b236-41c6-a92b-ea4273b5e01b tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:46.552025 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-42193a4b-b236-41c6-a92b-ea4273b5e01b tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:46.706257 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-42193a4b-b236-41c6-a92b-ea4273b5e01b tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:39:46.714978 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-42193a4b-b236-41c6-a92b-ea4273b5e01b tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1758 microversion: 2.1 time: 0.177075 Aug 28 01:39:46.715552 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1581/3390] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:39:46 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1758 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:39:47.736661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-69fb8b1a-571e-4608-82e7-97278f05ee34 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:39:47.737438 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-69fb8b1a-571e-4608-82e7-97278f05ee34 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:39:47.744447 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69fb8b1a-571e-4608-82e7-97278f05ee34 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:47.744932 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69fb8b1a-571e-4608-82e7-97278f05ee34 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:47.745428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69fb8b1a-571e-4608-82e7-97278f05ee34 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:47.746138 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69fb8b1a-571e-4608-82e7-97278f05ee34 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:47.746642 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69fb8b1a-571e-4608-82e7-97278f05ee34 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:47.747062 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-69fb8b1a-571e-4608-82e7-97278f05ee34 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:47.903586 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-69fb8b1a-571e-4608-82e7-97278f05ee34 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:39:47.912673 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-69fb8b1a-571e-4608-82e7-97278f05ee34 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1758 microversion: 2.1 time: 0.179844 Aug 28 01:39:47.913292 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1810/3391] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:39:47 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1758 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:39:48.934652 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b6493f56-e465-4e4f-a463-6c8c58a6aa82 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:39:48.935325 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b6493f56-e465-4e4f-a463-6c8c58a6aa82 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:39:48.942493 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6493f56-e465-4e4f-a463-6c8c58a6aa82 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:48.942980 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6493f56-e465-4e4f-a463-6c8c58a6aa82 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:48.943415 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6493f56-e465-4e4f-a463-6c8c58a6aa82 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:48.944118 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6493f56-e465-4e4f-a463-6c8c58a6aa82 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:48.944573 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6493f56-e465-4e4f-a463-6c8c58a6aa82 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:48.945027 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b6493f56-e465-4e4f-a463-6c8c58a6aa82 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:49.094675 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b6493f56-e465-4e4f-a463-6c8c58a6aa82 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:39:49.104194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b6493f56-e465-4e4f-a463-6c8c58a6aa82 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1758 microversion: 2.1 time: 0.173466 Aug 28 01:39:49.104779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1582/3392] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:39:48 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1758 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:39:50.126567 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-984574bd-1ff4-4535-bb5b-6bd7ea3efea9 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:39:50.127175 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-984574bd-1ff4-4535-bb5b-6bd7ea3efea9 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:39:50.134020 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-984574bd-1ff4-4535-bb5b-6bd7ea3efea9 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:50.134504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-984574bd-1ff4-4535-bb5b-6bd7ea3efea9 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:50.134917 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-984574bd-1ff4-4535-bb5b-6bd7ea3efea9 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:50.135619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-984574bd-1ff4-4535-bb5b-6bd7ea3efea9 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:50.136105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-984574bd-1ff4-4535-bb5b-6bd7ea3efea9 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:50.136520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-984574bd-1ff4-4535-bb5b-6bd7ea3efea9 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:50.286226 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-984574bd-1ff4-4535-bb5b-6bd7ea3efea9 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:39:50.304547 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-984574bd-1ff4-4535-bb5b-6bd7ea3efea9 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1758 microversion: 2.1 time: 0.176186 Aug 28 01:39:50.305739 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1811/3393] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:39:50 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1758 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:39:51.327620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bff0de27-92da-4448-8f35-fa2848eda6be tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:39:51.328224 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bff0de27-92da-4448-8f35-fa2848eda6be tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:39:51.335279 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bff0de27-92da-4448-8f35-fa2848eda6be tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:51.335759 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bff0de27-92da-4448-8f35-fa2848eda6be tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:51.336185 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bff0de27-92da-4448-8f35-fa2848eda6be tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:51.336876 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bff0de27-92da-4448-8f35-fa2848eda6be tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:51.337351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bff0de27-92da-4448-8f35-fa2848eda6be tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:51.337765 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bff0de27-92da-4448-8f35-fa2848eda6be tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:51.504228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bff0de27-92da-4448-8f35-fa2848eda6be tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:39:51.512803 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bff0de27-92da-4448-8f35-fa2848eda6be tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1758 microversion: 2.1 time: 0.189363 Aug 28 01:39:51.513409 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1583/3394] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:39:51 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1758 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:39:52.535187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d7c50413-d025-4d12-bbdf-0fb6aef1f600 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:39:52.535751 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d7c50413-d025-4d12-bbdf-0fb6aef1f600 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:39:52.542664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c50413-d025-4d12-bbdf-0fb6aef1f600 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:52.543147 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c50413-d025-4d12-bbdf-0fb6aef1f600 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:52.543609 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c50413-d025-4d12-bbdf-0fb6aef1f600 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:52.544648 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c50413-d025-4d12-bbdf-0fb6aef1f600 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:52.544784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c50413-d025-4d12-bbdf-0fb6aef1f600 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:52.545171 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d7c50413-d025-4d12-bbdf-0fb6aef1f600 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:52.696655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d7c50413-d025-4d12-bbdf-0fb6aef1f600 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:39:52.705073 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d7c50413-d025-4d12-bbdf-0fb6aef1f600 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1758 microversion: 2.1 time: 0.173990 Aug 28 01:39:52.705669 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1812/3395] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:39:52 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1758 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:39:53.729316 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-775a0b4c-ab28-4f32-bbf4-67d78722a717 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:39:53.730040 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-775a0b4c-ab28-4f32-bbf4-67d78722a717 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:39:53.742480 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-775a0b4c-ab28-4f32-bbf4-67d78722a717 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:53.743340 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-775a0b4c-ab28-4f32-bbf4-67d78722a717 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:53.744085 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-775a0b4c-ab28-4f32-bbf4-67d78722a717 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:53.745320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-775a0b4c-ab28-4f32-bbf4-67d78722a717 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:53.746109 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-775a0b4c-ab28-4f32-bbf4-67d78722a717 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:53.746876 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-775a0b4c-ab28-4f32-bbf4-67d78722a717 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:53.917770 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-775a0b4c-ab28-4f32-bbf4-67d78722a717 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:39:53.926883 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-775a0b4c-ab28-4f32-bbf4-67d78722a717 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1758 microversion: 2.1 time: 0.203923 Aug 28 01:39:53.927504 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1584/3396] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:39:53 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1758 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:39:54.948457 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c08dfe1d-0f4b-4977-bc42-fc51295912e5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:39:54.949134 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c08dfe1d-0f4b-4977-bc42-fc51295912e5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:39:54.957133 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c08dfe1d-0f4b-4977-bc42-fc51295912e5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:54.957641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c08dfe1d-0f4b-4977-bc42-fc51295912e5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:54.958037 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c08dfe1d-0f4b-4977-bc42-fc51295912e5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:54.958769 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c08dfe1d-0f4b-4977-bc42-fc51295912e5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:54.959225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c08dfe1d-0f4b-4977-bc42-fc51295912e5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:54.959638 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c08dfe1d-0f4b-4977-bc42-fc51295912e5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:55.122204 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c08dfe1d-0f4b-4977-bc42-fc51295912e5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:39:55.131252 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c08dfe1d-0f4b-4977-bc42-fc51295912e5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1758 microversion: 2.1 time: 0.186613 Aug 28 01:39:55.131877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1813/3397] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:39:54 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1758 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:39:56.153323 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-070b43df-8eae-4fe6-b63f-246b22eeb18c tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:39:56.153886 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-070b43df-8eae-4fe6-b63f-246b22eeb18c tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:39:56.160986 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-070b43df-8eae-4fe6-b63f-246b22eeb18c tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:56.161484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-070b43df-8eae-4fe6-b63f-246b22eeb18c tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:56.161927 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-070b43df-8eae-4fe6-b63f-246b22eeb18c tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:56.162638 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-070b43df-8eae-4fe6-b63f-246b22eeb18c tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:56.163079 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-070b43df-8eae-4fe6-b63f-246b22eeb18c tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:56.163484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-070b43df-8eae-4fe6-b63f-246b22eeb18c tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:56.335962 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-070b43df-8eae-4fe6-b63f-246b22eeb18c tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:39:56.344598 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-070b43df-8eae-4fe6-b63f-246b22eeb18c tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1758 microversion: 2.1 time: 0.195163 Aug 28 01:39:56.345222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1585/3398] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:39:56 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1758 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:39:57.367589 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6075ce14-7f52-42ff-8cd3-701a87ae98b2 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:39:57.368414 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6075ce14-7f52-42ff-8cd3-701a87ae98b2 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:39:57.379443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6075ce14-7f52-42ff-8cd3-701a87ae98b2 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:57.380268 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6075ce14-7f52-42ff-8cd3-701a87ae98b2 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:57.381008 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6075ce14-7f52-42ff-8cd3-701a87ae98b2 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:57.382223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6075ce14-7f52-42ff-8cd3-701a87ae98b2 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:57.383023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6075ce14-7f52-42ff-8cd3-701a87ae98b2 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:57.383757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6075ce14-7f52-42ff-8cd3-701a87ae98b2 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:57.556864 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6075ce14-7f52-42ff-8cd3-701a87ae98b2 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:39:57.565611 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6075ce14-7f52-42ff-8cd3-701a87ae98b2 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1758 microversion: 2.1 time: 0.203241 Aug 28 01:39:57.566212 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1814/3399] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:39:57 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1758 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:39:58.589538 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8fecbf8c-e8ff-4aaa-bc9e-09a03c229029 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:39:58.590298 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8fecbf8c-e8ff-4aaa-bc9e-09a03c229029 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:39:58.597447 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fecbf8c-e8ff-4aaa-bc9e-09a03c229029 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:58.597935 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fecbf8c-e8ff-4aaa-bc9e-09a03c229029 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:58.598352 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fecbf8c-e8ff-4aaa-bc9e-09a03c229029 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:58.599064 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fecbf8c-e8ff-4aaa-bc9e-09a03c229029 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:58.599492 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fecbf8c-e8ff-4aaa-bc9e-09a03c229029 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:58.599902 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8fecbf8c-e8ff-4aaa-bc9e-09a03c229029 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:58.764302 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8fecbf8c-e8ff-4aaa-bc9e-09a03c229029 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:39:58.773489 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8fecbf8c-e8ff-4aaa-bc9e-09a03c229029 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1758 microversion: 2.1 time: 0.188036 Aug 28 01:39:58.774096 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1586/3400] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:39:58 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1758 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:39:59.799031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-cbfe4a60-7cfb-4267-b080-bbfa97680b0f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:39:59.799825 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-cbfe4a60-7cfb-4267-b080-bbfa97680b0f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:39:59.807092 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbfe4a60-7cfb-4267-b080-bbfa97680b0f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:59.807564 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbfe4a60-7cfb-4267-b080-bbfa97680b0f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:59.807994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbfe4a60-7cfb-4267-b080-bbfa97680b0f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:59.808673 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbfe4a60-7cfb-4267-b080-bbfa97680b0f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:39:59.809162 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbfe4a60-7cfb-4267-b080-bbfa97680b0f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:39:59.809589 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-cbfe4a60-7cfb-4267-b080-bbfa97680b0f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:39:59.967257 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-cbfe4a60-7cfb-4267-b080-bbfa97680b0f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:39:59.976090 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-cbfe4a60-7cfb-4267-b080-bbfa97680b0f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1758 microversion: 2.1 time: 0.181007 Aug 28 01:39:59.976676 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1815/3401] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:39:59 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1758 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:40:01.000841 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-17bce5f9-6865-444e-b6fc-4920b5fde977 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:40:01.001781 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-17bce5f9-6865-444e-b6fc-4920b5fde977 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:40:01.010856 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-17bce5f9-6865-444e-b6fc-4920b5fde977 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:40:01.011341 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-17bce5f9-6865-444e-b6fc-4920b5fde977 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:40:01.011779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-17bce5f9-6865-444e-b6fc-4920b5fde977 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:40:01.012479 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-17bce5f9-6865-444e-b6fc-4920b5fde977 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:40:01.012931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-17bce5f9-6865-444e-b6fc-4920b5fde977 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:40:01.013381 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-17bce5f9-6865-444e-b6fc-4920b5fde977 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:40:01.163473 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-17bce5f9-6865-444e-b6fc-4920b5fde977 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:40:01.175209 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-17bce5f9-6865-444e-b6fc-4920b5fde977 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1761 microversion: 2.1 time: 0.180362 Aug 28 01:40:01.175957 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1587/3402] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:40:00 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1761 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:41:04.577915 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 28 01:41:04.578545 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Sun Aug 28 01:41:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b/action (ip 10.222.0.42) !!! Aug 28 01:41:04.579585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=16 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:41:04.580314 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=48 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:41:04.584834 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-da2ffb77-b985-46c2-ab1a-fc982753ca21 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-161210991"}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:41:04.585389 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-da2ffb77-b985-46c2-ab1a-fc982753ca21 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:41:04.593612 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da2ffb77-b985-46c2-ab1a-fc982753ca21 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:04.594159 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da2ffb77-b985-46c2-ab1a-fc982753ca21 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:04.594587 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da2ffb77-b985-46c2-ab1a-fc982753ca21 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:04.595399 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da2ffb77-b985-46c2-ab1a-fc982753ca21 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:04.595716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da2ffb77-b985-46c2-ab1a-fc982753ca21 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:04.596116 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-da2ffb77-b985-46c2-ab1a-fc982753ca21 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:04.917709 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.network.security_group_api [None req-da2ffb77-b985-46c2-ab1a-fc982753ca21 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Removing security group e930cdad-cd1f-4811-a044-abe37b02f46d from port e12ff0e5-7520-4d3f-b645-8686e263d5b8 Aug 28 01:41:05.254102 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-da2ffb77-b985-46c2-ab1a-fc982753ca21 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "POST /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b/action" status: 202 len: 0 microversion: 2.1 time: 0.677940 Aug 28 01:41:05.254758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1816/3403] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Aug 28 01:41:04 2022] POST /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b/action => generated 0 bytes in 679 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 01:41:05.262916 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=18 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:41:05.263667 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=58,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:41:05.363397 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-7b846f4d-90ef-4672-8b1b-1f1ec24e281f req-0ddd01be-b225-44b5-bb2d-ea03b598c4ac service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5aa21554-b4b9-49b1-9fd4-d802097e1a1b", "tag": "e12ff0e5-7520-4d3f-b645-8686e263d5b8"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:41:05.373730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-7b846f4d-90ef-4672-8b1b-1f1ec24e281f req-0ddd01be-b225-44b5-bb2d-ea03b598c4ac service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:05.374253 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-7b846f4d-90ef-4672-8b1b-1f1ec24e281f req-0ddd01be-b225-44b5-bb2d-ea03b598c4ac service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:05.374930 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-7b846f4d-90ef-4672-8b1b-1f1ec24e281f req-0ddd01be-b225-44b5-bb2d-ea03b598c4ac service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:05.387569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-7b846f4d-90ef-4672-8b1b-1f1ec24e281f req-0ddd01be-b225-44b5-bb2d-ea03b598c4ac service nova] Creating event network-changed:e12ff0e5-7520-4d3f-b645-8686e263d5b8 for instance 5aa21554-b4b9-49b1-9fd4-d802097e1a1b on hv-cinder-maste Aug 28 01:41:05.393414 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-7b846f4d-90ef-4672-8b1b-1f1ec24e281f req-0ddd01be-b225-44b5-bb2d-ea03b598c4ac service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.133447 Aug 28 01:41:05.394344 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1588/3404] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:41:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:41:06.679939 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-37a1a1ea-1be3-4f4c-9f33-1919372fd3f8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:41:06.680452 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-37a1a1ea-1be3-4f4c-9f33-1919372fd3f8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:41:06.687050 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-37a1a1ea-1be3-4f4c-9f33-1919372fd3f8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:06.687520 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-37a1a1ea-1be3-4f4c-9f33-1919372fd3f8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:06.687941 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-37a1a1ea-1be3-4f4c-9f33-1919372fd3f8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:06.688619 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-37a1a1ea-1be3-4f4c-9f33-1919372fd3f8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:06.689089 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-37a1a1ea-1be3-4f4c-9f33-1919372fd3f8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:06.689487 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-37a1a1ea-1be3-4f4c-9f33-1919372fd3f8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:06.770737 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.objects.instance [None req-37a1a1ea-1be3-4f4c-9f33-1919372fd3f8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lazy-loading 'services' on Instance uuid 5aa21554-b4b9-49b1-9fd4-d802097e1a1b {{(pid=120525) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:41:06.846797 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-37a1a1ea-1be3-4f4c-9f33-1919372fd3f8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "DELETE /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b/os-volume_attachments/1e6d6a7b-6972-4389-a732-0024a18db314" status: 202 len: 0 microversion: 2.1 time: 0.171408 Aug 28 01:41:06.847680 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1817/3405] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Aug 28 01:41:06 2022] DELETE /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b/os-volume_attachments/1e6d6a7b-6972-4389-a732-0024a18db314 => generated 0 bytes in 173 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 01:41:06.858853 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-e409b77d-cdcb-4afb-9c4f-b92c03e3dc74 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:41:06.861831 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-e409b77d-cdcb-4afb-9c4f-b92c03e3dc74 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:41:06.872043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e409b77d-cdcb-4afb-9c4f-b92c03e3dc74 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:06.872672 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e409b77d-cdcb-4afb-9c4f-b92c03e3dc74 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:06.873385 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e409b77d-cdcb-4afb-9c4f-b92c03e3dc74 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:06.874224 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e409b77d-cdcb-4afb-9c4f-b92c03e3dc74 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:06.874733 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e409b77d-cdcb-4afb-9c4f-b92c03e3dc74 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:06.875216 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-e409b77d-cdcb-4afb-9c4f-b92c03e3dc74 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:06.905453 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-e409b77d-cdcb-4afb-9c4f-b92c03e3dc74 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053007 Aug 28 01:41:06.906450 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1589/3406] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:41:06 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:41:07.410947 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-d51afbf2-cb0c-46a8-a298-db1d0f66ba81 req-b2ec33d7-51b8-4946-8701-5af59335c708 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5aa21554-b4b9-49b1-9fd4-d802097e1a1b", "tag": "e12ff0e5-7520-4d3f-b645-8686e263d5b8"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:41:07.423047 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-d51afbf2-cb0c-46a8-a298-db1d0f66ba81 req-b2ec33d7-51b8-4946-8701-5af59335c708 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:07.423833 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-d51afbf2-cb0c-46a8-a298-db1d0f66ba81 req-b2ec33d7-51b8-4946-8701-5af59335c708 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:07.424264 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-d51afbf2-cb0c-46a8-a298-db1d0f66ba81 req-b2ec33d7-51b8-4946-8701-5af59335c708 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:07.436379 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-d51afbf2-cb0c-46a8-a298-db1d0f66ba81 req-b2ec33d7-51b8-4946-8701-5af59335c708 service nova] Creating event network-changed:e12ff0e5-7520-4d3f-b645-8686e263d5b8 for instance 5aa21554-b4b9-49b1-9fd4-d802097e1a1b on hv-cinder-maste Aug 28 01:41:07.441240 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-d51afbf2-cb0c-46a8-a298-db1d0f66ba81 req-b2ec33d7-51b8-4946-8701-5af59335c708 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034788 Aug 28 01:41:07.441829 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1818/3407] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:41:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:41:07.918174 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-396ac151-a358-4c9a-96e9-2bd16048b441 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:41:07.920415 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-396ac151-a358-4c9a-96e9-2bd16048b441 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:41:07.926935 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-396ac151-a358-4c9a-96e9-2bd16048b441 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:07.928215 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-396ac151-a358-4c9a-96e9-2bd16048b441 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:07.928696 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-396ac151-a358-4c9a-96e9-2bd16048b441 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:07.929653 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-396ac151-a358-4c9a-96e9-2bd16048b441 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:07.930218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-396ac151-a358-4c9a-96e9-2bd16048b441 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:07.930703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-396ac151-a358-4c9a-96e9-2bd16048b441 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:07.964677 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-396ac151-a358-4c9a-96e9-2bd16048b441 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050495 Aug 28 01:41:07.965296 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1590/3408] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:41:07 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:41:08.978431 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e5193ccf-581c-4c9a-b6e9-3b4587c18ba8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:41:08.980665 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e5193ccf-581c-4c9a-b6e9-3b4587c18ba8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:41:08.987279 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e5193ccf-581c-4c9a-b6e9-3b4587c18ba8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:08.987763 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e5193ccf-581c-4c9a-b6e9-3b4587c18ba8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:08.988191 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e5193ccf-581c-4c9a-b6e9-3b4587c18ba8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:08.988883 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e5193ccf-581c-4c9a-b6e9-3b4587c18ba8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:08.989336 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e5193ccf-581c-4c9a-b6e9-3b4587c18ba8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:08.989750 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e5193ccf-581c-4c9a-b6e9-3b4587c18ba8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:09.019126 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e5193ccf-581c-4c9a-b6e9-3b4587c18ba8 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.045429 Aug 28 01:41:09.019664 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1819/3409] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Aug 28 01:41:08 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Aug 28 01:41:09.033018 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7e2cdb0c-53a7-4bce-ae59-fc330a04ba1d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:41:09.033559 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-7e2cdb0c-53a7-4bce-ae59-fc330a04ba1d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:41:09.042958 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e2cdb0c-53a7-4bce-ae59-fc330a04ba1d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:09.043562 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e2cdb0c-53a7-4bce-ae59-fc330a04ba1d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:09.044144 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e2cdb0c-53a7-4bce-ae59-fc330a04ba1d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:09.045261 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e2cdb0c-53a7-4bce-ae59-fc330a04ba1d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:09.045903 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e2cdb0c-53a7-4bce-ae59-fc330a04ba1d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:09.046518 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-7e2cdb0c-53a7-4bce-ae59-fc330a04ba1d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:09.258288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-7e2cdb0c-53a7-4bce-ae59-fc330a04ba1d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] HTTP exception thrown: Instance 5aa21554-b4b9-49b1-9fd4-d802097e1a1b is not attached to volume 1e6d6a7b-6972-4389-a732-0024a18db314 Aug 28 01:41:09.260779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7e2cdb0c-53a7-4bce-ae59-fc330a04ba1d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Request method failure captured: Aug 28 01:41:09.260779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: DELETE /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b/os-volume_attachments/1e6d6a7b-6972-4389-a732-0024a18db314 HTTP/1.1 Aug 28 01:41:09.260779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:41:09.260779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:41:09.260779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:41:09.260779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:41:09.260779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:41:09.260779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:41:09.260779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:41:09.260779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:41:09.260779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCsaiYjx6hkP8LalTlGUkXaedL4hodMc863R9Tte9X_auHcbo76Kw8uWZpCrs_okviykBPDCBYTN8snp6d8SEsTbN8i4uh5mRzPzY0swXNUjza4optXT00Gz6n3hFLiCdtFen8as7W6KRAB6d6NmZJHO7e0zT87yWC_3BEl-D3idW_zI Aug 28 01:41:09.260779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:41:09.260779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:41:09.260779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:41:09.260779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:41:09.260779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:41:09.260779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:41:09.260779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 8bd4256c51f64ef9aa0f21ea2fcb55e7 Aug 28 01:41:09.260779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-TestMinimumBasicScenario-1487453377 Aug 28 01:41:09.260779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:41:09.260779 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:41:09.262354 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_8bd4256c51f64ef9aa0f21ea2fcb55e7", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/8bd4256c51f64ef9aa0f21ea2fcb55e7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/8bd4256c51f64ef9aa0f21ea2fcb55e7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/8bd4256c51f64ef9aa0f21ea2fcb55e7"}]}] Aug 28 01:41:09.262354 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-TestMinimumBasicScenario-1487453377 Aug 28 01:41:09.262354 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 8bd4256c51f64ef9aa0f21ea2fcb55e7 Aug 28 01:41:09.262354 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1487453377 Aug 28 01:41:09.262354 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-TestMinimumBasicScenario-1487453377-project Aug 28 01:41:09.262354 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:41:09.262354 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:41:09.262354 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: c5bc73cffe1e4fa095051e109b5f0d0c Aug 28 01:41:09.262354 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-TestMinimumBasicScenario-1487453377-project Aug 28 01:41:09.262354 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:41:09.262354 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Instance 5aa21554-b4b9-49b1-9fd4-d802097e1a1b is not attached to volume 1e6d6a7b-6972-4389-a732-0024a18db314 Aug 28 01:41:09.263068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'server_id': '5aa21554-b4b9-49b1-9fd4-d802097e1a1b', 'id': '1e6d6a7b-6972-4389-a732-0024a18db314'} Aug 28 01:41:09.263068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:41:09.263068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:41:09.263068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Aug 28 01:41:09.263068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Aug 28 01:41:09.263068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Aug 28 01:41:09.263068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Aug 28 01:41:09.263068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Aug 28 01:41:09.263068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Aug 28 01:41:09.263068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 1e6d6a7b-6972-4389-a732-0024a18db314. Aug 28 01:41:09.263068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:41:09.263068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:41:09.263068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:41:09.263068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:41:09.263068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:41:09.263068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:41:09.263068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:41:09.263068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:41:09.263068 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:41:09.264226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:41:09.264226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Aug 28 01:41:09.264226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Aug 28 01:41:09.264226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5aa21554-b4b9-49b1-9fd4-d802097e1a1b is not attached to volume 1e6d6a7b-6972-4389-a732-0024a18db314 Aug 28 01:41:09.264226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:41:09.264226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:41:09.264226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:41:09.264226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:41:09.264226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:41:09.264226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:41:09.264226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:41:09.264226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:41:09.264226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5aa21554-b4b9-49b1-9fd4-d802097e1a1b is not attached to volume 1e6d6a7b-6972-4389-a732-0024a18db314 Aug 28 01:41:09.264226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:41:09.264226 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-7e2cdb0c-53a7-4bce-ae59-fc330a04ba1d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Returning 404 to user: Instance 5aa21554-b4b9-49b1-9fd4-d802097e1a1b is not attached to volume 1e6d6a7b-6972-4389-a732-0024a18db314 {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:41:09.265258 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-7e2cdb0c-53a7-4bce-ae59-fc330a04ba1d tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "DELETE /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b/os-volume_attachments/1e6d6a7b-6972-4389-a732-0024a18db314" status: 404 len: 154 microversion: 2.1 time: 0.235494 Aug 28 01:41:09.265258 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1591/3410] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Aug 28 01:41:09 2022] DELETE /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b/os-volume_attachments/1e6d6a7b-6972-4389-a732-0024a18db314 => generated 154 bytes in 237 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:41:10.418235 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-12f54dc5-0c0a-44c5-8cde-06d971c7c13e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:41:10.418772 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-12f54dc5-0c0a-44c5-8cde-06d971c7c13e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:41:10.425560 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12f54dc5-0c0a-44c5-8cde-06d971c7c13e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:10.426031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12f54dc5-0c0a-44c5-8cde-06d971c7c13e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:10.426460 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12f54dc5-0c0a-44c5-8cde-06d971c7c13e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:10.427140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12f54dc5-0c0a-44c5-8cde-06d971c7c13e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:10.427551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12f54dc5-0c0a-44c5-8cde-06d971c7c13e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:10.427998 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12f54dc5-0c0a-44c5-8cde-06d971c7c13e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:10.453012 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-12f54dc5-0c0a-44c5-8cde-06d971c7c13e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Going to try to terminate instance {{(pid=120525) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 28 01:41:10.537413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-12f54dc5-0c0a-44c5-8cde-06d971c7c13e tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "DELETE /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 204 len: 0 microversion: 2.1 time: 0.122683 Aug 28 01:41:10.538133 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1820/3411] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 01:41:10 2022] DELETE /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 28 01:41:10.546676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-a7150538-5f8d-49a3-9d53-ca0df462171b tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:41:10.547308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-a7150538-5f8d-49a3-9d53-ca0df462171b tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:41:10.553830 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7150538-5f8d-49a3-9d53-ca0df462171b tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:10.554312 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7150538-5f8d-49a3-9d53-ca0df462171b tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:10.554705 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7150538-5f8d-49a3-9d53-ca0df462171b tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:10.555468 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7150538-5f8d-49a3-9d53-ca0df462171b tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:10.555932 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7150538-5f8d-49a3-9d53-ca0df462171b tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:10.556351 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-a7150538-5f8d-49a3-9d53-ca0df462171b tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:10.702873 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-a7150538-5f8d-49a3-9d53-ca0df462171b tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:41:10.710851 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-a7150538-5f8d-49a3-9d53-ca0df462171b tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1556 microversion: 2.1 time: 0.168075 Aug 28 01:41:10.711432 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1592/3412] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:41:10 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1556 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:41:11.733848 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-16a3a11d-1d70-4f51-99cc-cd7e377d7ba5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:41:11.734493 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-16a3a11d-1d70-4f51-99cc-cd7e377d7ba5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:41:11.741581 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16a3a11d-1d70-4f51-99cc-cd7e377d7ba5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:11.742055 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16a3a11d-1d70-4f51-99cc-cd7e377d7ba5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:11.742456 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16a3a11d-1d70-4f51-99cc-cd7e377d7ba5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:11.743176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16a3a11d-1d70-4f51-99cc-cd7e377d7ba5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:11.743615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16a3a11d-1d70-4f51-99cc-cd7e377d7ba5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:11.744031 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-16a3a11d-1d70-4f51-99cc-cd7e377d7ba5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:11.892088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-16a3a11d-1d70-4f51-99cc-cd7e377d7ba5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:41:11.901403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-16a3a11d-1d70-4f51-99cc-cd7e377d7ba5 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1556 microversion: 2.1 time: 0.171814 Aug 28 01:41:11.902000 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1821/3413] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:41:11 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1556 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:41:12.922660 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-55d3eb12-f9c2-4c0f-8e99-453c77a98dd3 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:41:12.923252 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-55d3eb12-f9c2-4c0f-8e99-453c77a98dd3 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:41:12.930000 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-55d3eb12-f9c2-4c0f-8e99-453c77a98dd3 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:12.930482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-55d3eb12-f9c2-4c0f-8e99-453c77a98dd3 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:12.930905 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-55d3eb12-f9c2-4c0f-8e99-453c77a98dd3 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:12.931582 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-55d3eb12-f9c2-4c0f-8e99-453c77a98dd3 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:12.932022 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-55d3eb12-f9c2-4c0f-8e99-453c77a98dd3 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:12.932429 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-55d3eb12-f9c2-4c0f-8e99-453c77a98dd3 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:13.381461 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-55d3eb12-f9c2-4c0f-8e99-453c77a98dd3 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c5bc73cffe1e4fa095051e109b5f0d0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8bd4256c51f64ef9aa0f21ea2fcb55e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:41:13.390109 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-55d3eb12-f9c2-4c0f-8e99-453c77a98dd3 tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 200 len: 1556 microversion: 2.1 time: 0.471319 Aug 28 01:41:13.390965 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1593/3414] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:41:12 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 1556 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:41:13.562749 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-d34f9801-1181-4106-99d7-a2b4e53a07b4 req-69a540d7-1751-4d9b-afaf-0e715660958a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5aa21554-b4b9-49b1-9fd4-d802097e1a1b", "name": "network-vif-deleted", "tag": "e12ff0e5-7520-4d3f-b645-8686e263d5b8"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:41:13.573895 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-d34f9801-1181-4106-99d7-a2b4e53a07b4 req-69a540d7-1751-4d9b-afaf-0e715660958a service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:13.574369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-d34f9801-1181-4106-99d7-a2b4e53a07b4 req-69a540d7-1751-4d9b-afaf-0e715660958a service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:13.574795 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-d34f9801-1181-4106-99d7-a2b4e53a07b4 req-69a540d7-1751-4d9b-afaf-0e715660958a service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:13.586249 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-d34f9801-1181-4106-99d7-a2b4e53a07b4 req-69a540d7-1751-4d9b-afaf-0e715660958a service nova] Creating event network-vif-deleted:e12ff0e5-7520-4d3f-b645-8686e263d5b8 for instance 5aa21554-b4b9-49b1-9fd4-d802097e1a1b on hv-cinder-maste Aug 28 01:41:13.591469 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-d34f9801-1181-4106-99d7-a2b4e53a07b4 req-69a540d7-1751-4d9b-afaf-0e715660958a service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032695 Aug 28 01:41:13.592241 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1822/3415] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:41:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:41:14.414383 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-225eea04-9bc9-4753-a219-2aa173bb8f5f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:41:14.415145 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-225eea04-9bc9-4753-a219-2aa173bb8f5f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] [instance: 5aa21554-b4b9-49b1-9fd4-d802097e1a1b] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:41:14.421864 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-225eea04-9bc9-4753-a219-2aa173bb8f5f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:14.422371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-225eea04-9bc9-4753-a219-2aa173bb8f5f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:14.422795 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-225eea04-9bc9-4753-a219-2aa173bb8f5f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:14.423471 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-225eea04-9bc9-4753-a219-2aa173bb8f5f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:14.423905 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-225eea04-9bc9-4753-a219-2aa173bb8f5f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:14.424321 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-225eea04-9bc9-4753-a219-2aa173bb8f5f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:14.438157 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-225eea04-9bc9-4753-a219-2aa173bb8f5f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] HTTP exception thrown: Instance 5aa21554-b4b9-49b1-9fd4-d802097e1a1b could not be found. Aug 28 01:41:14.439904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-225eea04-9bc9-4753-a219-2aa173bb8f5f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Request method failure captured: Aug 28 01:41:14.439904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b HTTP/1.1 Aug 28 01:41:14.439904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:41:14.439904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:41:14.439904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:41:14.439904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:41:14.439904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:41:14.439904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:41:14.439904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:41:14.439904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:41:14.439904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCsaiYjx6hkP8LalTlGUkXaedL4hodMc863R9Tte9X_auHcbo76Kw8uWZpCrs_okviykBPDCBYTN8snp6d8SEsTbN8i4uh5mRzPzY0swXNUjza4optXT00Gz6n3hFLiCdtFen8as7W6KRAB6d6NmZJHO7e0zT87yWC_3BEl-D3idW_zI Aug 28 01:41:14.439904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:41:14.439904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:41:14.439904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:41:14.439904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:41:14.439904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:41:14.439904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:41:14.439904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: 8bd4256c51f64ef9aa0f21ea2fcb55e7 Aug 28 01:41:14.439904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-TestMinimumBasicScenario-1487453377 Aug 28 01:41:14.439904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:41:14.439904 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:41:14.441254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_8bd4256c51f64ef9aa0f21ea2fcb55e7", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/8bd4256c51f64ef9aa0f21ea2fcb55e7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/8bd4256c51f64ef9aa0f21ea2fcb55e7"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/8bd4256c51f64ef9aa0f21ea2fcb55e7"}]}] Aug 28 01:41:14.441254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-TestMinimumBasicScenario-1487453377 Aug 28 01:41:14.441254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: 8bd4256c51f64ef9aa0f21ea2fcb55e7 Aug 28 01:41:14.441254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1487453377 Aug 28 01:41:14.441254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-TestMinimumBasicScenario-1487453377-project Aug 28 01:41:14.441254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:41:14.441254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:41:14.441254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: c5bc73cffe1e4fa095051e109b5f0d0c Aug 28 01:41:14.441254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-TestMinimumBasicScenario-1487453377-project Aug 28 01:41:14.441254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:41:14.441254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Instance 5aa21554-b4b9-49b1-9fd4-d802097e1a1b could not be found. Aug 28 01:41:14.441254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'id': '5aa21554-b4b9-49b1-9fd4-d802097e1a1b'} Aug 28 01:41:14.441959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:41:14.441959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:41:14.441959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:41:14.441959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:41:14.441959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:41:14.441959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:41:14.441959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:41:14.441959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:41:14.441959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:41:14.441959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:41:14.441959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5aa21554-b4b9-49b1-9fd4-d802097e1a1b could not be found. Aug 28 01:41:14.441959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:41:14.441959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:41:14.441959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:41:14.441959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:41:14.441959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:41:14.441959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:41:14.441959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:41:14.441959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:41:14.441959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:41:14.443322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:41:14.443322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 01:41:14.443322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 01:41:14.443322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:41:14.443322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:41:14.443322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:41:14.443322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:41:14.443322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5aa21554-b4b9-49b1-9fd4-d802097e1a1b could not be found. Aug 28 01:41:14.443322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:41:14.443322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:41:14.443322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:41:14.443322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:41:14.443322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:41:14.443322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:41:14.443322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:41:14.443322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:41:14.443322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5aa21554-b4b9-49b1-9fd4-d802097e1a1b could not be found. Aug 28 01:41:14.443322 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:41:14.444615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-225eea04-9bc9-4753-a219-2aa173bb8f5f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Returning 404 to user: Instance 5aa21554-b4b9-49b1-9fd4-d802097e1a1b could not be found. {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:41:14.444615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-225eea04-9bc9-4753-a219-2aa173bb8f5f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b" status: 404 len: 111 microversion: 2.1 time: 0.032568 Aug 28 01:41:14.444615 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1594/3416] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:41:14 2022] GET /compute/v2.1/servers/5aa21554-b4b9-49b1-9fd4-d802097e1a1b => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:41:14.451134 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-38e50df6-ea33-4a70-be5a-538a93061f2f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] Calling method '.version_select at 0x7f0e720d5e50>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:41:14.476948 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-38e50df6-ea33-4a70-be5a-538a93061f2f tempest-TestMinimumBasicScenario-1487453377 tempest-TestMinimumBasicScenario-1487453377-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1223840802" status: 202 len: 0 microversion: 2.1 time: 0.029941 Aug 28 01:41:14.477599 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1823/3417] 10.222.0.42 () {58 vars in 1272 bytes} [Sun Aug 28 01:41:14 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1223840802 => generated 0 bytes in 32 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 01:41:32.457445 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=14 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:41:32.458186 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=34 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:41:32.665801 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f0e6dfd2-e95c-40c3-b323-c51832d98d3b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Action: 'create', calling method: .version_select at 0x7f0e720993a0>, body: {"keypair": {"name": "tempest-TestShelveInstance-1329873231", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAvuQ4etdORJpEcsn5n+24UrlJ//J3hHh1qsrvjE2AHVy0yN+q1CxvoMogXwqCDcAgkZIVzPRaM8Qzyf9tceFuYarkSPHvMdZxGYCUh8dfqoETrplwPNlYtjN/Z+RkABgg=="}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:41:32.681724 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-f0e6dfd2-e95c-40c3-b323-c51832d98d3b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Getting quotas for project b4ead68269cb4e19befec4146c93686d. Resources: {'key_pairs'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 01:41:32.686207 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.quota [None req-f0e6dfd2-e95c-40c3-b323-c51832d98d3b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Getting quotas for user 41a51cf16450452194671aa46f4f06c4 and project b4ead68269cb4e19befec4146c93686d. Resources: {'key_pairs'} {{(pid=120524) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 01:41:32.723604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f0e6dfd2-e95c-40c3-b323-c51832d98d3b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.271652 Aug 28 01:41:32.724563 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1595/3418] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Aug 28 01:41:32 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:41:57.687028 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=43,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:41:57.687778 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=22,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:41:57.691946 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-92048e91-dab0-4c8a-a5d1-c6e6f74e1ed7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-161325520", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1329873231", "security_groups": [{"name": "tempest-secgroup-smoke-1633188442"}], "networks": [{"uuid": "92119e62-e47d-4116-a754-e5fe0b14d661"}], "block_device_mapping_v2": [{"uuid": "42163888-a12c-44a4-9bef-e41e8997f577", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:41:57.820754 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.network.neutron [None req-92048e91-dab0-4c8a-a5d1-c6e6f74e1ed7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] validate_networks() for [('92119e62-e47d-4116-a754-e5fe0b14d661', None, None, None, None, None)] {{(pid=120525) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Aug 28 01:41:58.356634 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-92048e91-dab0-4c8a-a5d1-c6e6f74e1ed7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Flavor limits 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Aug 28 01:41:58.357093 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-92048e91-dab0-4c8a-a5d1-c6e6f74e1ed7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Image limits 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Aug 28 01:41:58.357580 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-92048e91-dab0-4c8a-a5d1-c6e6f74e1ed7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Flavor pref 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Aug 28 01:41:58.357964 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-92048e91-dab0-4c8a-a5d1-c6e6f74e1ed7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Image pref 0:0:0 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Aug 28 01:41:58.358395 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.virt.hardware [None req-92048e91-dab0-4c8a-a5d1-c6e6f74e1ed7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120525) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Aug 28 01:41:58.367757 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92048e91-dab0-4c8a-a5d1-c6e6f74e1ed7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:58.368250 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92048e91-dab0-4c8a-a5d1-c6e6f74e1ed7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:58.368792 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92048e91-dab0-4c8a-a5d1-c6e6f74e1ed7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:58.369492 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92048e91-dab0-4c8a-a5d1-c6e6f74e1ed7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:58.369871 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92048e91-dab0-4c8a-a5d1-c6e6f74e1ed7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:58.370364 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-92048e91-dab0-4c8a-a5d1-c6e6f74e1ed7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:58.389042 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-92048e91-dab0-4c8a-a5d1-c6e6f74e1ed7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Getting quotas for project b4ead68269cb4e19befec4146c93686d. Resources: {'ram', 'instances', 'cores'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:393}} Aug 28 01:41:58.394076 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.quota [None req-92048e91-dab0-4c8a-a5d1-c6e6f74e1ed7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Getting quotas for user 41a51cf16450452194671aa46f4f06c4 and project b4ead68269cb4e19befec4146c93686d. Resources: {'ram', 'instances', 'cores'} {{(pid=120525) _get_quotas /opt/stack/nova/nova/quota.py:383}} Aug 28 01:41:58.456651 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-92048e91-dab0-4c8a-a5d1-c6e6f74e1ed7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Going to run 1 instances... {{(pid=120525) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Aug 28 01:41:58.458936 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-92048e91-dab0-4c8a-a5d1-c6e6f74e1ed7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='42163888-a12c-44a4-9bef-e41e8997f577',volume_size=None,volume_type=None)] {{(pid=120525) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Aug 28 01:41:58.612625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-92048e91-dab0-4c8a-a5d1-c6e6f74e1ed7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.928107 Aug 28 01:41:58.613637 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1824/3419] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Aug 28 01:41:57 2022] POST /compute/v2.1/servers => generated 406 bytes in 930 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Aug 28 01:41:58.625247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-831a78c0-4c96-45a4-accf-1ca510f29396 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:41:58.625960 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-831a78c0-4c96-45a4-accf-1ca510f29396 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:41:58.629659 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=4 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:41:58.630405 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=11,nova_api:INSERT=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:41:58.704121 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-831a78c0-4c96-45a4-accf-1ca510f29396 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:41:58.711197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-831a78c0-4c96-45a4-accf-1ca510f29396 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1088 microversion: 2.1 time: 0.089807 Aug 28 01:41:58.711759 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1596/3420] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:41:58 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1088 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:41:59.744826 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-d77fb7b7-f8fc-4fc6-8106-765659d214e7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:41:59.745824 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-d77fb7b7-f8fc-4fc6-8106-765659d214e7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:41:59.755741 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d77fb7b7-f8fc-4fc6-8106-765659d214e7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:59.756223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d77fb7b7-f8fc-4fc6-8106-765659d214e7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:59.756645 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d77fb7b7-f8fc-4fc6-8106-765659d214e7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:59.757368 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d77fb7b7-f8fc-4fc6-8106-765659d214e7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:41:59.757840 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d77fb7b7-f8fc-4fc6-8106-765659d214e7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:41:59.758259 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-d77fb7b7-f8fc-4fc6-8106-765659d214e7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:41:59.827150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-d77fb7b7-f8fc-4fc6-8106-765659d214e7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:41:59.835433 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-d77fb7b7-f8fc-4fc6-8106-765659d214e7 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1186 microversion: 2.1 time: 0.096393 Aug 28 01:41:59.836790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1825/3421] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:41:59 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1186 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:42:00.857215 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-1c4f933a-3adf-4bba-b7e6-4112a3e78ac4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:42:00.857811 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-1c4f933a-3adf-4bba-b7e6-4112a3e78ac4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:42:00.864558 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c4f933a-3adf-4bba-b7e6-4112a3e78ac4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:00.865099 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c4f933a-3adf-4bba-b7e6-4112a3e78ac4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:00.865530 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c4f933a-3adf-4bba-b7e6-4112a3e78ac4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:00.866221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c4f933a-3adf-4bba-b7e6-4112a3e78ac4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:00.866667 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c4f933a-3adf-4bba-b7e6-4112a3e78ac4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:00.867085 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-1c4f933a-3adf-4bba-b7e6-4112a3e78ac4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:01.132718 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-1c4f933a-3adf-4bba-b7e6-4112a3e78ac4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:42:01.142673 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-1c4f933a-3adf-4bba-b7e6-4112a3e78ac4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1260 microversion: 2.1 time: 0.289257 Aug 28 01:42:01.143259 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1597/3422] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:42:00 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1260 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:42:01.478966 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-c651a4a8-7798-44ca-8ddf-50efa5f91a9e req-62143960-cdcb-4039-ae7a-272b9bf2e214 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c", "tag": "3469e5fa-3521-4b70-8e0b-f9edded0bf75"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:42:01.490479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-c651a4a8-7798-44ca-8ddf-50efa5f91a9e req-62143960-cdcb-4039-ae7a-272b9bf2e214 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:01.490953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-c651a4a8-7798-44ca-8ddf-50efa5f91a9e req-62143960-cdcb-4039-ae7a-272b9bf2e214 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:01.491374 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-c651a4a8-7798-44ca-8ddf-50efa5f91a9e req-62143960-cdcb-4039-ae7a-272b9bf2e214 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:01.503559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-c651a4a8-7798-44ca-8ddf-50efa5f91a9e req-62143960-cdcb-4039-ae7a-272b9bf2e214 service nova] Creating event network-changed:3469e5fa-3521-4b70-8e0b-f9edded0bf75 for instance ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c on hv-cinder-maste Aug 28 01:42:01.509010 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-c651a4a8-7798-44ca-8ddf-50efa5f91a9e req-62143960-cdcb-4039-ae7a-272b9bf2e214 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034634 Aug 28 01:42:01.509805 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1826/3423] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:42:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:42:02.163943 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5e822c47-fd7d-4e19-86c8-364f83c33aea tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:42:02.164595 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5e822c47-fd7d-4e19-86c8-364f83c33aea tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:42:02.171256 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e822c47-fd7d-4e19-86c8-364f83c33aea tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:02.171727 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e822c47-fd7d-4e19-86c8-364f83c33aea tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:02.172175 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e822c47-fd7d-4e19-86c8-364f83c33aea tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:02.172890 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e822c47-fd7d-4e19-86c8-364f83c33aea tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:02.173380 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e822c47-fd7d-4e19-86c8-364f83c33aea tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:02.173794 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5e822c47-fd7d-4e19-86c8-364f83c33aea tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:02.314036 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5e822c47-fd7d-4e19-86c8-364f83c33aea tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:42:02.323103 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5e822c47-fd7d-4e19-86c8-364f83c33aea tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1260 microversion: 2.1 time: 0.162916 Aug 28 01:42:02.323671 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1598/3424] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:42:02 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1260 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:42:03.346919 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-f0a8fe5e-6ea4-4010-89ef-947078bafafb tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:42:03.347650 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-f0a8fe5e-6ea4-4010-89ef-947078bafafb tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:42:03.357166 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0a8fe5e-6ea4-4010-89ef-947078bafafb tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:03.357790 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0a8fe5e-6ea4-4010-89ef-947078bafafb tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:03.358428 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0a8fe5e-6ea4-4010-89ef-947078bafafb tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:03.359496 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0a8fe5e-6ea4-4010-89ef-947078bafafb tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:03.360352 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0a8fe5e-6ea4-4010-89ef-947078bafafb tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:03.360792 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-f0a8fe5e-6ea4-4010-89ef-947078bafafb tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:03.581701 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-f0a8fe5e-6ea4-4010-89ef-947078bafafb tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:42:03.589891 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-f0a8fe5e-6ea4-4010-89ef-947078bafafb tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1260 microversion: 2.1 time: 0.249195 Aug 28 01:42:03.590481 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1827/3425] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:42:03 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1260 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:42:04.610959 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-998523e8-4ef0-4c86-9224-652c725fc8f2 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:42:04.611575 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-998523e8-4ef0-4c86-9224-652c725fc8f2 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:42:04.618347 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-998523e8-4ef0-4c86-9224-652c725fc8f2 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:04.618826 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-998523e8-4ef0-4c86-9224-652c725fc8f2 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:04.619258 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-998523e8-4ef0-4c86-9224-652c725fc8f2 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:04.619942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-998523e8-4ef0-4c86-9224-652c725fc8f2 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:04.620392 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-998523e8-4ef0-4c86-9224-652c725fc8f2 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:04.620788 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-998523e8-4ef0-4c86-9224-652c725fc8f2 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:04.767829 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-998523e8-4ef0-4c86-9224-652c725fc8f2 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:42:04.776247 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-998523e8-4ef0-4c86-9224-652c725fc8f2 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1260 microversion: 2.1 time: 0.169246 Aug 28 01:42:04.776821 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1599/3426] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:42:04 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1260 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:42:05.802686 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9b9a6db3-0ec7-43bb-bce8-31431d567a42 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:42:05.803718 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9b9a6db3-0ec7-43bb-bce8-31431d567a42 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:42:05.814674 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b9a6db3-0ec7-43bb-bce8-31431d567a42 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:05.815125 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b9a6db3-0ec7-43bb-bce8-31431d567a42 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:05.815590 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b9a6db3-0ec7-43bb-bce8-31431d567a42 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:05.816277 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b9a6db3-0ec7-43bb-bce8-31431d567a42 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:05.816726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b9a6db3-0ec7-43bb-bce8-31431d567a42 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:05.817193 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9b9a6db3-0ec7-43bb-bce8-31431d567a42 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:05.959529 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9b9a6db3-0ec7-43bb-bce8-31431d567a42 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:42:05.970629 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9b9a6db3-0ec7-43bb-bce8-31431d567a42 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1260 microversion: 2.1 time: 0.174519 Aug 28 01:42:05.970913 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1828/3427] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:42:05 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1260 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:42:06.991211 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-3ca33a36-b2fb-4109-b67a-659218127d6d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:42:06.991837 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-3ca33a36-b2fb-4109-b67a-659218127d6d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:42:06.998769 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ca33a36-b2fb-4109-b67a-659218127d6d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:06.999259 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ca33a36-b2fb-4109-b67a-659218127d6d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:06.999684 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ca33a36-b2fb-4109-b67a-659218127d6d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:07.000372 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ca33a36-b2fb-4109-b67a-659218127d6d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:07.000838 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ca33a36-b2fb-4109-b67a-659218127d6d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:07.001295 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-3ca33a36-b2fb-4109-b67a-659218127d6d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:07.148651 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-3ca33a36-b2fb-4109-b67a-659218127d6d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:42:07.156857 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-3ca33a36-b2fb-4109-b67a-659218127d6d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1260 microversion: 2.1 time: 0.169533 Aug 28 01:42:07.157516 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1600/3428] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:42:06 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1260 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:42:08.177947 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-12f8f704-b199-4728-8517-f57db61c3467 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:42:08.178525 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-12f8f704-b199-4728-8517-f57db61c3467 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:42:08.186607 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12f8f704-b199-4728-8517-f57db61c3467 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:08.187097 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12f8f704-b199-4728-8517-f57db61c3467 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:08.187530 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12f8f704-b199-4728-8517-f57db61c3467 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:08.188401 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12f8f704-b199-4728-8517-f57db61c3467 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:08.189032 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12f8f704-b199-4728-8517-f57db61c3467 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:08.189458 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-12f8f704-b199-4728-8517-f57db61c3467 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:08.336732 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-12f8f704-b199-4728-8517-f57db61c3467 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:42:08.345162 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-12f8f704-b199-4728-8517-f57db61c3467 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1260 microversion: 2.1 time: 0.171152 Aug 28 01:42:08.345733 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1829/3429] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:42:08 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1260 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:42:09.366759 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-0274a74f-1a21-44bd-8922-8e656cf5434a tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:42:09.367343 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-0274a74f-1a21-44bd-8922-8e656cf5434a tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:42:09.370242 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell0:SELECT=2 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:42:09.375979 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0274a74f-1a21-44bd-8922-8e656cf5434a tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:09.376454 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0274a74f-1a21-44bd-8922-8e656cf5434a tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:09.376880 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0274a74f-1a21-44bd-8922-8e656cf5434a tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:09.377585 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0274a74f-1a21-44bd-8922-8e656cf5434a tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:09.378029 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0274a74f-1a21-44bd-8922-8e656cf5434a tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:09.378448 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-0274a74f-1a21-44bd-8922-8e656cf5434a tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:09.518124 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-0274a74f-1a21-44bd-8922-8e656cf5434a tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:42:09.526228 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-0274a74f-1a21-44bd-8922-8e656cf5434a tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1260 microversion: 2.1 time: 0.163527 Aug 28 01:42:09.526800 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1601/3430] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:42:09 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1260 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:42:10.550857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-3d296ae0-4518-4656-9050-6d8ba96dfc6f tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:42:10.551502 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-3d296ae0-4518-4656-9050-6d8ba96dfc6f tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:42:10.553507 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=2 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:42:10.554143 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell0:SELECT=6 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:42:10.560858 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d296ae0-4518-4656-9050-6d8ba96dfc6f tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:10.561378 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d296ae0-4518-4656-9050-6d8ba96dfc6f tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:10.561808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d296ae0-4518-4656-9050-6d8ba96dfc6f tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:10.562564 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d296ae0-4518-4656-9050-6d8ba96dfc6f tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:10.563134 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d296ae0-4518-4656-9050-6d8ba96dfc6f tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:10.563583 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-3d296ae0-4518-4656-9050-6d8ba96dfc6f tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:10.707615 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-3d296ae0-4518-4656-9050-6d8ba96dfc6f tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:42:10.715830 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-3d296ae0-4518-4656-9050-6d8ba96dfc6f tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1260 microversion: 2.1 time: 0.171458 Aug 28 01:42:10.716403 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1830/3431] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:42:10 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1260 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:42:11.737732 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-407df3f1-f478-4959-b0d1-3ed227ca1d52 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:42:11.738411 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-407df3f1-f478-4959-b0d1-3ed227ca1d52 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:42:11.746304 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-407df3f1-f478-4959-b0d1-3ed227ca1d52 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:11.746848 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-407df3f1-f478-4959-b0d1-3ed227ca1d52 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:11.747336 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-407df3f1-f478-4959-b0d1-3ed227ca1d52 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:11.748147 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-407df3f1-f478-4959-b0d1-3ed227ca1d52 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:11.748648 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-407df3f1-f478-4959-b0d1-3ed227ca1d52 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:11.749136 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-407df3f1-f478-4959-b0d1-3ed227ca1d52 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:11.885656 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-407df3f1-f478-4959-b0d1-3ed227ca1d52 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:42:11.893820 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-407df3f1-f478-4959-b0d1-3ed227ca1d52 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1260 microversion: 2.1 time: 0.160294 Aug 28 01:42:11.894395 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1602/3432] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:42:11 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1260 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:42:12.915728 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a8f0dc3d-f534-412d-a9f8-3f8ad5259074 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:42:12.916321 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-a8f0dc3d-f534-412d-a9f8-3f8ad5259074 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:42:12.923155 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a8f0dc3d-f534-412d-a9f8-3f8ad5259074 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:12.923644 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a8f0dc3d-f534-412d-a9f8-3f8ad5259074 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:12.924072 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a8f0dc3d-f534-412d-a9f8-3f8ad5259074 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:12.924758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a8f0dc3d-f534-412d-a9f8-3f8ad5259074 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:12.925225 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a8f0dc3d-f534-412d-a9f8-3f8ad5259074 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:12.925644 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-a8f0dc3d-f534-412d-a9f8-3f8ad5259074 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:13.085190 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-a8f0dc3d-f534-412d-a9f8-3f8ad5259074 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:42:13.093407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a8f0dc3d-f534-412d-a9f8-3f8ad5259074 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1260 microversion: 2.1 time: 0.181575 Aug 28 01:42:13.093975 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1831/3433] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:42:12 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1260 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:42:13.980484 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-2e7a5586-e7bd-4f79-8aa7-9e783db2e340 req-6a927587-25ee-43fb-862d-97c2122f913a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c", "name": "network-vif-plugged", "status": "completed", "tag": "3469e5fa-3521-4b70-8e0b-f9edded0bf75"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:42:13.990387 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-2e7a5586-e7bd-4f79-8aa7-9e783db2e340 req-6a927587-25ee-43fb-862d-97c2122f913a service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:13.991083 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-2e7a5586-e7bd-4f79-8aa7-9e783db2e340 req-6a927587-25ee-43fb-862d-97c2122f913a service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:13.991505 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-2e7a5586-e7bd-4f79-8aa7-9e783db2e340 req-6a927587-25ee-43fb-862d-97c2122f913a service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:14.005360 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-2e7a5586-e7bd-4f79-8aa7-9e783db2e340 req-6a927587-25ee-43fb-862d-97c2122f913a service nova] Creating event network-vif-plugged:3469e5fa-3521-4b70-8e0b-f9edded0bf75 for instance ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c on hv-cinder-maste Aug 28 01:42:14.010465 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-2e7a5586-e7bd-4f79-8aa7-9e783db2e340 req-6a927587-25ee-43fb-862d-97c2122f913a service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.091685 Aug 28 01:42:14.011051 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1603/3434] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:42:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:42:14.119920 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9d374af6-408b-464c-a066-4101dfe11630 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:42:14.120498 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9d374af6-408b-464c-a066-4101dfe11630 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:42:14.127569 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d374af6-408b-464c-a066-4101dfe11630 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:14.128049 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d374af6-408b-464c-a066-4101dfe11630 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:14.128479 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d374af6-408b-464c-a066-4101dfe11630 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:14.129209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d374af6-408b-464c-a066-4101dfe11630 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:14.129655 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d374af6-408b-464c-a066-4101dfe11630 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:14.130103 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9d374af6-408b-464c-a066-4101dfe11630 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:14.294391 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9d374af6-408b-464c-a066-4101dfe11630 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:42:14.304129 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9d374af6-408b-464c-a066-4101dfe11630 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1260 microversion: 2.1 time: 0.188051 Aug 28 01:42:14.304702 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1832/3435] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:42:14 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1260 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:42:15.325931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-d0bb70a6-ae13-42ba-b2d2-b02d1834c115 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:42:15.326621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-d0bb70a6-ae13-42ba-b2d2-b02d1834c115 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:42:15.333661 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0bb70a6-ae13-42ba-b2d2-b02d1834c115 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:15.334141 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0bb70a6-ae13-42ba-b2d2-b02d1834c115 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:15.334572 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0bb70a6-ae13-42ba-b2d2-b02d1834c115 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:15.335267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0bb70a6-ae13-42ba-b2d2-b02d1834c115 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:15.335718 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0bb70a6-ae13-42ba-b2d2-b02d1834c115 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:15.336112 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-d0bb70a6-ae13-42ba-b2d2-b02d1834c115 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:15.476308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-d0bb70a6-ae13-42ba-b2d2-b02d1834c115 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:42:15.484389 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-d0bb70a6-ae13-42ba-b2d2-b02d1834c115 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1260 microversion: 2.1 time: 0.163245 Aug 28 01:42:15.485030 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1604/3436] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:42:15 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1260 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:42:16.506277 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-912cf6bb-b2ea-4af6-9794-ffff99c3e63d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:42:16.506956 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-912cf6bb-b2ea-4af6-9794-ffff99c3e63d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:42:16.514633 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-912cf6bb-b2ea-4af6-9794-ffff99c3e63d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:16.515162 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-912cf6bb-b2ea-4af6-9794-ffff99c3e63d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:16.515640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-912cf6bb-b2ea-4af6-9794-ffff99c3e63d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:16.516407 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-912cf6bb-b2ea-4af6-9794-ffff99c3e63d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:16.516907 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-912cf6bb-b2ea-4af6-9794-ffff99c3e63d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:16.517386 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-912cf6bb-b2ea-4af6-9794-ffff99c3e63d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:16.661744 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-912cf6bb-b2ea-4af6-9794-ffff99c3e63d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:42:16.673608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-912cf6bb-b2ea-4af6-9794-ffff99c3e63d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1443 microversion: 2.1 time: 0.171345 Aug 28 01:42:16.674383 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1833/3437] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:42:16 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1443 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:42:16.695471 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-86f88d68-2e94-4e45-8b66-99746f41b9fa tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:42:16.696046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-86f88d68-2e94-4e45-8b66-99746f41b9fa tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:42:16.702870 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86f88d68-2e94-4e45-8b66-99746f41b9fa tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:16.703359 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86f88d68-2e94-4e45-8b66-99746f41b9fa tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:16.703776 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86f88d68-2e94-4e45-8b66-99746f41b9fa tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:16.704454 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86f88d68-2e94-4e45-8b66-99746f41b9fa tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:16.704899 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86f88d68-2e94-4e45-8b66-99746f41b9fa tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:16.705343 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-86f88d68-2e94-4e45-8b66-99746f41b9fa tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:16.840110 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-86f88d68-2e94-4e45-8b66-99746f41b9fa tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:42:16.851317 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-86f88d68-2e94-4e45-8b66-99746f41b9fa tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1443 microversion: 2.1 time: 0.159663 Aug 28 01:42:16.851879 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1605/3438] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:42:16 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1443 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:42:18.797230 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-ddf31dbf-257b-4ca9-9fa3-a4c60058f2cf req-fb003197-b40e-42ba-8ffb-d45149a4b82c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c", "tag": "3469e5fa-3521-4b70-8e0b-f9edded0bf75"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:42:18.807763 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-ddf31dbf-257b-4ca9-9fa3-a4c60058f2cf req-fb003197-b40e-42ba-8ffb-d45149a4b82c service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:42:18.808222 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-ddf31dbf-257b-4ca9-9fa3-a4c60058f2cf req-fb003197-b40e-42ba-8ffb-d45149a4b82c service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:42:18.808640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-ddf31dbf-257b-4ca9-9fa3-a4c60058f2cf req-fb003197-b40e-42ba-8ffb-d45149a4b82c service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:42:18.820353 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-ddf31dbf-257b-4ca9-9fa3-a4c60058f2cf req-fb003197-b40e-42ba-8ffb-d45149a4b82c service nova] Creating event network-changed:3469e5fa-3521-4b70-8e0b-f9edded0bf75 for instance ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c on hv-cinder-maste Aug 28 01:42:18.825613 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-ddf31dbf-257b-4ca9-9fa3-a4c60058f2cf req-fb003197-b40e-42ba-8ffb-d45149a4b82c service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032754 Aug 28 01:42:18.826307 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1834/3439] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:42:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:44:32.651337 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Aug 28 01:44:32.651990 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Sun Aug 28 01:44:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c/action (ip 10.222.0.42) !!! Aug 28 01:44:32.653358 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 28 01:44:32.653514 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Sun Aug 28 01:44:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c/action (ip 10.222.0.42) !!! Aug 28 01:44:32.655032 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 28 01:44:32.655201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Sun Aug 28 01:44:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c/action (ip 10.222.0.42) !!! Aug 28 01:44:32.656138 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=22 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:44:32.656873 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=50 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:44:32.661699 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-90fbf928-96c5-4c3b-95ef-535794881c72 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:44:32.662254 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-90fbf928-96c5-4c3b-95ef-535794881c72 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:32.673046 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90fbf928-96c5-4c3b-95ef-535794881c72 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:32.673522 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90fbf928-96c5-4c3b-95ef-535794881c72 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:32.673942 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90fbf928-96c5-4c3b-95ef-535794881c72 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:32.674617 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90fbf928-96c5-4c3b-95ef-535794881c72 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:32.675058 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90fbf928-96c5-4c3b-95ef-535794881c72 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:32.675469 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-90fbf928-96c5-4c3b-95ef-535794881c72 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:32.699939 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.objects.instance [None req-90fbf928-96c5-4c3b-95ef-535794881c72 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lazy-loading 'flavor' on Instance uuid ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c {{(pid=120524) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Aug 28 01:44:32.790931 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-90fbf928-96c5-4c3b-95ef-535794881c72 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "POST /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c/action" status: 202 len: 0 microversion: 2.1 time: 0.141596 Aug 28 01:44:32.791692 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1606/3440] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Aug 28 01:44:32 2022] POST /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c/action => generated 0 bytes in 144 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 01:44:32.798547 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: ERROR oslo.messaging._drivers.impl_rabbit [-] [c920d2e8-ed1b-49b0-a050-78ee4a07b0a8] AMQP server on 10.222.0.42:5672 is unreachable: Too many heartbeats missed. Trying again in 1 seconds.: amqp.exceptions.ConnectionForced: Too many heartbeats missed Aug 28 01:44:32.800201 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Too many heartbeats missed Aug 28 01:44:32.801351 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 28 01:44:32.801467 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Sun Aug 28 01:44:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c (ip 10.222.0.42) !!! Aug 28 01:44:32.803065 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=35,nova_api:INSERT=3 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:44:32.803457 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=54 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:44:32.809242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-9cd9cad5-def1-40cf-836a-172bcabc1a9e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:32.809984 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-9cd9cad5-def1-40cf-836a-172bcabc1a9e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:32.819712 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9cd9cad5-def1-40cf-836a-172bcabc1a9e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:32.820238 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9cd9cad5-def1-40cf-836a-172bcabc1a9e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:32.820668 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9cd9cad5-def1-40cf-836a-172bcabc1a9e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:32.821413 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9cd9cad5-def1-40cf-836a-172bcabc1a9e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:32.821870 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9cd9cad5-def1-40cf-836a-172bcabc1a9e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:32.822288 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-9cd9cad5-def1-40cf-836a-172bcabc1a9e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:32.962576 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-9cd9cad5-def1-40cf-836a-172bcabc1a9e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:32.970667 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-9cd9cad5-def1-40cf-836a-172bcabc1a9e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1568 microversion: 2.1 time: 0.174170 Aug 28 01:44:32.971242 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1835/3441] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:32 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1568 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:33.993079 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-2ca13005-c913-4574-8619-9388ce355663 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:33.993760 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-2ca13005-c913-4574-8619-9388ce355663 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:34.000018 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ca13005-c913-4574-8619-9388ce355663 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:34.000509 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ca13005-c913-4574-8619-9388ce355663 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:34.000934 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ca13005-c913-4574-8619-9388ce355663 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:34.001636 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ca13005-c913-4574-8619-9388ce355663 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:34.002079 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ca13005-c913-4574-8619-9388ce355663 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:34.002493 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-2ca13005-c913-4574-8619-9388ce355663 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:34.149337 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-2ca13005-c913-4574-8619-9388ce355663 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:34.157760 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-2ca13005-c913-4574-8619-9388ce355663 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1568 microversion: 2.1 time: 0.169644 Aug 28 01:44:34.158311 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1607/3442] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:33 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1568 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:35.179168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-e7a34726-35eb-40a2-9ae1-4cd58e705d4e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:35.179786 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-e7a34726-35eb-40a2-9ae1-4cd58e705d4e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:35.187758 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e7a34726-35eb-40a2-9ae1-4cd58e705d4e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:35.188235 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e7a34726-35eb-40a2-9ae1-4cd58e705d4e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:35.188666 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e7a34726-35eb-40a2-9ae1-4cd58e705d4e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:35.189728 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e7a34726-35eb-40a2-9ae1-4cd58e705d4e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:35.190155 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e7a34726-35eb-40a2-9ae1-4cd58e705d4e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:35.190551 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-e7a34726-35eb-40a2-9ae1-4cd58e705d4e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:35.197301 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO oslo.messaging._drivers.impl_rabbit [-] [c920d2e8-ed1b-49b0-a050-78ee4a07b0a8] Reconnected to AMQP server on 10.222.0.42:5672 via [amqp] client with port 54208. Aug 28 01:44:35.341209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-e7a34726-35eb-40a2-9ae1-4cd58e705d4e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:35.349328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-e7a34726-35eb-40a2-9ae1-4cd58e705d4e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1568 microversion: 2.1 time: 0.174032 Aug 28 01:44:35.349903 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1836/3443] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:35 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1568 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:36.372162 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b4835c80-033f-4ea6-bb2e-c1426662f5fd tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:36.372759 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b4835c80-033f-4ea6-bb2e-c1426662f5fd tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:36.380791 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4835c80-033f-4ea6-bb2e-c1426662f5fd tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:36.381371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4835c80-033f-4ea6-bb2e-c1426662f5fd tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:36.381894 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4835c80-033f-4ea6-bb2e-c1426662f5fd tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:36.382697 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4835c80-033f-4ea6-bb2e-c1426662f5fd tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:36.383306 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4835c80-033f-4ea6-bb2e-c1426662f5fd tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:36.383866 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b4835c80-033f-4ea6-bb2e-c1426662f5fd tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:36.515571 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b4835c80-033f-4ea6-bb2e-c1426662f5fd tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:36.523896 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b4835c80-033f-4ea6-bb2e-c1426662f5fd tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1568 microversion: 2.1 time: 0.156636 Aug 28 01:44:36.524436 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1608/3444] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:36 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1568 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:37.557784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-185be407-6e91-4b7a-bdb5-67869e8bbf26 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:37.559272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-185be407-6e91-4b7a-bdb5-67869e8bbf26 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:37.571369 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-185be407-6e91-4b7a-bdb5-67869e8bbf26 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:37.572121 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-185be407-6e91-4b7a-bdb5-67869e8bbf26 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:37.572876 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-185be407-6e91-4b7a-bdb5-67869e8bbf26 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:37.574104 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-185be407-6e91-4b7a-bdb5-67869e8bbf26 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:37.574898 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-185be407-6e91-4b7a-bdb5-67869e8bbf26 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:37.576515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-185be407-6e91-4b7a-bdb5-67869e8bbf26 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:37.795722 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-185be407-6e91-4b7a-bdb5-67869e8bbf26 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:37.804011 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-185be407-6e91-4b7a-bdb5-67869e8bbf26 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1568 microversion: 2.1 time: 0.257911 Aug 28 01:44:37.804585 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1837/3445] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:37 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1568 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:38.828371 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-384aa3f0-a696-44aa-ba90-bd0f379e8e58 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:38.829123 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-384aa3f0-a696-44aa-ba90-bd0f379e8e58 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:38.838218 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-384aa3f0-a696-44aa-ba90-bd0f379e8e58 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:38.838808 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-384aa3f0-a696-44aa-ba90-bd0f379e8e58 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:38.839380 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-384aa3f0-a696-44aa-ba90-bd0f379e8e58 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:38.840434 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-384aa3f0-a696-44aa-ba90-bd0f379e8e58 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:38.840910 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-384aa3f0-a696-44aa-ba90-bd0f379e8e58 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:38.841583 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-384aa3f0-a696-44aa-ba90-bd0f379e8e58 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:38.987591 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-384aa3f0-a696-44aa-ba90-bd0f379e8e58 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:38.995998 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-384aa3f0-a696-44aa-ba90-bd0f379e8e58 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1568 microversion: 2.1 time: 0.173709 Aug 28 01:44:38.996562 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1609/3446] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:38 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1568 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:40.025574 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-1ccd4104-07f0-4ce6-9998-1f48ce223ac3 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:40.026328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-1ccd4104-07f0-4ce6-9998-1f48ce223ac3 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:40.034049 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ccd4104-07f0-4ce6-9998-1f48ce223ac3 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:40.034588 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ccd4104-07f0-4ce6-9998-1f48ce223ac3 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:40.035070 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ccd4104-07f0-4ce6-9998-1f48ce223ac3 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:40.035812 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ccd4104-07f0-4ce6-9998-1f48ce223ac3 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:40.036346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ccd4104-07f0-4ce6-9998-1f48ce223ac3 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:40.036815 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-1ccd4104-07f0-4ce6-9998-1f48ce223ac3 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:40.178851 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-1ccd4104-07f0-4ce6-9998-1f48ce223ac3 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:40.187857 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-1ccd4104-07f0-4ce6-9998-1f48ce223ac3 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1568 microversion: 2.1 time: 0.166672 Aug 28 01:44:40.188431 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1838/3447] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:40 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1568 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:41.210771 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cd4f6fab-2369-49ac-b91f-ddc3f4b316b1 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:41.211360 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cd4f6fab-2369-49ac-b91f-ddc3f4b316b1 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:41.218697 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cd4f6fab-2369-49ac-b91f-ddc3f4b316b1 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:41.219183 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cd4f6fab-2369-49ac-b91f-ddc3f4b316b1 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:41.219638 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cd4f6fab-2369-49ac-b91f-ddc3f4b316b1 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:41.220327 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cd4f6fab-2369-49ac-b91f-ddc3f4b316b1 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:41.220767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cd4f6fab-2369-49ac-b91f-ddc3f4b316b1 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:41.221222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cd4f6fab-2369-49ac-b91f-ddc3f4b316b1 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:41.369025 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-cd4f6fab-2369-49ac-b91f-ddc3f4b316b1 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:41.379222 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cd4f6fab-2369-49ac-b91f-ddc3f4b316b1 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1568 microversion: 2.1 time: 0.172498 Aug 28 01:44:41.380017 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1610/3448] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:41 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1568 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:42.400538 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-6bfe8402-3eff-4f54-a54a-e4a200f57c10 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:42.401200 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-6bfe8402-3eff-4f54-a54a-e4a200f57c10 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:42.408227 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bfe8402-3eff-4f54-a54a-e4a200f57c10 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:42.408734 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bfe8402-3eff-4f54-a54a-e4a200f57c10 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:42.409310 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bfe8402-3eff-4f54-a54a-e4a200f57c10 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:42.410168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bfe8402-3eff-4f54-a54a-e4a200f57c10 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:42.410347 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bfe8402-3eff-4f54-a54a-e4a200f57c10 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:42.410719 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-6bfe8402-3eff-4f54-a54a-e4a200f57c10 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:42.548176 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-6bfe8402-3eff-4f54-a54a-e4a200f57c10 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:42.558448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-6bfe8402-3eff-4f54-a54a-e4a200f57c10 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1568 microversion: 2.1 time: 0.161753 Aug 28 01:44:42.559091 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1839/3449] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:42 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1568 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:43.579996 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-bf122237-85b5-495e-9336-90b5cec353b8 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:43.580604 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-bf122237-85b5-495e-9336-90b5cec353b8 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:43.587933 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bf122237-85b5-495e-9336-90b5cec353b8 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:43.588412 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bf122237-85b5-495e-9336-90b5cec353b8 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:43.588839 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bf122237-85b5-495e-9336-90b5cec353b8 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:43.589553 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bf122237-85b5-495e-9336-90b5cec353b8 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:43.589964 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bf122237-85b5-495e-9336-90b5cec353b8 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:43.590379 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-bf122237-85b5-495e-9336-90b5cec353b8 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:43.736098 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-bf122237-85b5-495e-9336-90b5cec353b8 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:43.745624 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-bf122237-85b5-495e-9336-90b5cec353b8 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1568 microversion: 2.1 time: 0.169557 Aug 28 01:44:43.746201 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1611/3450] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:43 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1568 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:44.777123 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-214d23b2-272b-41a8-b8e1-72d4adc2458f tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:44.777787 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-214d23b2-272b-41a8-b8e1-72d4adc2458f tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:44.786183 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-214d23b2-272b-41a8-b8e1-72d4adc2458f tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:44.786661 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-214d23b2-272b-41a8-b8e1-72d4adc2458f tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:44.787087 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-214d23b2-272b-41a8-b8e1-72d4adc2458f tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:44.787770 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-214d23b2-272b-41a8-b8e1-72d4adc2458f tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:44.788214 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-214d23b2-272b-41a8-b8e1-72d4adc2458f tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:44.788625 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-214d23b2-272b-41a8-b8e1-72d4adc2458f tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:44.925571 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-214d23b2-272b-41a8-b8e1-72d4adc2458f tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:44.934105 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-214d23b2-272b-41a8-b8e1-72d4adc2458f tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1568 microversion: 2.1 time: 0.162666 Aug 28 01:44:44.934649 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1840/3451] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:44 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1568 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:45.958730 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-dcf7b78b-9dbb-491b-807d-b34d06b1e0bb tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:45.959281 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-dcf7b78b-9dbb-491b-807d-b34d06b1e0bb tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:45.966143 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dcf7b78b-9dbb-491b-807d-b34d06b1e0bb tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:45.966628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dcf7b78b-9dbb-491b-807d-b34d06b1e0bb tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:45.967063 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dcf7b78b-9dbb-491b-807d-b34d06b1e0bb tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:45.967749 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dcf7b78b-9dbb-491b-807d-b34d06b1e0bb tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:45.968197 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dcf7b78b-9dbb-491b-807d-b34d06b1e0bb tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:45.968620 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-dcf7b78b-9dbb-491b-807d-b34d06b1e0bb tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:46.121676 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-dcf7b78b-9dbb-491b-807d-b34d06b1e0bb tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:46.129739 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-dcf7b78b-9dbb-491b-807d-b34d06b1e0bb tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1569 microversion: 2.1 time: 0.175462 Aug 28 01:44:46.130421 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1612/3452] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:45 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1569 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:46.150125 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-48a80b66-3123-45da-90a8-c8e9806f7e25 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:44:46.150694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-48a80b66-3123-45da-90a8-c8e9806f7e25 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:46.157549 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48a80b66-3123-45da-90a8-c8e9806f7e25 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:46.158030 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48a80b66-3123-45da-90a8-c8e9806f7e25 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:46.158456 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48a80b66-3123-45da-90a8-c8e9806f7e25 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:46.159145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48a80b66-3123-45da-90a8-c8e9806f7e25 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:46.159608 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48a80b66-3123-45da-90a8-c8e9806f7e25 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:46.160000 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-48a80b66-3123-45da-90a8-c8e9806f7e25 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:46.206145 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-48a80b66-3123-45da-90a8-c8e9806f7e25 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120525) unshelve /opt/stack/nova/nova/compute/api.py:4591}} Aug 28 01:44:46.297272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-48a80b66-3123-45da-90a8-c8e9806f7e25 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "POST /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c/action" status: 202 len: 0 microversion: 2.1 time: 0.150987 Aug 28 01:44:46.300716 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1841/3453] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Aug 28 01:44:46 2022] POST /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c/action => generated 0 bytes in 153 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Aug 28 01:44:46.306282 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-8b581ad0-a5e3-4c64-8403-7c5049cbbf7b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:46.306972 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-8b581ad0-a5e3-4c64-8403-7c5049cbbf7b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:46.313482 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b581ad0-a5e3-4c64-8403-7c5049cbbf7b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:46.313964 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b581ad0-a5e3-4c64-8403-7c5049cbbf7b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:46.314738 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b581ad0-a5e3-4c64-8403-7c5049cbbf7b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:46.315420 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b581ad0-a5e3-4c64-8403-7c5049cbbf7b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:46.315858 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b581ad0-a5e3-4c64-8403-7c5049cbbf7b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:46.316271 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-8b581ad0-a5e3-4c64-8403-7c5049cbbf7b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:46.479110 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-8b581ad0-a5e3-4c64-8403-7c5049cbbf7b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:46.487932 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-8b581ad0-a5e3-4c64-8403-7c5049cbbf7b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1517 microversion: 2.1 time: 0.185697 Aug 28 01:44:46.488486 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1613/3454] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:46 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1517 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:47.509901 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-979b6479-26af-4b06-9631-47d9d87f0bf4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:47.510537 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-979b6479-26af-4b06-9631-47d9d87f0bf4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:47.517102 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-979b6479-26af-4b06-9631-47d9d87f0bf4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:47.517641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-979b6479-26af-4b06-9631-47d9d87f0bf4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:47.518190 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-979b6479-26af-4b06-9631-47d9d87f0bf4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:47.518945 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-979b6479-26af-4b06-9631-47d9d87f0bf4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:47.519430 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-979b6479-26af-4b06-9631-47d9d87f0bf4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:47.519886 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-979b6479-26af-4b06-9631-47d9d87f0bf4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:47.658148 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-979b6479-26af-4b06-9631-47d9d87f0bf4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:47.669541 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-979b6479-26af-4b06-9631-47d9d87f0bf4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1519 microversion: 2.1 time: 0.163800 Aug 28 01:44:47.670328 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1842/3455] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:47 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1519 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:48.575601 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-2ef3d79b-8766-49a8-a3a9-152c41ae1dc0 req-3f9fb8a6-6a9d-4c5f-b587-bd5ee6685b4c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c", "name": "network-vif-unplugged", "status": "completed", "tag": "3469e5fa-3521-4b70-8e0b-f9edded0bf75"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:44:48.587893 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-2ef3d79b-8766-49a8-a3a9-152c41ae1dc0 req-3f9fb8a6-6a9d-4c5f-b587-bd5ee6685b4c service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:48.588588 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-2ef3d79b-8766-49a8-a3a9-152c41ae1dc0 req-3f9fb8a6-6a9d-4c5f-b587-bd5ee6685b4c service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:48.589168 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-2ef3d79b-8766-49a8-a3a9-152c41ae1dc0 req-3f9fb8a6-6a9d-4c5f-b587-bd5ee6685b4c service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:48.605163 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-2ef3d79b-8766-49a8-a3a9-152c41ae1dc0 req-3f9fb8a6-6a9d-4c5f-b587-bd5ee6685b4c service nova] Creating event network-vif-unplugged:3469e5fa-3521-4b70-8e0b-f9edded0bf75 for instance ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c on hv-cinder-maste Aug 28 01:44:48.610949 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-2ef3d79b-8766-49a8-a3a9-152c41ae1dc0 req-3f9fb8a6-6a9d-4c5f-b587-bd5ee6685b4c service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.039572 Aug 28 01:44:48.611601 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1614/3456] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:44:48 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:44:48.693314 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-999e209f-12a3-4829-bce8-3fa8d239c3e0 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:48.693937 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-999e209f-12a3-4829-bce8-3fa8d239c3e0 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:48.700472 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-999e209f-12a3-4829-bce8-3fa8d239c3e0 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:48.700953 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-999e209f-12a3-4829-bce8-3fa8d239c3e0 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:48.701402 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-999e209f-12a3-4829-bce8-3fa8d239c3e0 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:48.702089 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-999e209f-12a3-4829-bce8-3fa8d239c3e0 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:48.702555 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-999e209f-12a3-4829-bce8-3fa8d239c3e0 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:48.702972 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-999e209f-12a3-4829-bce8-3fa8d239c3e0 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:48.920237 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-999e209f-12a3-4829-bce8-3fa8d239c3e0 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:48.931443 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-999e209f-12a3-4829-bce8-3fa8d239c3e0 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1575 microversion: 2.1 time: 0.242409 Aug 28 01:44:48.932023 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1843/3457] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:48 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1575 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:49.955494 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-5fda7b10-205d-4ec3-9eae-62791f01c560 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:49.956242 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-5fda7b10-205d-4ec3-9eae-62791f01c560 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:49.962893 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5fda7b10-205d-4ec3-9eae-62791f01c560 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:49.963355 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5fda7b10-205d-4ec3-9eae-62791f01c560 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:49.963760 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5fda7b10-205d-4ec3-9eae-62791f01c560 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:49.964455 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5fda7b10-205d-4ec3-9eae-62791f01c560 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:49.964893 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5fda7b10-205d-4ec3-9eae-62791f01c560 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:49.965337 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-5fda7b10-205d-4ec3-9eae-62791f01c560 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:50.103750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-5fda7b10-205d-4ec3-9eae-62791f01c560 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:50.112256 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-5fda7b10-205d-4ec3-9eae-62791f01c560 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1575 microversion: 2.1 time: 0.161477 Aug 28 01:44:50.112809 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1615/3458] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:49 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1575 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:51.136582 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-704b63d1-8fa4-4eb6-b743-be7d3c4df70b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:51.137194 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-704b63d1-8fa4-4eb6-b743-be7d3c4df70b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:51.144069 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-704b63d1-8fa4-4eb6-b743-be7d3c4df70b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:51.144547 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-704b63d1-8fa4-4eb6-b743-be7d3c4df70b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:51.145001 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-704b63d1-8fa4-4eb6-b743-be7d3c4df70b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:51.145694 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-704b63d1-8fa4-4eb6-b743-be7d3c4df70b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:51.146209 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-704b63d1-8fa4-4eb6-b743-be7d3c4df70b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:51.146646 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-704b63d1-8fa4-4eb6-b743-be7d3c4df70b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:51.300773 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-704b63d1-8fa4-4eb6-b743-be7d3c4df70b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:51.308842 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-704b63d1-8fa4-4eb6-b743-be7d3c4df70b tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1575 microversion: 2.1 time: 0.176198 Aug 28 01:44:51.309471 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1844/3459] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:51 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1575 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:52.338105 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-cfd8842c-9390-4fb2-8f66-0301df542918 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:52.338698 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-cfd8842c-9390-4fb2-8f66-0301df542918 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:52.345723 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cfd8842c-9390-4fb2-8f66-0301df542918 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:52.346205 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cfd8842c-9390-4fb2-8f66-0301df542918 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:52.346691 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cfd8842c-9390-4fb2-8f66-0301df542918 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:52.347379 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cfd8842c-9390-4fb2-8f66-0301df542918 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:52.347835 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cfd8842c-9390-4fb2-8f66-0301df542918 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:52.348261 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-cfd8842c-9390-4fb2-8f66-0301df542918 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:52.485609 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-cfd8842c-9390-4fb2-8f66-0301df542918 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:52.493872 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-cfd8842c-9390-4fb2-8f66-0301df542918 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1575 microversion: 2.1 time: 0.159916 Aug 28 01:44:52.494621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1616/3460] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:52 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1575 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:53.515924 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-8e126d0b-44cb-4f9f-90b1-a111eaa129a2 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:53.516515 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-8e126d0b-44cb-4f9f-90b1-a111eaa129a2 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:53.523448 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e126d0b-44cb-4f9f-90b1-a111eaa129a2 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:53.523930 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e126d0b-44cb-4f9f-90b1-a111eaa129a2 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:53.524357 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e126d0b-44cb-4f9f-90b1-a111eaa129a2 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:53.525080 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e126d0b-44cb-4f9f-90b1-a111eaa129a2 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:53.525528 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e126d0b-44cb-4f9f-90b1-a111eaa129a2 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:53.525946 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-8e126d0b-44cb-4f9f-90b1-a111eaa129a2 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:53.685892 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-8e126d0b-44cb-4f9f-90b1-a111eaa129a2 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:53.697052 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-8e126d0b-44cb-4f9f-90b1-a111eaa129a2 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1575 microversion: 2.1 time: 0.185026 Aug 28 01:44:53.697759 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1845/3461] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:53 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1575 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:54.721027 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-b32a3b24-7c53-41b5-b0ee-960325f2b74c tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:54.721705 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-b32a3b24-7c53-41b5-b0ee-960325f2b74c tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:54.729922 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b32a3b24-7c53-41b5-b0ee-960325f2b74c tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:54.730511 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b32a3b24-7c53-41b5-b0ee-960325f2b74c tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:54.730951 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b32a3b24-7c53-41b5-b0ee-960325f2b74c tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:54.731642 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b32a3b24-7c53-41b5-b0ee-960325f2b74c tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:54.732063 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b32a3b24-7c53-41b5-b0ee-960325f2b74c tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:54.732508 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-b32a3b24-7c53-41b5-b0ee-960325f2b74c tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:54.883723 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-b32a3b24-7c53-41b5-b0ee-960325f2b74c tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:54.893977 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-b32a3b24-7c53-41b5-b0ee-960325f2b74c tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1575 microversion: 2.1 time: 0.179085 Aug 28 01:44:54.894552 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1617/3462] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:54 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1575 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:55.915230 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-81f4846c-ee23-4555-979b-391cd783acc9 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:55.915950 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-81f4846c-ee23-4555-979b-391cd783acc9 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:55.924131 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81f4846c-ee23-4555-979b-391cd783acc9 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:55.924612 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81f4846c-ee23-4555-979b-391cd783acc9 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:55.925073 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81f4846c-ee23-4555-979b-391cd783acc9 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:55.925766 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81f4846c-ee23-4555-979b-391cd783acc9 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:55.926211 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81f4846c-ee23-4555-979b-391cd783acc9 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:55.926640 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-81f4846c-ee23-4555-979b-391cd783acc9 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:56.061088 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-81f4846c-ee23-4555-979b-391cd783acc9 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:56.071980 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-81f4846c-ee23-4555-979b-391cd783acc9 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1575 microversion: 2.1 time: 0.161026 Aug 28 01:44:56.072545 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1846/3463] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:55 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1575 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:57.093417 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f0c7301f-4f84-44d3-acee-523ac7615205 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:57.094025 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f0c7301f-4f84-44d3-acee-523ac7615205 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:57.100628 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0c7301f-4f84-44d3-acee-523ac7615205 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:57.101131 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0c7301f-4f84-44d3-acee-523ac7615205 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:57.101607 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0c7301f-4f84-44d3-acee-523ac7615205 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:57.102320 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0c7301f-4f84-44d3-acee-523ac7615205 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:57.102768 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0c7301f-4f84-44d3-acee-523ac7615205 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:57.103194 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f0c7301f-4f84-44d3-acee-523ac7615205 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:57.241251 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-f0c7301f-4f84-44d3-acee-523ac7615205 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:57.250565 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f0c7301f-4f84-44d3-acee-523ac7615205 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1575 microversion: 2.1 time: 0.161184 Aug 28 01:44:57.251150 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1618/3464] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:57 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1575 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:58.274124 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-fbf4fc1b-4d07-4556-a449-dd6d1ed25007 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:58.274730 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-fbf4fc1b-4d07-4556-a449-dd6d1ed25007 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:58.281665 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fbf4fc1b-4d07-4556-a449-dd6d1ed25007 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:58.282140 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fbf4fc1b-4d07-4556-a449-dd6d1ed25007 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:58.282569 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fbf4fc1b-4d07-4556-a449-dd6d1ed25007 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:58.283223 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fbf4fc1b-4d07-4556-a449-dd6d1ed25007 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:58.283726 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fbf4fc1b-4d07-4556-a449-dd6d1ed25007 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:58.284141 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-fbf4fc1b-4d07-4556-a449-dd6d1ed25007 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:58.429287 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-fbf4fc1b-4d07-4556-a449-dd6d1ed25007 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:58.437112 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-fbf4fc1b-4d07-4556-a449-dd6d1ed25007 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1575 microversion: 2.1 time: 0.166928 Aug 28 01:44:58.437683 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1847/3465] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:58 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1575 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:44:59.459189 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-02a45653-9f08-4428-90e8-0cd104a5df21 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:44:59.459882 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-02a45653-9f08-4428-90e8-0cd104a5df21 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:44:59.466801 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-02a45653-9f08-4428-90e8-0cd104a5df21 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:59.467314 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-02a45653-9f08-4428-90e8-0cd104a5df21 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:59.467746 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-02a45653-9f08-4428-90e8-0cd104a5df21 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:59.468430 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-02a45653-9f08-4428-90e8-0cd104a5df21 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:44:59.468881 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-02a45653-9f08-4428-90e8-0cd104a5df21 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:44:59.469338 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-02a45653-9f08-4428-90e8-0cd104a5df21 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:44:59.606402 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-02a45653-9f08-4428-90e8-0cd104a5df21 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:44:59.617221 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-02a45653-9f08-4428-90e8-0cd104a5df21 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1575 microversion: 2.1 time: 0.162158 Aug 28 01:44:59.617802 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1619/3466] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:44:59 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1575 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:45:00.637997 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-91cc7bb3-418b-4390-a86e-6bc601c99359 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:45:00.638628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-91cc7bb3-418b-4390-a86e-6bc601c99359 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:45:00.645519 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91cc7bb3-418b-4390-a86e-6bc601c99359 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:45:00.645994 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91cc7bb3-418b-4390-a86e-6bc601c99359 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:45:00.646424 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91cc7bb3-418b-4390-a86e-6bc601c99359 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:45:00.647107 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91cc7bb3-418b-4390-a86e-6bc601c99359 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:45:00.647559 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91cc7bb3-418b-4390-a86e-6bc601c99359 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:45:00.647973 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-91cc7bb3-418b-4390-a86e-6bc601c99359 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:45:00.697489 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-0c5afe85-1a32-48fb-98d1-e42caea20fc7 req-4cfa1678-2baf-4b24-9564-4f2fe7c9438d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c", "name": "network-vif-plugged", "status": "completed", "tag": "3469e5fa-3521-4b70-8e0b-f9edded0bf75"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:45:00.707313 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-0c5afe85-1a32-48fb-98d1-e42caea20fc7 req-4cfa1678-2baf-4b24-9564-4f2fe7c9438d service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:45:00.707784 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-0c5afe85-1a32-48fb-98d1-e42caea20fc7 req-4cfa1678-2baf-4b24-9564-4f2fe7c9438d service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:45:00.708266 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-0c5afe85-1a32-48fb-98d1-e42caea20fc7 req-4cfa1678-2baf-4b24-9564-4f2fe7c9438d service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:45:00.723299 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-0c5afe85-1a32-48fb-98d1-e42caea20fc7 req-4cfa1678-2baf-4b24-9564-4f2fe7c9438d service nova] Creating event network-vif-plugged:3469e5fa-3521-4b70-8e0b-f9edded0bf75 for instance ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c on hv-cinder-maste Aug 28 01:45:00.729301 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-0c5afe85-1a32-48fb-98d1-e42caea20fc7 req-4cfa1678-2baf-4b24-9564-4f2fe7c9438d service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036178 Aug 28 01:45:00.730308 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1620/3467] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:45:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:45:00.791564 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-91cc7bb3-418b-4390-a86e-6bc601c99359 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:45:00.800491 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-91cc7bb3-418b-4390-a86e-6bc601c99359 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1575 microversion: 2.1 time: 0.166373 Aug 28 01:45:00.801095 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1848/3468] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:45:00 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1575 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:45:01.822461 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-ef0e3e96-09c5-440e-b208-d8826f87434d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:45:01.823120 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-ef0e3e96-09c5-440e-b208-d8826f87434d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:45:01.829946 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ef0e3e96-09c5-440e-b208-d8826f87434d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:45:01.830443 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ef0e3e96-09c5-440e-b208-d8826f87434d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:45:01.830900 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ef0e3e96-09c5-440e-b208-d8826f87434d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:45:01.831621 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ef0e3e96-09c5-440e-b208-d8826f87434d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:45:01.832063 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ef0e3e96-09c5-440e-b208-d8826f87434d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:45:01.832464 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-ef0e3e96-09c5-440e-b208-d8826f87434d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:45:01.975920 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-ef0e3e96-09c5-440e-b208-d8826f87434d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:45:01.979584 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:45:01.984569 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-ef0e3e96-09c5-440e-b208-d8826f87434d tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1575 microversion: 2.1 time: 0.166803 Aug 28 01:45:01.985166 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1621/3469] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:45:01 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1575 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:45:03.007556 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-c6077838-ee2b-4669-9585-ae24acf9be25 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:45:03.008319 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-c6077838-ee2b-4669-9585-ae24acf9be25 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:45:03.011417 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=33,nova_api:UPDATE=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:45:03.019504 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6077838-ee2b-4669-9585-ae24acf9be25 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:45:03.020361 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6077838-ee2b-4669-9585-ae24acf9be25 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:45:03.020952 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6077838-ee2b-4669-9585-ae24acf9be25 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:45:03.021945 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6077838-ee2b-4669-9585-ae24acf9be25 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:45:03.022543 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6077838-ee2b-4669-9585-ae24acf9be25 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:45:03.023093 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-c6077838-ee2b-4669-9585-ae24acf9be25 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:45:03.027024 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=89,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:45:03.169877 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-c6077838-ee2b-4669-9585-ae24acf9be25 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:45:03.177914 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-c6077838-ee2b-4669-9585-ae24acf9be25 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1562 microversion: 2.1 time: 0.175308 Aug 28 01:45:03.178489 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1849/3470] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:45:03 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1562 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:46:33.467518 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Aug 28 01:46:33.468197 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: Sun Aug 28 01:46:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.42) !!! Aug 28 01:46:33.468920 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-9e4e8970-b292-4a51-bcc8-97001bde53da tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:46:33.469446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9e4e8970-b292-4a51-bcc8-97001bde53da tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:46:33.469705 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_api:SELECT=1 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:46:33.470813 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG dbcounter [-] [120525] Writing DB stats nova_cell1:SELECT=5 {{(pid=120525) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:46:33.473892 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG dbcounter [-] [120524] Writing DB stats nova_api:SELECT=32 {{(pid=120524) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 28 01:46:33.483051 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e4e8970-b292-4a51-bcc8-97001bde53da tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:46:33.483181 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e4e8970-b292-4a51-bcc8-97001bde53da tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:46:33.483835 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e4e8970-b292-4a51-bcc8-97001bde53da tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:46:33.484743 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e4e8970-b292-4a51-bcc8-97001bde53da tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:46:33.486535 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e4e8970-b292-4a51-bcc8-97001bde53da tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:46:33.486973 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-9e4e8970-b292-4a51-bcc8-97001bde53da tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:46:33.513814 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-9e4e8970-b292-4a51-bcc8-97001bde53da tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Going to try to terminate instance {{(pid=120524) delete /opt/stack/nova/nova/compute/api.py:2719}} Aug 28 01:46:33.581327 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [req-c4d23ed7-814c-4748-9811-20b8aab4ec4f req-59b6b7a0-af93-4c8a-92ba-3cc37837d1ba service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c", "tag": "3469e5fa-3521-4b70-8e0b-f9edded0bf75"}]} {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:46:33.591602 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-c4d23ed7-814c-4748-9811-20b8aab4ec4f req-59b6b7a0-af93-4c8a-92ba-3cc37837d1ba service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:46:33.592263 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-c4d23ed7-814c-4748-9811-20b8aab4ec4f req-59b6b7a0-af93-4c8a-92ba-3cc37837d1ba service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:46:33.592721 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [req-c4d23ed7-814c-4748-9811-20b8aab4ec4f req-59b6b7a0-af93-4c8a-92ba-3cc37837d1ba service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:46:33.600552 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-9e4e8970-b292-4a51-bcc8-97001bde53da tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "DELETE /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 204 len: 0 microversion: 2.1 time: 0.137739 Aug 28 01:46:33.601573 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1622/3471] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Aug 28 01:46:33 2022] DELETE /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Aug 28 01:46:33.606077 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.compute.server_external_events [req-c4d23ed7-814c-4748-9811-20b8aab4ec4f req-59b6b7a0-af93-4c8a-92ba-3cc37837d1ba service nova] Creating event network-changed:3469e5fa-3521-4b70-8e0b-f9edded0bf75 for instance ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c on hv-cinder-maste Aug 28 01:46:33.610901 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-678bd460-6cc0-4102-8d01-5fd5e600e582 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:46:33.611912 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-678bd460-6cc0-4102-8d01-5fd5e600e582 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:46:33.614808 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [req-c4d23ed7-814c-4748-9811-20b8aab4ec4f req-59b6b7a0-af93-4c8a-92ba-3cc37837d1ba service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.148968 Aug 28 01:46:33.615641 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1850/3472] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:46:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:46:33.617912 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-678bd460-6cc0-4102-8d01-5fd5e600e582 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:46:33.618326 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-678bd460-6cc0-4102-8d01-5fd5e600e582 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:46:33.618703 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-678bd460-6cc0-4102-8d01-5fd5e600e582 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:46:33.619408 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-678bd460-6cc0-4102-8d01-5fd5e600e582 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:46:33.619867 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-678bd460-6cc0-4102-8d01-5fd5e600e582 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:46:33.620283 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-678bd460-6cc0-4102-8d01-5fd5e600e582 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:46:33.813442 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.policy [None req-678bd460-6cc0-4102-8d01-5fd5e600e582 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120524) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:46:33.822020 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-678bd460-6cc0-4102-8d01-5fd5e600e582 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1568 microversion: 2.1 time: 0.215884 Aug 28 01:46:33.822682 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1623/3473] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:46:33 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1568 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:46:34.845080 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-dc4a66ca-782e-4445-bd9c-18adbabfdfb3 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:46:34.845785 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-dc4a66ca-782e-4445-bd9c-18adbabfdfb3 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:46:34.854639 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc4a66ca-782e-4445-bd9c-18adbabfdfb3 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:46:34.855325 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc4a66ca-782e-4445-bd9c-18adbabfdfb3 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:46:34.855983 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc4a66ca-782e-4445-bd9c-18adbabfdfb3 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:46:34.856990 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc4a66ca-782e-4445-bd9c-18adbabfdfb3 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:46:34.857628 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc4a66ca-782e-4445-bd9c-18adbabfdfb3 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:46:34.858272 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-dc4a66ca-782e-4445-bd9c-18adbabfdfb3 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:46:35.052150 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-dc4a66ca-782e-4445-bd9c-18adbabfdfb3 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:46:35.062434 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-dc4a66ca-782e-4445-bd9c-18adbabfdfb3 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1568 microversion: 2.1 time: 0.222031 Aug 28 01:46:35.063187 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1851/3474] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:46:34 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1568 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Aug 28 01:46:36.074877 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [req-2b83b7cf-554b-4055-ab0e-ed73ee57e96a req-ae398634-8d10-4eb7-9676-cbd9f67aff02 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c", "name": "network-vif-deleted", "tag": "3469e5fa-3521-4b70-8e0b-f9edded0bf75"}]} {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Aug 28 01:46:36.085168 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-59e6d130-cd56-4bbc-910a-d86f99afff7e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:46:36.085784 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.compute.api [None req-59e6d130-cd56-4bbc-910a-d86f99afff7e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120525) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:46:36.088076 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-2b83b7cf-554b-4055-ab0e-ed73ee57e96a req-ae398634-8d10-4eb7-9676-cbd9f67aff02 service nova] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:46:36.088589 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-2b83b7cf-554b-4055-ab0e-ed73ee57e96a req-ae398634-8d10-4eb7-9676-cbd9f67aff02 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:46:36.089109 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [req-2b83b7cf-554b-4055-ab0e-ed73ee57e96a req-ae398634-8d10-4eb7-9676-cbd9f67aff02 service nova] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:46:36.092754 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-59e6d130-cd56-4bbc-910a-d86f99afff7e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:46:36.093380 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-59e6d130-cd56-4bbc-910a-d86f99afff7e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:46:36.093906 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-59e6d130-cd56-4bbc-910a-d86f99afff7e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:46:36.094704 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-59e6d130-cd56-4bbc-910a-d86f99afff7e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:46:36.095244 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-59e6d130-cd56-4bbc-910a-d86f99afff7e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:46:36.095754 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG oslo_concurrency.lockutils [None req-59e6d130-cd56-4bbc-910a-d86f99afff7e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120525) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:46:36.120457 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.compute.server_external_events [req-2b83b7cf-554b-4055-ab0e-ed73ee57e96a req-ae398634-8d10-4eb7-9676-cbd9f67aff02 service nova] Creating event network-vif-deleted:3469e5fa-3521-4b70-8e0b-f9edded0bf75 for instance ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c on hv-cinder-maste Aug 28 01:46:36.126288 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [req-2b83b7cf-554b-4055-ab0e-ed73ee57e96a req-ae398634-8d10-4eb7-9676-cbd9f67aff02 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055120 Aug 28 01:46:36.127245 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1624/3475] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Aug 28 01:46:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Aug 28 01:46:36.175647 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.policy [None req-59e6d130-cd56-4bbc-910a-d86f99afff7e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '41a51cf16450452194671aa46f4f06c4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4ead68269cb4e19befec4146c93686d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120525) authorize /opt/stack/nova/nova/policy.py:199}} Aug 28 01:46:36.183816 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-59e6d130-cd56-4bbc-910a-d86f99afff7e tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 200 len: 1381 microversion: 2.1 time: 0.103586 Aug 28 01:46:36.184375 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1852/3476] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:46:36 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 1381 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Aug 28 01:46:37.207032 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f1371bfb-3f7d-42cf-9c1a-759b11c8a2a4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '>' {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:46:37.207743 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.compute.api [None req-f1371bfb-3f7d-42cf-9c1a-759b11c8a2a4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] [instance: ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c] Fetching instance by UUID {{(pid=120524) get /opt/stack/nova/nova/compute/api.py:2975}} Aug 28 01:46:37.215710 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1371bfb-3f7d-42cf-9c1a-759b11c8a2a4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:46:37.216186 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1371bfb-3f7d-42cf-9c1a-759b11c8a2a4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:46:37.216611 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1371bfb-3f7d-42cf-9c1a-759b11c8a2a4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:46:37.217326 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1371bfb-3f7d-42cf-9c1a-759b11c8a2a4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Acquiring lock "0cad982d-83ea-40cf-82f0-5c4394803725" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Aug 28 01:46:37.217767 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1371bfb-3f7d-42cf-9c1a-759b11c8a2a4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Aug 28 01:46:37.218178 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG oslo_concurrency.lockutils [None req-f1371bfb-3f7d-42cf-9c1a-759b11c8a2a4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Lock "0cad982d-83ea-40cf-82f0-5c4394803725" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120524) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Aug 28 01:46:37.231739 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.wsgi [None req-f1371bfb-3f7d-42cf-9c1a-759b11c8a2a4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] HTTP exception thrown: Instance ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c could not be found. Aug 28 01:46:37.233446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f1371bfb-3f7d-42cf-9c1a-759b11c8a2a4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Request method failure captured: Aug 28 01:46:37.233446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: request: GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c HTTP/1.1 Aug 28 01:46:37.233446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept: application/json Aug 28 01:46:37.233446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Accept-Encoding: identity Aug 28 01:46:37.233446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Connection: close Aug 28 01:46:37.233446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Length: 0 Aug 28 01:46:37.233446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Content-Type: application/json Aug 28 01:46:37.233446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Host: 10.222.0.42 Aug 28 01:46:37.233446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: Openstack-System-Scope: None Aug 28 01:46:37.233446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: User-Agent: python-urllib3/1.26.11 Aug 28 01:46:37.233446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Auth-Token: gAAAAABjCsfHJHQHuF3_BrGGfozVQ4qGN9lk9q3k9IZTtxClBinHuXvbqpV2TlfzAdxTkl3y8EzcqA_loVf2jE45fLm84jVZkQ3DvwFaGuN5DXAblC1QpyLH6lDSO2hiGbjM_OAl5golYs-RJPbgxzYKwfecRhlNeH7aD41h_Wa0h928anATTSs Aug 28 01:46:37.233446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Id: None Aug 28 01:46:37.233446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Domain-Name: None Aug 28 01:46:37.233446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Identity-Status: Confirmed Aug 28 01:46:37.233446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Is-Admin-Project: True Aug 28 01:46:37.233446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Id: default Aug 28 01:46:37.233446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Domain-Name: Default Aug 28 01:46:37.233446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Id: b4ead68269cb4e19befec4146c93686d Aug 28 01:46:37.233446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Project-Name: tempest-TestShelveInstance-1115579316 Aug 28 01:46:37.233446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Role: member,reader Aug 28 01:46:37.233446 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Roles: member,reader Aug 28 01:46:37.235043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_b4ead68269cb4e19befec4146c93686d", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b4ead68269cb4e19befec4146c93686d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/b4ead68269cb4e19befec4146c93686d"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/b4ead68269cb4e19befec4146c93686d"}]}] Aug 28 01:46:37.235043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant: tempest-TestShelveInstance-1115579316 Aug 28 01:46:37.235043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Id: b4ead68269cb4e19befec4146c93686d Aug 28 01:46:37.235043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-Tenant-Name: tempest-TestShelveInstance-1115579316 Aug 28 01:46:37.235043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User: tempest-TestShelveInstance-1115579316-project Aug 28 01:46:37.235043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Id: default Aug 28 01:46:37.235043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Domain-Name: Default Aug 28 01:46:37.235043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Id: 41a51cf16450452194671aa46f4f06c4 Aug 28 01:46:37.235043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: X-User-Name: tempest-TestShelveInstance-1115579316-project Aug 28 01:46:37.235043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: method: > Aug 28 01:46:37.235043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: exception: Instance ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c could not be found. Aug 28 01:46:37.235043 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: action_args: {'id': 'ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c'} Aug 28 01:46:37.236267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: {{(pid=120524) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Aug 28 01:46:37.236267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:46:37.236267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Aug 28 01:46:37.236267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Aug 28 01:46:37.236267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Aug 28 01:46:37.236267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Aug 28 01:46:37.236267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Aug 28 01:46:37.236267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Aug 28 01:46:37.236267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Aug 28 01:46:37.236267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Aug 28 01:46:37.236267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c could not be found. Aug 28 01:46:37.236267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:46:37.236267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:46:37.236267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:46:37.236267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:46:37.236267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:46:37.236267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:46:37.236267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Aug 28 01:46:37.236267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Aug 28 01:46:37.236267 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Aug 28 01:46:37.237750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Aug 28 01:46:37.237750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Aug 28 01:46:37.237750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = self._get_server( Aug 28 01:46:37.237750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Aug 28 01:46:37.237750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Aug 28 01:46:37.237750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Aug 28 01:46:37.237750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Aug 28 01:46:37.237750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c could not be found. Aug 28 01:46:37.237750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:46:37.237750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Aug 28 01:46:37.237750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:46:37.237750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Aug 28 01:46:37.237750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Aug 28 01:46:37.237750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Aug 28 01:46:37.237750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Aug 28 01:46:37.237750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Aug 28 01:46:37.237750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c could not be found. Aug 28 01:46:37.237750 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: ERROR nova.api.openstack.wsgi Aug 28 01:46:37.239155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: DEBUG nova.api.openstack.wsgi [None req-f1371bfb-3f7d-42cf-9c1a-759b11c8a2a4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Returning 404 to user: Instance ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c could not be found. {{(pid=120524) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Aug 28 01:46:37.239155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: INFO nova.api.openstack.requestlog [None req-f1371bfb-3f7d-42cf-9c1a-759b11c8a2a4 tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c" status: 404 len: 111 microversion: 2.1 time: 0.032156 Aug 28 01:46:37.239155 dv-cinder-master-3-iscsi devstack@n-api.service[120524]: [pid: 120524|app: 0|req: 1625/3477] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Aug 28 01:46:37 2022] GET /compute/v2.1/servers/ae4cfb9c-861d-4e55-8e58-c12c9ca7ca5c => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Aug 28 01:46:37.407346 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: DEBUG nova.api.openstack.wsgi [None req-a6dabe5c-d971-44db-aed1-13d2a9f54fbd tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] Calling method '.version_select at 0x7f0e71fd4d30>' {{(pid=120525) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Aug 28 01:46:37.435535 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: INFO nova.api.openstack.requestlog [None req-a6dabe5c-d971-44db-aed1-13d2a9f54fbd tempest-TestShelveInstance-1115579316 tempest-TestShelveInstance-1115579316-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1329873231" status: 202 len: 0 microversion: 2.1 time: 0.032301 Aug 28 01:46:37.436147 dv-cinder-master-3-iscsi devstack@n-api.service[120525]: [pid: 120525|app: 0|req: 1853/3478] 10.222.0.42 () {58 vars in 1254 bytes} [Sun Aug 28 01:46:37 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1329873231 => generated 0 bytes in 34 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0)